본문 바로가기

리눅스

우분투에서 Elasticsearch 8.x 설치 및 클러스터 구성 방법

728x90

우분투에서 Elasticsearch 8.x 설치 및 클러스터 구성 방법

운영 체제 환경 설정

호스트 파일에 노드 추가(/etc/hosts)

cat <<EOF >> /etc/hosts
### elasticsearch
192.168.56.71  node1
192.168.56.72  node2
192.168.56.73  node3
EOF

Elasticsearch 사용자의 ulimit을 설정

cat << EOF >> /etc/security/limits.conf

### elasticsearch
elasticsearch       hard    nofile   65535
elasticsearch       soft    nofile   65536
elasticsearch       hard    nproc    65536
elasticsearch       soft    nproc    65536
elasticsearch       hard    memlock  unlimited
elasticsearch       soft    memlock  unlimited
EOF

시스템의 커널 파라미터를 변경

  • 가상 메모리 지도의 최대 항목 수를 설정
echo "vm.max_map_count = 262144" | sudo tee -a /etc/sysctl.conf
  • 시스템이 스왑 파일을 사용하는 정책을 설정
echo "vm.swappiness = 1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

스왑 비활성화(disable swapping)

sudo swapoff -a
sudo sed -i '/swap/ s/^/#/' /etc/fstab

Elasticsearch 설치

sudo apt-get install -y apt-transport-https

Elasticsearch GPG 키 추가

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Elasticsearch 저장소 추가

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

APT 저장소 업데이트

sudo apt-get update

Elasticsearch 패키지 설치

sudo apt-get install -y elasticsearch

systemd 데몬 리로드

sudo systemctl daemon-reload

Elasticsearch 서비스 시작 및 활성화

sudo systemctl --now enable elasticsearch.service

Elasticsearch cert 디렉토리 백업

cd /etc/elasticsearch
tar cfz elasticsearch_certs-${HOSTNAME}-$(date '+%Y%m%d_%H%M').tar.gz certs

Elasticsearch에서 TLS/SSL을 사용하기 위해 인증서 생성

  • 자체 서명된 루트 인증 기관(Certificate Authority, CA)를 생성
/usr/share/elasticsearch/bin/elasticsearch-certutil ca \
--out /etc/elasticsearch/certs/elastic-stack-ca.p12 \
--days 3650
  • 서버 및 클라이언트용 TLS/SSL 인증서를 생성
/usr/share/elasticsearch/bin/elasticsearch-certutil cert \
--ca /etc/elasticsearch/certs/elastic-stack-ca.p12 \
--out /etc/elasticsearch/certs/elastic-certificates.p12 \
--days 3650
  • 소유자 설정
chown root:elasticsearch /etc/elasticsearch/certs/elastic-certificates.p12
  • 권한 설정
chmod g+wr /etc/elasticsearch/certs/elastic-certificates.p12

node2, node3 인증서 파일 전송(보내기)

rsync -avz /etc/elasticsearch/certs root@192.168.56.72:/etc/elasticsearch/.
rsync -avz /etc/elasticsearch/certs root@192.168.56.73:/etc/elasticsearch/.
더보기

sshd 설정

---

sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
sudo systemctl restart sshd

---

keystore 설정

  • elasticsearch keystore 파일 생성
/usr/share/elasticsearch/bin/elasticsearch-keystore create
  • elasticsearch keystore 추가 Elasticsearch 노드 간의 TLS/SSL 보안 통신을 설정
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
  • HTTP 통신을 위한 TLS/SSL 보안 통신을 설정
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password
  • elasticsearch keystore 목록
/usr/share/elasticsearch/bin/elasticsearch-keystore list
  • http.p12의 비밀번호 확인
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password
  • transport.p12의 비밀번호 확인
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password

Elasticsearch 서비스의 systemd 유닛 파일 설정

  • elasticsearch.service 파일 편집
vim /usr/lib/systemd/system/elasticsearch.service
[Service]
LimitMEMLOCK=infinity
  • systemd 데몬 리로드
sudo systemctl daemon-reload
  • Elasticsearch 서비스 재시작
sudo systemctl restart elasticsearch.service

Elasticsearch 환경 설정

  • Elasticsearch 환경 설정 파일 편집(/etc/elasticsearch/elasticsearch.yml)
vim /etc/elasticsearch/elasticsearch.yml
cluster.name: my-cluster

node.name: node1

path.data: /var/lib/elasticsearch

path.logs: /var/log/elasticsearch

bootstrap.memory_lock: true

network.host: 192.168.56.71

http.port: 9200

discovery.seed_hosts: ["node1", "node2", "node3"]

cluster.initial_master_nodes: ["node1", "node2", "node3"]

xpack.security.enabled: true

xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

http.host: 0.0.0.0
  • Elasticsearch 서비스 시작/재시작/중지
sudo systemctl restart elasticsearch.service
sudo systemctl stop elasticsearch.service
  • Elasticsearch 서비스 상태 확인
sudo systemctl status elasticsearch

elastic 사용자 패스워드 재설정

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
$ /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
This tool will reset the password of the [elastic] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y


Password for the [elastic] user successfully reset.
New value: 768NrzAwNO2vFz-VcRGI

built-in superuser(elastic)의 비밀번호를 셸에 환경 변수로 저장

export ELASTIC_PASSWORD="768NrzAwNO2vFz-VcRGI"
echo $ELASTIC_PASSWORD

elasticsearch data 초기화

더보기

---

sudo systemctl stop elasticsearch.service
rm -rf /var/lib/elasticsearch
mkdir -p /var/lib/elasticsearch
chown elasticsearch.elasticsearch /var/lib/elasticsearch
sudo systemctl start elasticsearch.service

---

Elasticsearch API 사용

Elasticsearch API를 통해 클러스터 상태를 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200
$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200
{
  "name" : "node1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Jn0qAEnrTMyxK1G0XDvW1Q",
  "version" : {
    "number" : "8.12.1",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "6185ba65d27469afabc9bc951cded6c17c21e3f3",
    "build_date" : "2024-02-01T13:07:13.727175297Z",
    "build_snapshot" : false,
    "lucene_version" : "9.9.2",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Cluster Health 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/health?pretty
$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "my-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

클러스터 통계 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/stats?pretty

노드 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/nodes?v
$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/nodes?v
ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
192.168.56.72            8          97   7    0.46    0.52     0.41 cdfhilmrstw -      node2
192.168.56.73           26          89   9    0.77    0.58     0.39 cdfhilmrstw *      node3
192.168.56.71           13          75  10    0.49    0.58     0.45 cdfhilmrstw -      node1

인덱스 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/indices?pretty
728x90

Kibana 설치

Kibana 패키지 설치

sudo apt-get install -y kibana

Kibana에서 사용하는 암호화 키를 생성

  • Kibana의 설정 파일인 kibana.yml에 키를 추가
/usr/share/kibana/bin/kibana-encryption-keys generate
$ /usr/share/kibana/bin/kibana-encryption-keys generate
## Kibana Encryption Key Generation Utility

The 'generate' command guides you through the process of setting encryption keys for:

xpack.encryptedSavedObjects.encryptionKey
    Used to encrypt stored objects such as dashboards and visualizations
    https://www.elastic.co/guide/en/kibana/current/xpack-security-secure-saved-objects.html#xpack-security-secure-saved-objects

xpack.reporting.encryptionKey
    Used to encrypt saved reports
    https://www.elastic.co/guide/en/kibana/current/reporting-settings-kb.html#general-reporting-settings

xpack.security.encryptionKey
    Used to encrypt session information
    https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#security-session-and-cookie-settings


Already defined settings are ignored and can be regenerated using the --force flag.  Check the documentation links for instructions on how to rotate encryption keys.
Definitions should be set in the kibana.yml used configure Kibana.

Settings:
xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3
cat <<EOF >> /etc/kibana/kibana.yml

## Kibana Encryption Key Generation Utility
xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3
EOF

kibana_system의 비밀번호를 재설정

/usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system
$ /usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system
This tool will reset the password of the [kibana_system] user.
You will be prompted to enter the password.
Please confirm that you would like to continue [y/N]y


Enter password for [kibana_system]:
Re-enter password for [kibana_system]:
Password for the [kibana_system] user successfully reset.

Kibana에서 사용할 인증서를 생성

mkdir -p /etc/kibana/certs
cd /etc/kibana/certs
cp /etc/elasticsearch/certs/elastic-stack-ca.p12 /etc/kibana/certs/.
cp /etc/elasticsearch/certs/elastic-certificates.p12 /etc/kibana/certs/.
openssl pkcs12 -in elastic-certificates.p12 -cacerts -nokeys -out CA.pem
openssl pkcs12 -in elastic-certificates.p12 -nocerts -nodes -out client.key
openssl pkcs12 -in elastic-certificates.p12 -clcerts -nokeys -out client.crt
chown -R kibana.kibana /etc/kibana/certs

Kibana 설정

  • Kibana 환경 설정 파일 편집(/etc/kibana/kibana.yml)
vim /etc/kibana/kibana.yml
server.port: 5601

server.host: "192.168.10.73"

elasticsearch.hosts: ["https://node1:9200","https://node2:9200","https://node3:9200"]

elasticsearch.username: "kibana_system"
elasticsearch.password: "kibana_system_password"

elasticsearch.ssl.certificate: /etc/kibana/certs/client.crt
elasticsearch.ssl.key: /etc/kibana/certs/client.key

elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/CA.pem" ]

elasticsearch.ssl.verificationMode: certificate

logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file

pid.file: /run/kibana/kibana.pid

xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3

systemd 데몬 리로드

sudo systemctl daemon-reload

Kibana 서비스 시작 및 활성화

sudo systemctl --now enable kibana.service

Kibana 서비스 시작/재시작/중지

sudo systemctl restart kibana
sudo systemctl status kibana

Kibana 접속(웹 브라우저)

http://localhost:5601

Elasticsearch_Kibana

 

참고URL

- elasticsearch guide : Install Elasticsearch with Debian Package

- kibana guide : Install Kibana with Debian package

 

728x90