우분투에서 Elasticsearch 8.x 설치 및 클러스터 구성 방법

728x90

우분투에서 Elasticsearch 8.x 설치 및 클러스터 구성 방법

운영 체제 환경 설정

호스트 파일에 노드 추가(/etc/hosts)

cat <<EOF >> /etc/hosts
### elasticsearch
192.168.56.71  node1
192.168.56.72  node2
192.168.56.73  node3
EOF

Elasticsearch 사용자의 ulimit을 설정

cat << EOF >> /etc/security/limits.conf

### elasticsearch
elasticsearch       hard    nofile   65535
elasticsearch       soft    nofile   65536
elasticsearch       hard    nproc    65536
elasticsearch       soft    nproc    65536
elasticsearch       hard    memlock  unlimited
elasticsearch       soft    memlock  unlimited
EOF

시스템의 커널 파라미터를 변경

가상 메모리 지도의 최대 항목 수를 설정

echo "vm.max_map_count = 262144" | sudo tee -a /etc/sysctl.conf

시스템이 스왑 파일을 사용하는 정책을 설정

echo "vm.swappiness = 1" | sudo tee -a /etc/sysctl.conf

sudo sysctl -p

스왑 비활성화(disable swapping)

sudo swapoff -a

sudo sed -i '/swap/ s/^/#/' /etc/fstab

Elasticsearch 설치

sudo apt-get install -y apt-transport-https

Elasticsearch GPG 키 추가

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Elasticsearch 저장소 추가

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

APT 저장소 업데이트

sudo apt-get update

Elasticsearch 패키지 설치

sudo apt-get install -y elasticsearch

systemd 데몬 리로드

sudo systemctl daemon-reload

Elasticsearch 서비스 시작 및 활성화

sudo systemctl --now enable elasticsearch.service

Elasticsearch cert 디렉토리 백업

cd /etc/elasticsearch

tar cfz elasticsearch_certs-${HOSTNAME}-$(date '+%Y%m%d_%H%M').tar.gz certs

Elasticsearch에서 TLS/SSL을 사용하기 위해 인증서 생성

자체 서명된 루트 인증 기관(Certificate Authority, CA)를 생성

/usr/share/elasticsearch/bin/elasticsearch-certutil ca \
--out /etc/elasticsearch/certs/elastic-stack-ca.p12 \
--days 3650

서버 및 클라이언트용 TLS/SSL 인증서를 생성

/usr/share/elasticsearch/bin/elasticsearch-certutil cert \
--ca /etc/elasticsearch/certs/elastic-stack-ca.p12 \
--out /etc/elasticsearch/certs/elastic-certificates.p12 \
--days 3650

소유자 설정

chown root:elasticsearch /etc/elasticsearch/certs/elastic-certificates.p12

권한 설정

chmod g+wr /etc/elasticsearch/certs/elastic-certificates.p12

node2, node3 인증서 파일 전송(보내기)

rsync -avz /etc/elasticsearch/certs [email protected]:/etc/elasticsearch/.

rsync -avz /etc/elasticsearch/certs [email protected]:/etc/elasticsearch/.

sshd 설정

---

sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

sudo systemctl restart sshd

---

keystore 설정

elasticsearch keystore 파일 생성

/usr/share/elasticsearch/bin/elasticsearch-keystore create

elasticsearch keystore 추가 Elasticsearch 노드 간의 TLS/SSL 보안 통신을 설정

/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password

/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

HTTP 통신을 위한 TLS/SSL 보안 통신을 설정

/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password

/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password

elasticsearch keystore 목록

/usr/share/elasticsearch/bin/elasticsearch-keystore list

http.p12의 비밀번호 확인

/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password

transport.p12의 비밀번호 확인

/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password

Elasticsearch 서비스의 systemd 유닛 파일 설정

elasticsearch.service 파일 편집

vim /usr/lib/systemd/system/elasticsearch.service

[Service]
LimitMEMLOCK=infinity

systemd 데몬 리로드

sudo systemctl daemon-reload

Elasticsearch 서비스 재시작

sudo systemctl restart elasticsearch.service

Elasticsearch 환경 설정

Elasticsearch 환경 설정 파일 편집(/etc/elasticsearch/elasticsearch.yml)

vim /etc/elasticsearch/elasticsearch.yml

cluster.name: my-cluster

node.name: node1

path.data: /var/lib/elasticsearch

path.logs: /var/log/elasticsearch

bootstrap.memory_lock: true

network.host: 192.168.56.71

http.port: 9200

discovery.seed_hosts: ["node1", "node2", "node3"]

cluster.initial_master_nodes: ["node1", "node2", "node3"]

xpack.security.enabled: true

xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

http.host: 0.0.0.0

Elasticsearch 서비스 시작/재시작/중지

sudo systemctl restart elasticsearch.service

sudo systemctl stop elasticsearch.service

Elasticsearch 서비스 상태 확인

sudo systemctl status elasticsearch

elastic 사용자 패스워드 재설정

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

$ /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
This tool will reset the password of the [elastic] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y


Password for the [elastic] user successfully reset.
New value: 768NrzAwNO2vFz-VcRGI

built-in superuser(elastic)의 비밀번호를 셸에 환경 변수로 저장

export ELASTIC_PASSWORD="768NrzAwNO2vFz-VcRGI"

echo $ELASTIC_PASSWORD

elasticsearch data 초기화

---

sudo systemctl stop elasticsearch.service

rm -rf /var/lib/elasticsearch

mkdir -p /var/lib/elasticsearch

chown elasticsearch.elasticsearch /var/lib/elasticsearch

sudo systemctl start elasticsearch.service

---

Elasticsearch API 사용

Elasticsearch API를 통해 클러스터 상태를 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200

$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200
{
  "name" : "node1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Jn0qAEnrTMyxK1G0XDvW1Q",
  "version" : {
    "number" : "8.12.1",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "6185ba65d27469afabc9bc951cded6c17c21e3f3",
    "build_date" : "2024-02-01T13:07:13.727175297Z",
    "build_snapshot" : false,
    "lucene_version" : "9.9.2",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Cluster Health 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/health?pretty

$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "my-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

클러스터 통계 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/stats?pretty

노드 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/nodes?v

$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/nodes?v
ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
192.168.56.72            8          97   7    0.46    0.52     0.41 cdfhilmrstw -      node2
192.168.56.73           26          89   9    0.77    0.58     0.39 cdfhilmrstw *      node3
192.168.56.71           13          75  10    0.49    0.58     0.45 cdfhilmrstw -      node1

인덱스 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/indices?pretty

728x90

Kibana 설치

Kibana 패키지 설치

sudo apt-get install -y kibana

Kibana에서 사용하는 암호화 키를 생성

Kibana의 설정 파일인 kibana.yml에 키를 추가

/usr/share/kibana/bin/kibana-encryption-keys generate

$ /usr/share/kibana/bin/kibana-encryption-keys generate
## Kibana Encryption Key Generation Utility

The 'generate' command guides you through the process of setting encryption keys for:

xpack.encryptedSavedObjects.encryptionKey
    Used to encrypt stored objects such as dashboards and visualizations
    https://www.elastic.co/guide/en/kibana/current/xpack-security-secure-saved-objects.html#xpack-security-secure-saved-objects

xpack.reporting.encryptionKey
    Used to encrypt saved reports
    https://www.elastic.co/guide/en/kibana/current/reporting-settings-kb.html#general-reporting-settings

xpack.security.encryptionKey
    Used to encrypt session information
    https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#security-session-and-cookie-settings


Already defined settings are ignored and can be regenerated using the --force flag.  Check the documentation links for instructions on how to rotate encryption keys.
Definitions should be set in the kibana.yml used configure Kibana.

Settings:
xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3

cat <<EOF >> /etc/kibana/kibana.yml

## Kibana Encryption Key Generation Utility
xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3
EOF

kibana_system의 비밀번호를 재설정

/usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system

$ /usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system
This tool will reset the password of the [kibana_system] user.
You will be prompted to enter the password.
Please confirm that you would like to continue [y/N]y


Enter password for [kibana_system]:
Re-enter password for [kibana_system]:
Password for the [kibana_system] user successfully reset.

Kibana에서 사용할 인증서를 생성

mkdir -p /etc/kibana/certs

cd /etc/kibana/certs

cp /etc/elasticsearch/certs/elastic-stack-ca.p12 /etc/kibana/certs/.

cp /etc/elasticsearch/certs/elastic-certificates.p12 /etc/kibana/certs/.

openssl pkcs12 -in elastic-certificates.p12 -cacerts -nokeys -out CA.pem

openssl pkcs12 -in elastic-certificates.p12 -nocerts -nodes -out client.key

openssl pkcs12 -in elastic-certificates.p12 -clcerts -nokeys -out client.crt

chown -R kibana.kibana /etc/kibana/certs

Kibana 설정

Kibana 환경 설정 파일 편집(/etc/kibana/kibana.yml)

vim /etc/kibana/kibana.yml

server.port: 5601

server.host: "192.168.10.73"

elasticsearch.hosts: ["https://node1:9200","https://node2:9200","https://node3:9200"]

elasticsearch.username: "kibana_system"
elasticsearch.password: "kibana_system_password"

elasticsearch.ssl.certificate: /etc/kibana/certs/client.crt
elasticsearch.ssl.key: /etc/kibana/certs/client.key

elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/CA.pem" ]

elasticsearch.ssl.verificationMode: certificate

logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file

pid.file: /run/kibana/kibana.pid

xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3

systemd 데몬 리로드

sudo systemctl daemon-reload

Kibana 서비스 시작 및 활성화

sudo systemctl --now enable kibana.service

Kibana 서비스 시작/재시작/중지

sudo systemctl restart kibana

sudo systemctl status kibana

Kibana 접속(웹 브라우저)

http://localhost:5601

참고URL

- elasticsearch guide : Install Elasticsearch with Debian Package

- kibana guide : Install Kibana with Debian package

728x90

저작자표시 비영리

'리눅스' 카테고리의 다른 글

우분투에 qos mod를 설치하고 설정하는 방법(mod_qos) (0)	2024.02.26
우분투에 HAProxy를 설치하는 방법 (0)	2024.02.23
Kibana를 설치하고 구성하는 방법(SSL/TLS 암호화 구성) (0)	2024.02.22
Elasticsearch에서 SSL/TLS를 구성하는 방법 (0)	2024.02.22
우분투에서 Elasticsearch를 설치하고 Kibana를 사용하여 클러스터 상태를 확인하는 방법 (0)	2024.02.18

변군이글루 미러 사이트(Mirror Site)

우분투에서 Elasticsearch 8.x 설치 및 클러스터 구성 방법