[draft] 우분투에 Grafana Loki를 설치하고 구성하는 방법

우분투에 Grafana Loki를 설치하고 구성하는 방법

Grafana Loki는 로그 집계를 위한 오픈소스 시스템으로 Grafana와 함께 사용하면 로그 데이터를 시각화하고 모니터링할 수 있습니다.

Grafana Loki란?

Grafana Loki는 Grafana Labs에서 개발한 오픈소스 로그 수집 및 저장 시스템입니다. Prometheus의 로그 버전이라고 생각하면 쉽게 이해할 수 있으며 높은 확장성과 빠른 쿼리 성능을 제공합니다. 특히, 로그 데이터를 효율적으로 저장하고 검색하는 데 탁월하며 Grafana와의 통합을 통해 시각화까지 지원합니다.

Grafana APT 리포지토리 설치

1. 패키지 업데이트 및 요구 사항 설치

sudo apt update

sudo apt install -y apt-transport-https software-properties-common wget

GPG 키 가져오기

sudo mkdir -p /etc/apt/keyrings/

wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null

리포지토리 추가(stable releases)

echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

패키지 목록 업데이트

sudo apt update

1. Grafana 설치 및 구성

Grafana 설치

sudo apt-get install -y grafana

Grafana 서비스 시작 및 활성화

sudo systemctl start grafana-server

sudo systemctl enable grafana-server

2. Grafana Loki 설치

Loki 설치

sudo apt-get install -y loki

$ sudo apt-get install -y loki
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  loki
0 upgraded, 1 newly installed, 0 to remove and 69 not upgraded.
Need to get 22.6 MB of archives.
After this operation, 72.4 MB of additional disk space will be used.
Get:1 https://apt.grafana.com stable/main amd64 loki amd64 3.0.0 [22.6 MB]
Fetched 22.6 MB in 4s (6,096 kB/s)
Selecting previously unselected package loki.
(Reading database ... 84666 files and directories currently installed.)
Preparing to unpack .../archives/loki_3.0.0_amd64.deb ...
Unpacking loki (3.0.0) ...
Setting up loki (3.0.0) ...
 Post Install of a clean install
Adding system user `loki' (UID 116) ...
Adding new user `loki' (UID 116) with group `nogroup' ...
Creating home directory `/home/loki' ...
 Reload the service unit from disk
 Unmask the service
 Set the preset flag for the service unit
Created symlink /etc/systemd/system/multi-user.target.wants/loki.service → /etc/systemd/system/loki.service.
 Set the enabled flag for the service unit
Scanning processes...                                                                                                                           
Scanning linux images...                                                                                                                        

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

Loki 정보 확인

loki -version

$ loki -version
loki, version 3.0.0 (branch: release-3.0.x, revision: b4f7181)
  build user:       root@0abc9c324f52
  build date:       2024-04-08T18:33:19Z
  go version:       go1.21.9
  platform:         linux/amd64
  tags:             netgo

Loki 설정 파일

vim /etc/loki/config.yml

sudo tee /etc/loki/loki-config.yml > /dev/null <<EOF
auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0
  chunk_target_size: 1536000
  chunk_block_size: 262144

schema_config:
  configs:
  - from: 2020-10-24
    store: boltdb
    object_store: filesystem
    schema: v11
    index:
      prefix: index_
      period: 168h

storage_config:
  boltdb:
    directory: /tmp/loki/index

  filesystem:
    directory: /tmp/loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s
EOF

---

Loki 설정 파일

vim /etc/loki/config.yml

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  instance_addr: 127.0.0.1
  path_prefix: /tmp/loki
  storage:
    filesystem:
      chunks_directory: /tmp/loki/chunks
      rules_directory: /tmp/loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

query_range:
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        max_size_mb: 100

schema_config:
  configs:
    - from: 2020-10-24
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

ruler:
  alertmanager_url: http://localhost:9093

# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/analytics/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
#analytics:
#  reporting_enabled: false

Loki 서비스 파일

vim /etc/systemd/system/loki.service

[Unit]
Description=Loki service
After=network.target

[Service]
Type=simple
User=loki
ExecStart=/usr/bin/loki -config.file /etc/loki/config.yml
# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec = 120
Restart = on-failure
RestartSec = 2

[Install]
WantedBy=multi-user.targetroot

---

Loki 서비스 시작 및 활성화

sudo systemctl daemon-reload

sudo systemctl start loki

sudo systemctl enable loki

Loki 실행

curl localhost:3100/metrics

3. Grafana 설정(Loki 데이터 소스 추가)

1. 웹 브라우저를 열고 http://localhost:3000으로 이동하여 Grafana에 로그인합니다.

• 기본 사용자 이름과 비밀번호는 모두 admin입니다.

2. 왼쪽 사이드바에서 "Configuration" -> "Data Sources"로 이동합니다.

3. "Add data source" 버튼을 클릭하고 "Loki"를 선택합니다.

• URL: http://localhost:3100
• 기타 설정은 기본값으로 둡니다.

4. Save & Test" 버튼을 클릭하여 연결을 테스트하고 저장합니다.

4. Promtail 설치 및 구성

Promtail은 Loki로 로그를 전송하는 에이전트입니다.

Promtail 설치

sudo apt-get install -y promtail

$ sudo apt-get install -y promtail
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  promtail
0 upgraded, 1 newly installed, 0 to remove and 69 not upgraded.
Need to get 28.6 MB of archives.
After this operation, 96.7 MB of additional disk space will be used.
Get:1 https://apt.grafana.com stable/main amd64 promtail amd64 3.0.0 [28.6 MB]
Fetched 28.6 MB in 5s (5,807 kB/s)   
Selecting previously unselected package promtail.
(Reading database ... 84670 files and directories currently installed.)
Preparing to unpack .../promtail_3.0.0_amd64.deb ...
Unpacking promtail (3.0.0) ...
Setting up promtail (3.0.0) ...
 Post Install of a clean install
Adding system user `promtail' (UID 117) ...
Adding new user `promtail' (UID 117) with group `nogroup' ...
Not creating home directory `/home/promtail'.
 Reload the service unit from disk
 Unmask the service
 Set the preset flag for the service unit
Created symlink /etc/systemd/system/multi-user.target.wants/promtail.service → /etc/systemd/system/promtail.service.
 Set the enabled flag for the service unit
Scanning processes...                                                                                                                           
Scanning linux images...                                                                                                                        

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

Promtail 구성 파일

sudo tee /etc/promtail/config.yml > /dev/null <<EOF
server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log
EOF

---

Promtail 구성 파일

vim /etc/promtail/config.yml

# This minimal config scrape only single log file.
# Primarily used in rpm/deb packaging where promtail service can be started during system init process.
# And too much scraping during init process can overload the complete system.
# https://github.com/grafana/loki/issues/11398

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
- url: http://localhost:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      #NOTE: Need to be modified to scrape any additional logs of the system.
      __path__: /var/log/messages

Promtail 서비스 파일

vim /etc/systemd/system/promtail.service

[Unit]
Description=Promtail service
After=network.target

[Service]
Type=simple
User=promtail
ExecStart=/usr/bin/promtail -config.file /etc/promtail/config.yml
# Give a reasonable amount of time for promtail to start up/shut down
TimeoutSec = 60
Restart = on-failure
RestartSec = 2

[Install]
WantedBy=multi-user.target

---

Promtail 서비스 시작 및 활성화

sudo systemctl daemon-reload

sudo systemctl --now enable promtail

5. Grafana에서 대시보드 설정

1. Grafana에 로그인한 상태에서 왼쪽 사이드바에서 "Create" -> "Import"로 이동합니다.

2. Grafana Labs의 Loki 대시보드 ID를 입력하거나 JSON 파일을 업로드하여 대시보드를 가져옵니다. 예를 들어, "Logs / App" 대시보드 ID는 13639입니다.

3. Load"를 클릭하고 필요한 설정을 입력한 후 "Import"를 클릭합니다.

 

Grafana에서 Loki를 사용하여 로그 데이터를 시각화하고 모니터링할 수 있습니다.

 

참고URL

- Grafana Labs : Documentation : Install Grafana Loki locally

- Grafana Labs : Grafana dashboards

 

sudo 적절한 업데이트