본문 바로가기

리눅스

[draft] Redis Cluster를 설정하는 방법

반응형

Redis Cluster를 설정하는 방법(redis cluster setup)

테스트 환경

운영체제 정보

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Redis Cluster 구성도

Redis Cluster

Redis Cluster(3 Master, 3 Slave) 서버 현황

호스트명 IP node#1 node#2 비고
pnode1 192.168.0.51 6381 6382  
pnode2 192.168.0.74 6381 6382  
pnode3 192.168.0.75 6381 6382  

1. 기본 정보

Port : 6379 (Cluster에서는 6381, 6382 사용)

Config file : /etc/redis/redis-<port>.conf

Log file : /var/log/redis-<port>.log

Data dir : /var/lib/redis/<port>

Executable : /usr/local/bin/redis-server

CLI : /usr/local/bin/redis-cli

2. 커널 파라미터 및 ulimit 설정

커널 파라미터

sudo vim /etc/sysctl.conf
net.core.somaxconn=8192
net.ipv4.tcp_max_syn_backlog=8192
net.core.netdev_max_backlog=32768
vm.overcommit_memory=1

적용

sudo sysctl -p

ulimit 설정

sudo tee -a /etc/security/limits.conf <<'EOF'
*       soft    nproc   unlimited
*       hard    nproc   unlimited
*       soft    nofile  65536
*       hard    nofile  65536
EOF

3. Redis 디렉터리 준비

Config, Log, Data 디렉토리 생성

mkdir -pv /var/lib/redis/{6379,6380,6381,6382}
mkdir -pv /var/log/redis
mkdir -pv /etc/redis

설정 파일 복사

cp /usr/local/src/redis-6.2.7/redis.conf /etc/redis/redis-6381.conf
cp /usr/local/src/redis-6.2.7/redis.conf /etc/redis/redis-6382.conf

4. Redis 설정 파일

redis-6381.conf

sudo vim /etc/redis/redis-6381.conf
bind 0.0.0.0
port 6381
daemonize yes

tcp-backlog 512
timeout 0
tcp-keepalive 300

dir /var/lib/redis/6381

cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 3000

pidfile /var/run/redis_6381.pid
logfile /var/log/redis-6381.log

appendonly yes
appendfilename appendonly.aof
appendfsync everysec

redis-6382.conf

sudo vim /etc/redis/redis-6382.conf
bind 0.0.0.0
port 6382
daemonize yes

tcp-backlog 512
timeout 0
tcp-keepalive 300

dir /var/lib/redis/6382

cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 3000

pidfile /var/run/redis_6382.pid
logfile /var/log/redis-6382.log

appendonly yes
appendfilename appendonly.aof
appendfsync everysec
더보기

---

vim cluster-setup.sh
#!/bin/bash

mkdir cluster
cd cluster

# 포트를 다르게 설정하여 6개의 Redis 인스턴스를 생성합니다.
for port in {7000..7005}; do
    mkdir ${port}
    cat > ${port}/redis.conf <<EOF
port ${port}
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
EOF
done
bash +x cluster-setup.sh
vim cluster-instance-start.sh
#!/bin/bash

# 각 디렉토리로 이동하여 Redis 인스턴스를 실행합니다.
for port in {7000..7005}; do
    src/redis-server ${port}/redis.conf
done
bash -x cluster-instance-start.sh

---

5. Redis 서버 실행

/usr/local/bin/redis-server /etc/redis/redis-6381.conf
/usr/local/bin/redis-server /etc/redis/redis-6382.conf

로그 확인

cat /var/log/redis-6381.log
215410:C 26 Oct 2022 23:10:47.997 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
215410:C 26 Oct 2022 23:10:47.997 # Redis version=6.2.7, bits=64, commit=00000000, modified=0, pid=215410, just started
215410:C 26 Oct 2022 23:10:47.997 # Configuration loaded
215410:M 26 Oct 2022 23:10:47.999 * Increased maximum number of open files to 10032 (it was originally set to 1024).
215410:M 26 Oct 2022 23:10:47.999 * monotonic clock: POSIX clock_gettime
215410:M 26 Oct 2022 23:10:48.004 * No cluster configuration found, I'm f3ee1bbf6718735d1944846f449bbac8063029de
215410:M 26 Oct 2022 23:10:48.013 # A key '__redis__compare_helper' was added to Lua globals which is not on the globals allow list nor listed on the deny list.
215410:M 26 Oct 2022 23:10:48.013 * Running mode=cluster, port=6381.
215410:M 26 Oct 2022 23:10:48.013 # Server initialized
215410:M 26 Oct 2022 23:10:48.014 * Ready to accept connections

프로세스 확인

ps -ef | grep redis-server
root      215410       1  0 23:10 ?        00:00:00 /usr/local/bin/redis-server 0.0.0.0:6381 [cluster]
root      221396       1  0 23:13 ?        00:00:00 /usr/local/bin/redis-server 0.0.0.0:6382 [cluster]

포트 확인

netstat -nltp | grep redis
tcp        0      0 0.0.0.0:16382           0.0.0.0:*               LISTEN      221396/redis-server
tcp        0      0 0.0.0.0:16381           0.0.0.0:*               LISTEN      215410/redis-server
tcp        0      0 0.0.0.0:6382            0.0.0.0:*               LISTEN      221396/redis-server
tcp        0      0 0.0.0.0:6381            0.0.0.0:*               LISTEN      215410/redis-server

6. Redis Cluster 구축

(1) Master 노드 생성

pnode1(192.168.0.51)

redis-cli --cluster create \
  192.168.0.51:6381 \
  192.168.0.74:6381 \
  192.168.0.75:6381
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
M: f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381
   slots:[0-5460] (5461 slots) master
M: 041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381
   slots:[5461-10922] (5462 slots) master
M: 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381
   slots:[10923-16383] (5461 slots) master
  • yes 입력 → 3개의 Master 노드가 슬롯을 자동으로 분배합니다.
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.0.51:6381)
M: f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381
   slots:[0-5460] (5461 slots) master
M: 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381
   slots:[10923-16383] (5461 slots) master
M: 041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

(2) Slave 노드 추가

pnode(192.168.0.51)

redis-cli --cluster add-node 192.168.0.74:6382 192.168.0.51:6381 --cluster-slave \
  --cluster-master-id $(redis-cli -p 6381 cluster nodes | grep 192.168.0.51 | grep master | awk '{print $1}')
>>> Adding node 192.168.0.74:6382 to cluster 192.168.0.51:6381
>>> Performing Cluster Check (using node 192.168.0.51:6381)
M: f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381
   slots:[0-5460] (5461 slots) master
M: 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381
   slots:[10923-16383] (5461 slots) master
M: 041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381
   slots:[5461-10922] (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.74:6382 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 192.168.0.51:6381.
[OK] New node added correctly.

pnode2(192.168.0.74)

redis-cli --cluster add-node 192.168.0.75:6382 192.168.0.74:6381 --cluster-slave \
  --cluster-master-id $(redis-cli -p 6381 cluster nodes | grep 192.168.0.74 | grep master | awk '{print $1}')

pnode3(192.168.0.75)

redis-cli --cluster add-node 192.168.0.51:6382 192.168.0.75:6381 --cluster-slave \
  --cluster-master-id $(redis-cli -p 6381 cluster nodes | grep 192.168.0.75 | grep master | awk '{print $1}')

7. 클러스터 상태 확인

노드 정보

redis-cli -p 6381 cluster nodes
f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381@16381 myself,master - 0 1666794298000 1 connected 0-5460
f6b6efbb92768db1925682b48f45a80e02d5217c 192.168.0.51:6382@16382 slave 0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 0 1666794299000 3 connected
0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381@16381 master - 0 1666794300474 3 connected 10923-16383
c591902cc55d2fd74429745c3a1ac44352bbc679 192.168.0.74:6382@16382 slave f3ee1bbf6718735d1944846f449bbac8063029de 0 1666794299809 1 connected
041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381@16381 master - 0 1666794299305 2 connected 5461-10922
e8cfa56223ed8670d9a4757c876019c8b115b20f 192.168.0.75:6382@16382 slave 041c618f6e2e38d795ed47a38424650bfe170ae7 0 1666794299106 2 connected

cluster nodes

클러스터 상태

redis-cli -p 6381 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:3
cluster_my_epoch:1
cluster_stats_messages_ping_sent:1182
cluster_stats_messages_pong_sent:1155
cluster_stats_messages_sent:2337
cluster_stats_messages_ping_received:1152
cluster_stats_messages_pong_received:1182
cluster_stats_messages_meet_received:3
cluster_stats_messages_received:2337

슬롯(slot) 확인

redis-cli -h 192.168.0.51 -p 6381 cluster slots
1) 1) (integer) 0
   2) (integer) 5460
   3) 1) "192.168.0.51"
      2) (integer) 6381
      3) "f3ee1bbf6718735d1944846f449bbac8063029de"
   4) 1) "192.168.0.74"
      2) (integer) 6382
      3) "c591902cc55d2fd74429745c3a1ac44352bbc679"
2) 1) (integer) 5461
   2) (integer) 10922
   3) 1) "192.168.0.74"
      2) (integer) 6381
      3) "041c618f6e2e38d795ed47a38424650bfe170ae7"
   4) 1) "192.168.0.75"
      2) (integer) 6382
      3) "e8cfa56223ed8670d9a4757c876019c8b115b20f"
3) 1) (integer) 10923
   2) (integer) 16383
   3) 1) "192.168.0.75"
      2) (integer) 6381
      3) "0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325"
   4) 1) "192.168.0.51"
      2) (integer) 6382
      3) "f6b6efbb92768db1925682b48f45a80e02d5217c"

8. 클러스터 동작 테스트

Ping 테스트

redis-cli -h 192.168.0.51 -p 6381 ping
PONG

데이터 저장/조회

  • SET
redis-cli -h 192.168.0.51 -p 6381 -c set foo bar
OK
redis-cli -h 192.168.0.51 -p 6381 -c set hello world
OK
  • GET
redis-cli -h 192.168.0.51 -p 6381 -c get foo
"bar"
redis-cli -h 192.168.0.51 -p 6381 -c get hello
"world"

Key 이동 확인(keys)

redis-cli -h 192.168.0.51 -p 6381 -c
192.168.0.51:6381> keys *
1) "hello"
192.168.0.51:6381> get foo
-> Redirected to slot [12182] located at 192.168.0.75:6381
"bar"
192.168.0.75:6381> keys *
1) "foo"
192.168.0.75:6381> save
OK
192.168.0.75:6381

9. 장애 조치 테스트

pnode3(192.168.0.75) 강제 종료 시 → 해당 노드의 Slave(192.168.0.51:6382)가 자동 승격됨.

redis-cli -p 6381 cluster nodes
f3ee1bbf6718735d1944846f449bbac8063029de 192.168.0.51:6381@16381 myself,master - 0 1666797240000 1 connected 0-5460
f6b6efbb92768db1925682b48f45a80e02d5217c 192.168.0.51:6382@16382 master - 0 1666797240000 4 connected 10923-16383
0fe7f083bd5bf9d10e8f1fa7ad99f8d64f762325 192.168.0.75:6381@16381 master,fail - 1666797191581 1666797190000 3 disconnected
c591902cc55d2fd74429745c3a1ac44352bbc679 192.168.0.74:6382@16382 slave f3ee1bbf6718735d1944846f449bbac8063029de 0 1666797240760 1 connected
041c618f6e2e38d795ed47a38424650bfe170ae7 192.168.0.74:6381@16381 master - 0 1666797240254 2 connected 5461-10922
e8cfa56223ed8670d9a4757c876019c8b115b20f 192.168.0.75:6382@16382 slave,fail 041c618f6e2e38d795ed47a38424650bfe170ae7 1666797191581 1666797190000 2 disconnected
  • 결과: cluster_state:ok 유지, Slave → Master 자동 승격 확인 가능.
redis-cli -p 6381 cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:4
cluster_my_epoch:1
cluster_stats_messages_ping_sent:9916
cluster_stats_messages_pong_sent:9804
cluster_stats_messages_fail_sent:3
cluster_stats_messages_auth-ack_sent:1
cluster_stats_messages_sent:19724
cluster_stats_messages_ping_received:9801
cluster_stats_messages_pong_received:9915
cluster_stats_messages_meet_received:3
cluster_stats_messages_fail_received:2
cluster_stats_messages_auth-req_received:1
cluster_stats_messages_received:19722

 

이 구성을 통해 데이터는 자동 분산 저장되며, 노드 장애 발생 시 자동 Failover로 안정적인 서비스를 보장할 수 있습니다.

 

참고URL

- 변군이글루 블로그 : 우분투에서 Redis 서버를 소스 코드로 컴파일하여 설치하는 방법

 

반응형