一、Redis Cluster 工作原理
在引入哨兵机制后，解决了Redis主从架构Master故障时的主从切换问题，保证了Redis服务可用性。但依旧无法解决单机节点出现的写入性能瓶颈（网卡速率、单机内存容量、并发数量）
1、早期为解决单机性能瓶颈问题采用的解决方案：
1、客户端分片：由客户端程序进行读写key的redis节点判断和分配，并且由客户端自行处理读写请求分配、高可用管理及故障转移操作
2、proxy代理模式：引入第三方代理程序，客户端通过连接proxy代理服务器对数据进行读写，由proxy程序进行读写判断分配，并对集群节点进行管理。但导致proxy又出现单点故障风险，并增加了一层数据处理环节
redis3.0 之后官方推出了无中心化的Cluster集群机制，集群中的每个节点单独保持当前节点数据和集群状态信息，每个节点需要和其余全部节点保持连接。
2、Redis Cluster工作特点
1、Redis cluster集群中的各个节点之间（采用ping机制）进行互联
2、集群中的某个节点失效，是有整个集群中超过半数的节点监测失效（sdown），才会真正判定为节点不可用（odown）
3、客户端无需配置连接proxy即可直连redis，应用程序需配置集群的所有redis节点IP
4、redis cluster将集群中所有的节点平均映射到0-16383（16384个）slot槽位上，将数据库的读写全部映射到对应的节点上操作，因此每增加n个集群主节点就能对redis进行n倍扩展，每个主节点将分担16384/n个slot
5、redis cluster预先将16384个槽位分配至集群中的主节点，当接收到对redis的写入请求时，会通过 CRC16(key)%16384 计算出当前key的所属slot编号，从而将数据写入到对应的主节点上，从而解决单机产生的性能瓶颈
3、Redis Cluster 架构设计
Redis Cluster 基本架构
Redis Cluster 主从架构
为解决Cluster集群中各Master节点的高可用性，可采用Cluster主从架构部署，将部分加入集群的节点设置为Slave角色，从而解决Master节点的高可用性
Cluster可自行实现类似哨兵机制的每组Master-Slave的主从切换，无需额外配置哨兵机制。并且支持每组节点的 1-n 模式（不支持1-n-n）
Redis Cluster 部署架构设计
3、Redis Cluster局限性
1、大多数时候客户端连接性能会“降低”，由于cluster在读写数据时需要先进行solt计算并重定向访问到对应节点上，造成了数据处理“延迟性”。
2、部分命令无法跨节点使用：mget、keys、scan、flush、sinter 等
3、客户端维护复杂：SDK 和 应用自身消耗增加（更多的数据连接池）
4、不支持多数据库：cluster 模式下仅支持使用db0
5、复制仅支持一层：不支持级联复制
6、key事务、Lua 的支持有限：key操作必须在同一个节点上，事务和Lua无法跨节点使用
二、Redis Cluster 相关配置详解
################################ REDIS CLUSTER ################################ Normal Redis instances can't be part of a Redis Cluster; only nodes that are# started as cluster nodes can. In order to start a Redis instance as a# cluster node enable the cluster support uncommenting the following:## cluster-enabled yes #启用Cluster模式# Every cluster node has a cluster configuration file. This file is not# intended to be edited by hand. It is created and updated by Redis nodes.# Every Redis Cluster node requires a different cluster configuration file.# Make sure that instances running in the same system do not have# overlapping cluster configuration file names.## cluster-config-file nodes-6379.conf #Cluster节点配置文件名称# Cluster node timeout is the amount of milliseconds a node must be unreachable# for it to be considered in failure state.# Most other internal time limits are multiple of the node timeout.## cluster-node-timeout 15000 #Cluster节点超时时间，单位：毫秒# A replica of a failing master will avoid to start a failover if its data# looks too old.## There is no simple way for a replica to actually have an exact measure of# its "data age", so the following two checks are performed:## 1) If there are multiple replicas able to failover, they exchange messages# in order to try to give an advantage to the replica with the best# replication offset (more data from the master processed).# Replicas will try to get their rank by offset, and apply to the start# of the failover a delay proportional to their rank.## 2) Every single replica computes the time of the last interaction with# its master. This can be the last ping or command received (if the master# is still in the "connected" state), or the time that elapsed since the# disconnection with the master (if the replication link is currently down).# If the last interaction is too old, the replica will not try to failover# at all.## The point "2" can be tuned by user. Specifically a replica will not perform# the failover if, since the last interaction with the master, the time# elapsed is greater than:## (node-timeout * replica-validity-factor) + repl-ping-replica-period## So for example if node-timeout is 30 seconds, and the replica-validity-factor# is 10, and assuming a default repl-ping-replica-period of 10 seconds, the# replica will not try to failover if it was not able to talk with the master# for longer than 310 seconds.## A large replica-validity-factor may allow replicas with too old data to failover# a master, while a too small value may prevent the cluster from being able to# elect a replica at all.## For maximum availability, it is possible to set the replica-validity-factor# to a value of 0, which means, that replicas will always try to failover the# master regardless of the last time they interacted with the master.# (However they'll always try to apply a delay proportional to their# offset rank).## Zero is the only value able to guarantee that when all the partitions heal# the cluster will always be able to continue.## cluster-replica-validity-factor 10
# 集群副本有效性因子（0表示slave将始终尝试故障转移为master，默认被注释掉）# (node-timeout * replica-validity-factor) + repl-ping-replica-period# 计算时间（秒），如果slave无法与master通信的时间超过了该时间，则slave将不会尝试故障转移# Cluster replicas are able to migrate to orphaned masters, that are masters# that are left without working replicas. This improves the cluster ability# to resist to failures as otherwise an orphaned master can't be failed over# in case of failure if it has no working replicas.## Replicas migrate to orphaned masters only if there are still at least a# given number of other working replicas for their old master. This number# is the "migration barrier". A migration barrier of 1 means that a replica# will migrate only if there is at least 1 other working replica for its master# and so forth. It usually reflects the number of replicas you want for every# master in your cluster.## Default is 1 (replicas migrate only if their masters remain with at least# one replica). To disable migration just set it to a very large value.# A value of 0 can be set but is useful only for debugging and dangerous# in production.## cluster-migration-barrier 1 #集群迁移屏障，仅当master至少有1个其他工作副本时，副本才会迁移# By default Redis Cluster nodes stop accepting queries if they detect there# is at least an hash slot uncovered (no available node is serving it).# This way if the cluster is partially down (for example a range of hash slots# are no longer covered) all the cluster becomes, eventually, unavailable.# It automatically returns available as soon as all the slots are covered again.## However sometimes you want the subset of the cluster which is working,# to continue to accept queries for the part of the key space that is still# covered. In order to do so, just set the cluster-require-full-coverage# option to no.## cluster-require-full-coverage yes # 集群需要全覆盖（默认为 yes，被注释掉）# 如果redis集群节点检测到至少有一个未覆盖的哈希槽（没有可用的节点提供服务），则会停止接受查询# 如果集群部分关闭（例如不再覆盖一系列哈希槽），整个集群将不可用# 当所有插槽再次被覆盖，集群将自动返回可用状态# This option, when set to yes, prevents replicas from trying to failover its# master during master failures. However the master can still perform a# manual failover, if forced to do so.## This is useful in different scenarios, especially in the case of multiple# data center operations, where we want one side to never be promoted if not# in the case of a total DC failure.## cluster-replica-no-failover no #主节点故障期间不启用故障转移# In order to setup your cluster make sure to read the documentation# available at http://redis.io web site.########################## CLUSTER DOCKER/NAT support ######################### In certain deployments, Redis Cluster nodes address discovery fails, because# addresses are NAT-ted or because ports are forwarded (the typical case is# Docker and other containers).## In order to make Redis Cluster working in such environments, a static# configuration where each node knows its public address is needed. The# following two options are used for this scope, and are:## * cluster-announce-ip# * cluster-announce-port# * cluster-announce-bus-port## Each instruct the node about its address, client port, and cluster message# bus port. The information is then published in the header of the bus packets# so that other nodes will be able to correctly map the address of the node# publishing the information.## If the above options are not used, the normal Redis Cluster auto-detection# will be used instead.## Note that when remapped, the bus port may not be at the fixed offset of# clients port + 10000, so you can specify any port and bus-port depending# on how they get remapped. If the bus-port is not set, a fixed offset of# 10000 will be used as usually.## Example:## cluster-announce-ip 10.1.1.5# cluster-announce-port 6379# cluster-announce-bus-port 6380
# 配置集群节点的公共地址（默认被注释掉，使用redis群集自动检测）# 节点的ip地址，客户端端口，集群消息总线端口
三、Redis Cluster部署0、基础环境准备所有redis节点采用相同硬件配置，相同密码，清空所有数据
| IP | 角色 | slot | |
| node1 | 10.0.0.20 | M1 | 0-5460 |
| node2 | 10.0.0.21 | M2 | 5461-10922 |
| node3 | 10.0.0.22 | M3 | 10923-16383 |
| node4 | 10.0.0.23 | 集群扩容缩容备用（M4） | |
| node5 | 10.0.0.30 | S1 | |
| node6 | 10.0.0.31 | S2 | |
| node7 | 10.0.0.32 | S3 | |
| node8 | 10.0.0.33 | 集群扩容缩容备用（S4） |
初始化安装6台 redis服务，并启用Cluster模式
[root@Redis-Ubuntu1804-node1:~]# apt install redis -y[root@Redis-Ubuntu1804-node1:~]# sed -i.bak -e 's/bind 127.0.0.1/bind 0.0.0.0/' -e '/masterauth/a masterauth redis' -e '/# requirepass/a requirepass redis' -e '/# cluster-enabled yes/a cluster-enabled yes' -e '/# cluster-config-file/a cluster-config-file nodes-6379.conf' -e '/# cluster-require-full-coverage yes/a cluster-require-full-coverage yes' /etc/redis/redis.conf[root@Redis-Ubuntu1804-node1:~]# systemctl restart redis[root@Redis-Ubuntu1804-node1:~]# ss -ntlState Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 127.0.0.53%lo:53 0.0.0.0:* LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 127.0.0.1:6010 0.0.0.0:* LISTEN 0 128 0.0.0.0:16379 0.0.0.0:* LISTEN 0 128 0.0.0.0:6379 0.0.0.0:* LISTEN 0 128 [::]:22 [::]:* LISTEN 0 128 [::1]:6010 [::]:* [root@Redis-Ubuntu1804-node1:~]# ps -ef | grep redisredis 2098 1 0 01:38 ? 00:00:00 /usr/bin/redis-server 0.0.0.0:6379 [cluster]root 2112 1236 0 01:39 pts/0 00:00:00 grep --color=auto redis[root@Redis-Ubuntu1804-node1:~]#
1、基本模式手动部署
基本步骤
1、各节点服务器上安装redis服务，启用Cluster配置
2、通过meet 将节点添加至集群中
3、为各Master节点分配槽位 0~16383 ，必须全部分配完成，集群才可用
4、指定各节点主从关系
1.1、查看当前节点状态信息
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes98bb2740b622645ff51fd07bc994bb22e94feaaa :6379@16379 myself,master - 0 0 0 connected[root@Redis-Ubuntu1804-node1:~]#
1.2、将所有节点加入到当前集群中
Cluster节点操作相关命令
1、加入节点：将 ip 和 port 所指定的节点添加到集群当中，让它成为集群的一份子。
1）CLUSTER MEET
2、移除节点：
1）CLUSTER FORGET
2)、redis-trib.rb del-node
3、设置主从节点：
CLUSTERREPLICATE将当前节点设置为node_id指定的节点的从节点。
4、节点数据备份到硬盘：
CLUSTERSAVECONFIG将节点的配置文件保存到硬盘里面。
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.20 6379[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.20 6379OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.21 6379OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.22 6379OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.30 6379OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.31 6379OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.32 6379OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 myself,master - 0 1681926433000 1 connected9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 master - 0 1681926434205 5 connected55fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 master - 0 1681926435212 4 connected6717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 master - 0 1681926433000 3 connectedd9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681926434000 0 connectedbdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681926433199 2 connected[root@Redis-Ubuntu1804-node1:~]#
1.3、查看当前集群信息
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster infocluster_state:failcluster_slots_assigned:0cluster_slots_ok:0cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:0cluster_current_epoch:5cluster_my_epoch:1cluster_stats_messages_ping_sent:70cluster_stats_messages_pong_sent:89cluster_stats_messages_meet_sent:6cluster_stats_messages_sent:165cluster_stats_messages_ping_received:88cluster_stats_messages_pong_received:76cluster_stats_messages_meet_received:1cluster_stats_messages_received:165[root@Redis-Ubuntu1804-node1:~]#
1.4、为集群节点分配槽位
Cluster 槽位相关命令
CLUSTER ADDSLOTS [slot …] 将一个或多个槽（slot）指派（assign）给当前节点。CLUSTER DELSLOTS [slot …] 移除一个或多个槽对当前节点的指派。
CLUSTER FLUSHSLOTS 移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。
CLUSTER SETSLOT NODE 将槽 slot 指派给 node_id 指定的节点，如果槽已经指派给另一个节点，那么先让另一个节点删除该槽>，然后再进行指派。
CLUSTER SETSLOT MIGRATING 将本节点的槽 slot 迁移到 node_id 指定的节点中。
CLUSTER SETSLOT IMPORTING 从 node_id 指定的节点中导入槽 slot 到本节点。
CLUSTER SETSLOT STABLE 取消对槽 slot 的导入（import）或者迁移（migrate）。
使用命令redis-cli -h -p -a –no-auth-warning cluster addslots 进行槽位分配
[root@Client-Ubuntu-1804-250:~/script/redis]# cat add_slots.sh #!/bin/bash##********************************************************************#Author: janzen#Date: 2023-04-20#FileName： add_slots.sh#Description： The test script#Copyright (C): 2023 All rights reserved#********************************************************************host=$1port=$2start=$3end=$4passwd=redisfor slot in `seq $start $end`; do redis-cli -h $host -p $port -a $passwd --no-auth-warning cluster addslots $slot && echo $slot add to node $host:$portdone[root@Client-Ubuntu-1804-250:~/script/redis]# [root@Client-Ubuntu-1804-250:~/script/redis]# ./add_slots.sh 10.0.0.20 6379 0 5460 >> addslot.log[root@Client-Ubuntu-1804-250:~/script/redis]# ./add_slots.sh 10.0.0.21 6379 5461 10922 >> addslot.log[root@Client-Ubuntu-1804-250:~/script/redis]# ./add_slots.sh 10.0.0.22 6379 10923 16383 >> addslot.log
##查看当前集群信息，槽位已完成分配
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 myself,master - 0 1681929286000 1 connected 0-546055fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 master - 0 1681929287000 4 connected9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 master - 0 1681929287000 5 connected6717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 master - 0 1681929288247 3 connectedd9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681929286000 0 connected 5461-10922bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681929289256 2 connected 10923-16383[root@Redis-Ubuntu1804-node1:~]#
1.5、创建 Master-Slave 主从关系
使用命令redis-cli -h -p -a –no-auth-warning cluster replicate
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 cluster replicate 98bb2740b622645ff51fd07bc994bb22e94feaaaOK##观察主节点上的replication信息[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 info Replication# Replicationrole:masterconnected_slaves:1slave0:ip=10.0.0.30,port=6379,state=online,offset=350,lag=1master_replid:6a823070fcbc13b4f4eb7b5d13bf1d2f4625a1a2master_replid2:0000000000000000000000000000000000000000master_repl_offset:350second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:350[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 myself,master - 0 1681929989000 1 connected 0-546055fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 master - 0 1681929992372 4 connected9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 master - 0 1681929991000 5 connected6717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 slave 98bb2740b622645ff51fd07bc994bb22e94feaaa 0 1681929992000 3 connectedd9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681929989351 0 connected 5461-10922bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681929991364 2 connected 10923-16383##观察从节点上的replication信息[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 info Replication# Replicationrole:slavemaster_host:10.0.0.20master_port:6379master_link_status:upmaster_last_io_seconds_ago:0master_sync_in_progress:0slave_repl_offset:462slave_priority:100slave_read_only:1connected_slaves:0master_replid:6a823070fcbc13b4f4eb7b5d13bf1d2f4625a1a2master_replid2:0000000000000000000000000000000000000000master_repl_offset:462second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:462[root@Redis-Ubuntu1804-node1:~]#
继续完成主从组配置
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.31 cluster replicate d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bedOK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.32 cluster replicate bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2OK
##观察集群节点状态[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 cluster nodes9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 slave bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 0 1681930655876 5 connected98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 master - 0 1681930654870 1 connected 0-546055fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 slave d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 0 1681930656886 4 connectedbdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681930655000 2 connected 10923-163836717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 myself,slave 98bb2740b622645ff51fd07bc994bb22e94feaaa 0 1681930654000 3 connectedd9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681930653863 0 connected 5461-10922
1.6、验证Cluster集群访问
[root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.20 set node1 10.0.0.20OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.21 set node2 10.0.0.21OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.22 set node3 10.0.0.22OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.30 set node5 10.0.0.24OK[root@Redis-Ubuntu1804-node1:~]# [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.31 get node1"10.0.0.20"[root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.31 get node2"10.0.0.21"[root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.21 get node6"10.0.0.31"[root@Redis-Ubuntu1804-node1:~]#
2、基于 Redis 5 Cluster集群部署–cluster 常用命令
[root@Redis-Ubuntu1804-node3:~]# redis-cli -a redis --cluster helpWarning: Using a password with '-a' or '-u' option on the command line interface may not be safe.Cluster Manager Commands: create host1:port1 ... hostN:portN ##创建集群 --cluster-replicas ##指定每个集群的slave节点数，通常为1 check host:port ##检查集群信息 --cluster-search-multiple-owners info host:port ##查看集群节点基本信息 fix host:port ##修复集群 --cluster-search-multiple-owners reshard host:port ##在线热机迁移指定的slots数据 --cluster-from --cluster-to --cluster-slots --cluster-yes --cluster-timeout --cluster-pipeline --cluster-replace rebalance host:port ##平衡集群在线节点的slots数量 --cluster-weight --cluster-use-empty-masters --cluster-timeout --cluster-simulate --cluster-pipeline --cluster-threshold --cluster-replace add-node new_host:new_port existing_host:existing_port ##添加新的节点到集群中 --cluster-slave --cluster-master-id del-node host:port node_id ##删除集群中的节点 call host:port command arg arg .. arg ##在集群上的所有节点执行命令 set-timeout host:port milliseconds ##设置节点超时时间 import host:port ##导入外部redis数据到当前集群 --cluster-from --cluster-copy --cluster-replace help For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
2.0、基本环境准备
[root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis --versionredis-cli 5.0.14[root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis --no-auth-warning cluster nodese603091426e2ce29629ca51fb5e7dafb2f0c9524 :6379@16379 myself,master - 0 0 0 connected[root@Redis-Ubuntu1804-node4:~]#
2.1、创建Cluster集群
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --cluster create 10.0.0.20:6379 10.0.0.21:6379 10.0.0.22:6379 10.0.0.30:6379 10.0.0.31:6379 10.0.0.32:6379 --cluster-replicas 1Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.>>> Performing hash slots allocation on 6 nodes...Master[0] -> Slots 0 - 5460Master[1] -> Slots 5461 - 10922Master[2] -> Slots 10923 - 16383Adding replica 10.0.0.31:6379 to 10.0.0.20:6379Adding replica 10.0.0.32:6379 to 10.0.0.21:6379Adding replica 10.0.0.30:6379 to 10.0.0.22:6379M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) masterM: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) masterM: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots:[10923-16383] (5461 slots) masterS: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fCan I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated>>> Assign a different config epoch to each node>>> Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)M: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots: (0 slots) slave replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
2.2、查看集群状态
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis cluster nodesWarning: Using a password with '-a' or '-u' option on the command line interface may not be safe.07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master - 0 1681998239529 3 connected 10923-16383a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1681998238520 2 connected 5461-109226a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1681998237000 6 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1681998238000 5 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 slave 07a36af61eb1f887419f2266b2360b4f795bb7a3 0 1681998237510 4 connected6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1681998236000 1 connected 0-5460[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis cluster infoWarning: Using a password with '-a' or '-u' option on the command line interface may not be safe.cluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:6cluster_my_epoch:1cluster_stats_messages_ping_sent:290cluster_stats_messages_pong_sent:282cluster_stats_messages_sent:572cluster_stats_messages_ping_received:277cluster_stats_messages_pong_received:290cluster_stats_messages_meet_received:5cluster_stats_messages_received:572[root@Redis-Ubuntu1804-node1:~]# [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (07a36af6...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]#
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning --cluster check 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (07a36af6...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)M: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots: (0 slots) slave replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
2.3、验证集群写入
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster nodes07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master - 0 1681998730094 3 connected 10923-16383a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1681998729087 2 connected 5461-109226a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1681998728000 6 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1681998728079 5 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 slave 07a36af61eb1f887419f2266b2360b4f795bb7a3 0 1681998729000 4 connected6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1681998726000 1 connected 0-5460#写入 key1：value1
###经过slot计算 key1 不属于当前节点
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key1 value1(error) MOVED 9189 10.0.0.21:6379
##计算 key 值所属的slot号[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key1(integer) 9189
##结合 nodes 信息，切换到对应节点上进行写入[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning set key1 value1OK
##get命令也只可查看当前节点上的数据[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning get key1(error) MOVED 9189 10.0.0.21:6379[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning get key1"value1"
##可以用 -c 参数通过连接到集群进行数据操作，就无需反复切换节点（key1、key4 不在当前节点所属的slot范围，出现错误提示）[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key1(integer) 9189[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key2(integer) 4998[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key3(integer) 935[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key4(integer) 13120[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning set key1 value1OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key1 value1(error) MOVED 9189 10.0.0.21:6379[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key2 value2OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key3 value3OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key4 value4(error) MOVED 13120 10.0.0.22:6379[root@Redis-Ubuntu1804-node1:~]#
##使用 -c 参数后执行成功
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning set key1 value1
OK
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning set key4 value4
OK
[root@Redis-Ubuntu1804-node1:~]#
2.4、模拟故障验证集群故障切换
启用脚本模拟客户端数据读写
故障前集群信息状态
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master - 0 1681999895495 3 connected 10923-16383a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1681999896502 2 connected 5461-109226a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1681999894488 6 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1681999895000 5 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 slave 07a36af61eb1f887419f2266b2360b4f795bb7a3 0 1681999892000 4 connected6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1681999893000 1 connected 0-5460[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 97 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (07a36af6...) -> 100 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 92 keys | 5462 slots | 1 slaves.[OK] 289 keys in 3 masters.0.02 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 99 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (07a36af6...) -> 103 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 94 keys | 5462 slots | 1 slaves.[OK] 296 keys in 3 masters.0.02 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)M: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots: (0 slots) slave replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
停用 10.0.0.22 节点
[root@Redis-Ubuntu1804-node3:~]# systemctl stop redis_6379.service [root@Redis-Ubuntu1804-node3:~]# 观察故障期间集群状态及日志
##10.0.0.22节点标记 faild，并提升它的从节点 10.0.0.30 为master
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379Could not connect to Redis at 10.0.0.22:6379: Connection refused*** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.10.0.0.20:6379 (6b142fe3...) -> 131 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 126 keys | 5462 slots | 1 slaves.[OK] 257 keys in 2 masters.0.02 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots: (0 slots) slave replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[ERR] Not all 16384 slots are covered by nodes.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379Could not connect to Redis at 10.0.0.22:6379: Connection refused*** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.10.0.0.20:6379 (6b142fe3...) -> 131 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 126 keys | 5462 slots | 1 slaves.[OK] 257 keys in 2 masters.0.02 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master,fail - 1682000024732 1682000022415 3 disconnecteda4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682000042000 2 connected 5461-109226a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682000040567 6 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682000042582 5 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682000041575 7 connected 10923-163836b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1682000041000 1 connected 0-5460[root@Redis-Ubuntu1804-node1:~]#
17519:M 20 Apr 2023 22:14:01.600 * Marking node 07a36af61eb1f887419f2266b2360b4f795bb7a3 as failing (quorum reached).17519:M 20 Apr 2023 22:14:01.600 # Cluster state changed: fail17519:M 20 Apr 2023 22:14:02.167 # Failover auth granted to 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 for epoch 717519:M 20 Apr 2023 22:14:02.177 # Cluster state changed: ok
客户端日志出现了约4s 的服务不可用
恢复 10.0.0.22 节点
观察集群节点信息及日志
##10.0.0.22节点被添加为slave节点，并将master指向10.0.0.30
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.[OK] 445 keys in 3 masters.0.03 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.[OK] 445 keys in 3 masters.0.03 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
##日志记录清除 10.0.0.22 节点上的faild 标记，并作为slave节点启动17519:M 20 Apr 2023 22:14:01.600 * Marking node 07a36af61eb1f887419f2266b2360b4f795bb7a3 as failing (quorum reached).17519:M 20 Apr 2023 22:14:01.600 # Cluster state changed: fail17519:M 20 Apr 2023 22:14:02.167 # Failover auth granted to 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 for epoch 717519:M 20 Apr 2023 22:14:02.177 # Cluster state changed: ok17519:M 20 Apr 2023 22:17:23.792 * Clear FAIL state for node 07a36af61eb1f887419f2266b2360b4f795bb7a3: master without slots is reachable again.
3、基于Redis 3/4 Cluster集群部署3.0、基本环境准备
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis cluster nodes1c2fee65ee50cdf66c862d04710672286b305f55 :6379@16379 myself,master - 0 0 0 connected[root@Redis-Ubuntu1804-node1:~]# redis-cli --versionredis-cli 4.0.9[root@Redis-Ubuntu1804-node1:~]#
3.1、准备 redis-trib.rb 工具查找 redis-trib.rb 工具路径，将二进制文件复制到 /usr/bin 目录下
[root@Redis-Ubuntu1804-node1:~]# find / -name redis-trib.rb/usr/share/doc/redis-tools/examples/redis-trib.rb[root@Redis-Ubuntu1804-node1:~]# cp /usr/share/doc/redis-tools/examples/redis-trib.rb /usr/bin/
安装ruby运行环境
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb /usr/bin/env: ‘ruby’: No such file or directory[root@Redis-Ubuntu1804-node1:~]# rubyCommand 'ruby' not found, but can be installed with:snap install ruby # version 3.2.2, orapt install rubySee 'snap info ruby' for additional versions.[root@Redis-Ubuntu1804-node1:~]# apt-cache madison ruby ruby | 1:2.5.1 | http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 Packages[root@Redis-Ubuntu1804-node1:~]# apt install ruby -yReading package lists... DoneBuilding dependency tree Reading state information... DoneThe following packages were automatically installed and are no longer required: linux-headers-4.15.0-151 linux-headers-4.15.0-151-generic linux-image-4.15.0-151-generic linux-modules-4.15.0-151-generic linux-modules-extra-4.15.0-151-genericUse 'apt autoremove' to remove them.The following additional packages will be installed: fonts-lato javascript-common libjs-jquery libruby2.5 rake ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert ruby-test-unit ruby2.5 rubygems-integration unzip zipSuggested packages: apache2 | lighttpd | httpd ri ruby-dev bundlerThe following NEW packages will be installed: fonts-lato javascript-common libjs-jquery libruby2.5 rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert ruby-test-unit ruby2.5 rubygems-integration unzip zip0 upgraded, 15 newly installed, 0 to remove and 62 not upgraded.Need to get 6499 kB of archives.After this operation, 29.0 MB of additional disk space will be used.Get:1 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 fonts-lato all 2.0-2 [2698 kB]Get:2 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 javascript-common all 11 [6066 B]Get:3 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 libjs-jquery all 3.2.1-1 [152 kB]Get:4 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 rubygems-integration all 1.11 [4994 B]Get:5 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 ruby2.5 amd64 2.5.1-1ubuntu1.13 [48.6 kB]Get:6 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby amd64 1:2.5.1 [5712 B]Get:7 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 rake all 12.3.1-1ubuntu0.1 [44.9 kB]Get:8 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-did-you-mean all 1.2.0-2 [9700 B]Get:9 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-minitest all 5.10.3-1 [38.6 kB]Get:10 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-net-telnet all 0.1.1-2 [12.6 kB]Get:11 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-power-assert all 0.3.0-1 [7952 B]Get:12 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-test-unit all 3.2.5-1 [61.1 kB]Get:13 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libruby2.5 amd64 2.5.1-1ubuntu1.13 [3074 kB] Get:14 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 unzip amd64 6.0-21ubuntu1.2 [168 kB] Get:15 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 zip amd64 3.0-11build1 [167 kB] Fetched 6499 kB in 9s (722 kB/s) Selecting previously unselected package fonts-lato.(Reading database ... 144903 files and directories currently installed.)Preparing to unpack .../00-fonts-lato_2.0-2_all.deb ...Unpacking fonts-lato (2.0-2) ...Selecting previously unselected package javascript-common.Preparing to unpack .../01-javascript-common_11_all.deb ...Unpacking javascript-common (11) ...Selecting previously unselected package libjs-jquery.Preparing to unpack .../02-libjs-jquery_3.2.1-1_all.deb ...Unpacking libjs-jquery (3.2.1-1) ...Selecting previously unselected package rubygems-integration.Preparing to unpack .../03-rubygems-integration_1.11_all.deb ...Unpacking rubygems-integration (1.11) ...Selecting previously unselected package ruby2.5.Preparing to unpack .../04-ruby2.5_2.5.1-1ubuntu1.13_amd64.deb ...Unpacking ruby2.5 (2.5.1-1ubuntu1.13) ...Selecting previously unselected package ruby.Preparing to unpack .../05-ruby_1%3a2.5.1_amd64.deb ...Unpacking ruby (1:2.5.1) ...Selecting previously unselected package rake.Preparing to unpack .../06-rake_12.3.1-1ubuntu0.1_all.deb ...Unpacking rake (12.3.1-1ubuntu0.1) ...Selecting previously unselected package ruby-did-you-mean.Preparing to unpack .../07-ruby-did-you-mean_1.2.0-2_all.deb ...Unpacking ruby-did-you-mean (1.2.0-2) ...Selecting previously unselected package ruby-minitest.Preparing to unpack .../08-ruby-minitest_5.10.3-1_all.deb ...Unpacking ruby-minitest (5.10.3-1) ...Selecting previously unselected package ruby-net-telnet.Preparing to unpack .../09-ruby-net-telnet_0.1.1-2_all.deb ...Unpacking ruby-net-telnet (0.1.1-2) ...Selecting previously unselected package ruby-power-assert.Preparing to unpack .../10-ruby-power-assert_0.3.0-1_all.deb ...Unpacking ruby-power-assert (0.3.0-1) ...Selecting previously unselected package ruby-test-unit.Preparing to unpack .../11-ruby-test-unit_3.2.5-1_all.deb ...Unpacking ruby-test-unit (3.2.5-1) ...Selecting previously unselected package libruby2.5:amd64.Preparing to unpack .../12-libruby2.5_2.5.1-1ubuntu1.13_amd64.deb ...Unpacking libruby2.5:amd64 (2.5.1-1ubuntu1.13) ...Selecting previously unselected package unzip.Preparing to unpack .../13-unzip_6.0-21ubuntu1.2_amd64.deb ...Unpacking unzip (6.0-21ubuntu1.2) ...Selecting previously unselected package zip.Preparing to unpack .../14-zip_3.0-11build1_amd64.deb ...Unpacking zip (3.0-11build1) ...Setting up libjs-jquery (3.2.1-1) ...Setting up unzip (6.0-21ubuntu1.2) ...Setting up zip (3.0-11build1) ...Setting up fonts-lato (2.0-2) ...Setting up ruby-did-you-mean (1.2.0-2) ...Setting up ruby-net-telnet (0.1.1-2) ...Setting up rubygems-integration (1.11) ...Setting up javascript-common (11) ...Setting up ruby-minitest (5.10.3-1) ...Setting up ruby-power-assert (0.3.0-1) ...Setting up ruby2.5 (2.5.1-1ubuntu1.13) ...Setting up ruby (1:2.5.1) ...Setting up ruby-test-unit (3.2.5-1) ...Setting up rake (12.3.1-1ubuntu0.1) ...Setting up libruby2.5:amd64 (2.5.1-1ubuntu1.13) ...Processing triggers for mime-support (3.60ubuntu1) ...Processing triggers for libc-bin (2.27-3ubuntu1.5) ...Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
解决 redis-rtib.rb 执行报错
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb Traceback (most recent call last): 2: from /usr/bin/redis-trib.rb:25:in `' 1: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'/usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- redis (LoadError)
##使用 gem 工具安装 ruby 的redis模块[root@Redis-Ubuntu1804-node1:~]# gem install redisFetching: redis-5.0.6.gem (100%)Successfully installed redis-5.0.6Parsing documentation for redis-5.0.6Installing ri documentation for redis-5.0.6Done installing documentation for redis after 0 seconds1 gem installed
##离线模式安装 gem
[root@Redis-Ubuntu1804-node1:~]# wget https://rubygems.org/downloads/redis-4.0.3.gem
[root@Redis-Ubuntu1804-node1:~]# gem install redis-4.0.3.gem
redis-trib.rb 常用命令
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rbUsage: redis-trib create host1:port1 ... hostN:portN #创建集群 --replicas #指定每个master节点的副本数，即每组主从的slave节点数，通常为1 check host:port #检查集群信息 info host:port #查看集群主机信息 fix host:port #修复集群 --timeout reshard host:port #在线热机迁移指定的slots数据 --from --to --slots --yes --timeout --pipeline rebalance host:port #平衡集群在线节点的slots数量 --weight --auto-weights --use-empty-masters --timeout --simulate --pipeline --threshold add-node new_host:new_port existing_host:existing_port #添加新的节点到集群中 --slave --master-id del-node host:port node_id #删除集群中的指定节点 set-timeout host:port milliseconds #设置节点超时时间 call host:port command arg arg .. arg #在集群上所有节点上执行命令 import host:port #导入外部redis服务器数据到当前集群 --from --copy --replace help (show this help)For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
3.2、修改redis-trib.rb 添加连接redis密码
[root@Redis-Ubuntu1804-node1:~]# sed -i 's/@r = Redis.new.*/@r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60, :password => "'redis'")/' /usr/bin/redis-trib.rb
[root@Redis-Ubuntu1804-node1:~]# vim /usr/bin/redis-trib.rb def connect(o={}) return if @r print "Connecting to node #{self}: " if $verbose STDOUT.flush begin @r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60, :password => 'redis') @r.ping rescue xputs "[ERR] Sorry, can't connect to node #{self}" exit 1 if o[:abort] @r = nil end xputs "OK" if $verbose end
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 0 slots | 0 slaves.[OK] 0 keys in 1 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]#
3.3、创建Cluster集群
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb create --replicas 1 10.0.0.20:6379 10.0.0.21:6379 10.0.0.22:6379 10.0.0.30:6379 10.0.0.31:6379 10.0.0.32:6379>>> Creating cluster>>> Performing hash slots allocation on 6 nodes...Using 3 masters:10.0.0.20:637910.0.0.21:637910.0.0.22:6379Adding replica 10.0.0.31:6379 to 10.0.0.20:6379Adding replica 10.0.0.32:6379 to 10.0.0.21:6379Adding replica 10.0.0.30:6379 to 10.0.0.22:6379M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) masterM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) masterM: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) masterS: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bS: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 replicates 1c2fee65ee50cdf66c862d04710672286b305f55S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eCan I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated>>> Assign a different config epoch to each node>>> Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join......>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
集群创建失败处理
#在所有节点上手动清空数据，并重置集群设置
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 --no-auth-warning flushall[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 --no-auth-warning cluster reset
3.4、查看集群状态
查看集群信息自动生产的配置文件
[root@Redis-Ubuntu1804-node1:~]# cat /etc/redis/redis.conf | grep cluster-conf# cluster-config-file nodes-6379.confcluster-config-file nodes-6379.conf[root@Redis-Ubuntu1804-node1:~]# find / -name nodes-6379.conf/var/lib/redis/nodes-6379.conf[root@Redis-Ubuntu1804-node1:~]# cat /var/lib/redis/nodes-6379.conf f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379@16379 slave 709b09b96f19209e7f0a1e3cfc5c035678491e4b 0 1681979886000 4 connected1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379@16379 myself,master - 0 1681979886000 1 connected 0-5460337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379@16379 master - 0 1681979888000 2 connected 5461-10922c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379@16379 slave 1c2fee65ee50cdf66c862d04710672286b305f55 0 1681979888959 5 connected709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379@16379 master - 0 1681979887000 3 connected 10923-163830428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379@16379 slave 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 0 1681979888000 6 connectedvars currentEpoch 6 lastVoteEpoch 0[root@Redis-Ubuntu1804-node1:~]#
查看集群状态信息
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 cluster infocluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:6cluster_my_epoch:1cluster_stats_messages_ping_sent:1000cluster_stats_messages_pong_sent:1033cluster_stats_messages_sent:2033cluster_stats_messages_ping_received:1028cluster_stats_messages_pong_received:1000cluster_stats_messages_meet_received:5cluster_stats_messages_received:2033[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 cluster nodesf9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379@16379 slave 709b09b96f19209e7f0a1e3cfc5c035678491e4b 0 1681980883096 4 connected1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379@16379 myself,master - 0 1681980882000 1 connected 0-5460337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379@16379 master - 0 1681980885110 2 connected 5461-10922c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379@16379 slave 1c2fee65ee50cdf66c862d04710672286b305f55 0 1681980883000 5 connected709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379@16379 master - 0 1681980884000 3 connected 10923-163830428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379@16379 slave 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 0 1681980884103 6 connected[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 info replication# Replicationrole:masterconnected_slaves:1slave0:ip=10.0.0.31,port=6379,state=online,offset=1428,lag=1master_replid:33b67c12391d0787c01305ee41ccba34db5c843bmaster_replid2:0000000000000000000000000000000000000000master_repl_offset:1428second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:1428[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.31 info replication# Replicationrole:slavemaster_host:10.0.0.20master_port:6379master_link_status:upmaster_last_io_seconds_ago:6master_sync_in_progress:0slave_repl_offset:1554slave_priority:100slave_read_only:1connected_slaves:0master_replid:33b67c12391d0787c01305ee41ccba34db5c843bmaster_replid2:0000000000000000000000000000000000000000master_repl_offset:1554second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:1554
3.5、模拟故障验证集群故障切换
使用脚本写入数据，查看集群信息
[root@Client-Ubuntu-1804-250:~/script/redis]# ./redis_cluster_newData.py 100[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 30 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 31 keys | 5462 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 39 keys | 5461 slots | 1 slaves.[OK] 100 keys in 3 masters.0.01 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 -c get keys11"value11"[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 get keys11"value11"[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 get keys12(error) MOVED 14046 10.0.0.22:6379[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.22 get keys12"value12"
关闭 10.0.0.22 redis服务，模拟故障
[root@Redis-Ubuntu1804-node3:~]# systemctl stop redis[root@Redis-Ubuntu1804-node3:~]# ##初始信息[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.##故障发生第一时间[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379[ERR] Sorry, can't connect to node 10.0.0.22:6379*** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 709b09b96f19209e7f0a1e3cfc5c035678491e4b.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[ERR] Not all 16384 slots are covered by nodes.##自动提升Msater[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)M: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots:10923-16383 (5461 slots) master 0 additional replica(s)M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
1021:M 20 Apr 17:45:33.030 * Marking node 709b09b96f19209e7f0a1e3cfc5c035678491e4b as failing (quorum reached).1021:M 20 Apr 17:45:33.030 # Cluster state changed: fail1021:M 20 Apr 17:45:34.067 # Failover auth granted to f9e73307b3146b25b51cec9a861463a616ed322b for epoch 71021:M 20 Apr 17:45:34.070 # Cluster state changed: ok
重新启动 10.0.0.22 redis服务，将被自动设置为slave节点
[root@Redis-Ubuntu1804-node3:~]# systemctl start redis[root@Redis-Ubuntu1804-node3:~]# [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)M: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55S: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots: (0 slots) slave replicates f9e73307b3146b25b51cec9a861463a616ed322bS: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
1014:M 20 Apr 17:45:33.030 * Marking node 709b09b96f19209e7f0a1e3cfc5c035678491e4b as failing (quorum reached).1014:M 20 Apr 17:45:33.030 # Cluster state changed: fail1014:M 20 Apr 17:45:34.067 # Failover auth granted to f9e73307b3146b25b51cec9a861463a616ed322b for epoch 71014:M 20 Apr 17:45:34.070 # Cluster state changed: ok1014:M 20 Apr 17:51:47.852 * Clear FAIL state for node 709b09b96f19209e7f0a1e3cfc5c035678491e4b: master without slots is reachable again.
4、Cluster集群节点维护4.1、集群维护之动态扩容
当生产环境集群负载达到上限，需要扩充Master节点以环境集群性能瓶颈，将涉及到集群扩容操作
*通常生产环境下建议集群Master节点数应设置为奇数（3、5、7），以避免出现脑裂现象
使用集群管理工具的 add-node 命令
add-node new_host:new_port existing_host:existing_port #添加新的节点到集群中 –slave –master-id
4.1.1、Redis5 集群节点扩容查看当前集群节点信息
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.[OK] 445 keys in 3 masters.0.03 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
将节点 10.0.0.23 添加至集群中
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster add-node 10.0.0.23:6379 10.0.0.20:6379>>> Adding node 10.0.0.23:6379 to cluster 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.>>> Send CLUSTER MEET to node 10.0.0.23:6379 to make it join the cluster.[OK] New node added correctly.[root@Redis-Ubuntu1804-node1:~]#
##10.0.0.23 节点已经加入到集群中，但是未分配任何槽位数据，也没有从节点
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.10.0.0.23:6379 (ac947d4f...) -> 0 keys | 0 slots | 0 slaves.10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.[OK] 445 keys in 4 masters.0.03 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]#
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.10.0.0.23:6379 (ac947d4f...) -> 0 keys | 0 slots | 0 slaves.10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.[OK] 445 keys in 4 masters.0.03 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots: (0 slots) masterS: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
备份并清空集群所有Master节点上的数据，为新节点分配槽位
reshard host:port #在线热机迁移指定的slots数据 --from --to --slots --yes --timeout --pipeline
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 --no-auth-warning flushallOK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning flushallOK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 --no-auth-warning flushallOK
##为新节点分配槽位
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots: (0 slots) masterS: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 4096 ##需要进行重新分配的槽位数What is the receiving node ID? ac947d4fc970b79c2364ce305441906b462f8b65 ##接收分配槽位的节点 node idPlease enter all the source node IDs. ##提供重新分配槽位的节点node id Type 'all' to use all the nodes as source nodes for the hash slots. ##输入all将从拥有槽位的所有节点进行自动分配 Type 'done' once you entered all the source nodes IDs. ##输入提供槽位的节点node id，输入done结束Source node #1: all......... Moving slot 12280 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 Moving slot 12281 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 Moving slot 12282 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 Moving slot 12283 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 Moving slot 12284 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 Moving slot 12285 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 Moving slot 12286 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 Moving slot 12287 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34Do you want to proceed with the proposed reshard plan (yes/no)? yes ##确认槽位分配方案.........Moving slot 12280 from 10.0.0.30:6379 to 10.0.0.23:6379: Moving slot 12281 from 10.0.0.30:6379 to 10.0.0.23:6379: Moving slot 12282 from 10.0.0.30:6379 to 10.0.0.23:6379: Moving slot 12283 from 10.0.0.30:6379 to 10.0.0.23:6379: Moving slot 12284 from 10.0.0.30:6379 to 10.0.0.23:6379: Moving slot 12285 from 10.0.0.30:6379 to 10.0.0.23:6379: Moving slot 12286 from 10.0.0.30:6379 to 10.0.0.23:6379: Moving slot 12287 from 10.0.0.30:6379 to 10.0.0.23:6379:
查看槽位分配最终结果
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.23:6379 (ac947d4f...) -> 0 keys | 4096 slots | 0 slaves.10.0.0.30:6379 (5c9789e1...) -> 0 keys | 4096 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]#
将 10.0.0.33 添加至集群中，并设置为 10.0.0.23 的从节点
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster add-node 10.0.0.33:6379 10.0.0.20:6379 --cluster-slave --cluster-master-id ac947d4fc970b79c2364ce305441906b462f8b65>>> Adding node 10.0.0.33:6379 to cluster 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[1365-5460] (4096 slots) master 1 additional replica(s)S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) masterS: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.>>> Send CLUSTER MEET to node 10.0.0.33:6379 to make it join the cluster.Waiting for the cluster to join>>> Configure node as replica of 10.0.0.23:6379.[OK] New node added correctly.[root@Redis-Ubuntu1804-node1:~]#
查看集群节点信息
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.23:6379 (ac947d4f...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 0 keys | 4096 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[1365-5460] (4096 slots) master 1 additional replica(s)S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379 slots: (0 slots) slave replicates ac947d4fc970b79c2364ce305441906b462f8b65S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
4.1.2、Redis 3/4 集群节点扩容当前集群节点信息
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
将节点 10.0.0.23 添加至集群中
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb add-node 10.0.0.23:6379 10.0.0.20:6379>>> Adding node 10.0.0.23:6379 to cluster 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.>>> Send CLUSTER MEET to node 10.0.0.23:6379 to make it join the cluster.[OK] New node added correctly.[root@Redis-Ubuntu1804-node1:~]#
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.23:6379 (0e504f59...) -> 0 keys | 0 slots | 0 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots: (0 slots) master 0 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
备份并清空集群所有Master节点上的数据，为新节点分配槽位
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb call 10.0.0.20:6379 flushall>>> Calling FLUSHALL10.0.0.20:6379: OK10.0.0.31:6379: READONLY You can't write against a read only slave.10.0.0.22:6379: OK10.0.0.32:6379: READONLY You can't write against a read only slave.10.0.0.23:6379: OK10.0.0.30:6379: READONLY You can't write against a read only slave.10.0.0.21:6379: OK[root@Redis-Ubuntu1804-node1:~]#
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots: (0 slots) master 0 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 4096What is the receiving node ID? 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aPlease enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.Source node #1:allDo you want to proceed with the proposed reshard plan (yes/no)? yes.........Moving slot 12281 from 10.0.0.22:6379 to 10.0.0.23:6379: Moving slot 12282 from 10.0.0.22:6379 to 10.0.0.23:6379: Moving slot 12283 from 10.0.0.22:6379 to 10.0.0.23:6379: Moving slot 12284 from 10.0.0.22:6379 to 10.0.0.23:6379: Moving slot 12285 from 10.0.0.22:6379 to 10.0.0.23:6379: Moving slot 12286 from 10.0.0.22:6379 to 10.0.0.23:6379: Moving slot 12287 from 10.0.0.22:6379 to 10.0.0.23:6379:
查看槽位分配最终结果
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.23:6379 (0e504f59...) -> 0 keys | 4096 slots | 0 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 4096 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:1365-5460 (4096 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:12288-16383 (4096 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots:0-1364,5461-6826,10923-12287 (4096 slots) master 0 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:6827-10922 (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
将 10.0.0.33 添加至集群中，并设置为 10.0.0.23 的从节点
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb add-node --slave --master-id 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.33:6379 10.0.0.20:6379>>> Adding node 10.0.0.33:6379 to cluster 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:1365-5460 (4096 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:12288-16383 (4096 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots:0-1364,5461-6826,10923-12287 (4096 slots) master 0 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:6827-10922 (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.>>> Send CLUSTER MEET to node 10.0.0.33:6379 to make it join the cluster.Waiting for the cluster to join.>>> Configure node as replica of 10.0.0.23:6379.[OK] New node added correctly.[root@Redis-Ubuntu1804-node1:~]#
查看集群节点信息
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.23:6379 (0e504f59...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 4096 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:1365-5460 (4096 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:12288-16383 (4096 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379 slots: (0 slots) slave replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots:0-1364,5461-6826,10923-12287 (4096 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:6827-10922 (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
4.2、集群维护之动态缩容
当生产环境集群中的某台服务器存在故障风险 或 集群性能闲置需要对集群节点进行服务器替换或资源回收操作时，需要涉及到动态缩容操作
下线Master节点10.0.0.23 及对应的 Slave节点 10.0.0.33
del-node host:port node_id
4.2.1、Redis 5 集群节点缩容查看当前集群状态
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.23:6379 (ac947d4f...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 0 keys | 4096 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[1365-5460] (4096 slots) master 1 additional replica(s)S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379 slots: (0 slots) slave replicates ac947d4fc970b79c2364ce305441906b462f8b65S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
将删除Master节点的槽位分配至其他可用Master节点
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[1365-5460] (4096 slots) master 1 additional replica(s)S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379 slots: (0 slots) slave replicates ac947d4fc970b79c2364ce305441906b462f8b65S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 1365What is the receiving node ID? 6b142fe3438c3ea488c34366e5ea5a298f89f518Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.Source node #1: ac947d4fc970b79c2364ce305441906b462f8b65Source node #2: doneDo you want to proceed with the proposed reshard plan (yes/no)? yes[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379 slots: (0 slots) slave replicates ac947d4fc970b79c2364ce305441906b462f8b65S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[6827-10922] (4096 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots:[5461-6826],[10923-12287] (2731 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 1366What is the receiving node ID? a4b387d82ac06e5cbf212e48c781e1c27b50320fPlease enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.Source node #1: ac947d4fc970b79c2364ce305441906b462f8b65Source node #2: doneDo you want to proceed with the proposed reshard plan (yes/no)? yes[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379 slots: (0 slots) slave replicates ac947d4fc970b79c2364ce305441906b462f8b65S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379 slots: (0 slots) slave replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379 slots:[10923-12287] (1365 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots:[12288-16383] (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 1365What is the receiving node ID? 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.Source node #1: ac947d4fc970b79c2364ce305441906b462f8b65Source node #2: doneDo you want to proceed with the proposed reshard plan (yes/no)? yes
查看当前槽位分配结果
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.10.0.0.23:6379 (ac947d4f...) -> 0 keys | 0 slots | 0 slaves.10.0.0.30:6379 (5c9789e1...) -> 0 keys | 5461 slots | 2 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]#
删除下线节点
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007712000 15 connected07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007713000 15 connecteda4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682007713079 14 connected 5461-10922ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379@16379 master - 0 1682007710000 12 connected6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682007711000 14 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682007712000 13 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682007714086 15 connected 10923-163836b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1682007709000 13 connected 0-5460[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster del-node 10.0.0.20:6379 ac947d4fc970b79c2364ce305441906b462f8b65>>> Removing node ac947d4fc970b79c2364ce305441906b462f8b65 from cluster 10.0.0.20:6379>>> Sending CLUSTER FORGET messages to the cluster...>>> SHUTDOWN the node.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster del-node 10.0.0.20:6379 829eef61e20def1a36e02a99f1650c9267be2108>>> Removing node 829eef61e20def1a36e02a99f1650c9267be2108 from cluster 10.0.0.20:6379>>> Sending CLUSTER FORGET messages to the cluster...>>> SHUTDOWN the node.
查看缩容后的集群状态
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682008285000 15 connecteda4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682008285000 14 connected 5461-109226a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682008286181 14 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682008283157 13 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682008285173 15 connected 10923-163836b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1682008283000 13 connected 0-5460[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 0 keys | 5461 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]#
清理下线节点node配置
被移除集群的redis节点进程将被自动关闭，需要先清理原node配置后，重新启动服务，否则节点将误以为自身仍在集群中
[root@Redis-Ubuntu1804-node4:~]# cat /app/redis/data/nodes-6379.conf 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 master - 0 1682007475000 13 connected 0-54606a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682007476000 14 connected829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007474000 15 connectedac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379@16379 myself,master - 0 1682007470000 12 connecteda4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682007475000 14 connected 5461-109225c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682007477017 15 connected 10923-1638307a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007476000 15 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682007476010 13 connectedvars currentEpoch 15 lastVoteEpoch 0[root@Redis-Ubuntu1804-node4:~]# rm -rf /app/redis/data/nodes-6379.conf
4.2.2、Redis 3/4 集群节点缩容查看当前集群状态
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.23:6379 (0e504f59...) -> 0 keys | 4096 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 4096 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:1365-5460 (4096 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:12288-16383 (4096 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379 slots: (0 slots) slave replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots:0-1364,5461-6826,10923-12287 (4096 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:6827-10922 (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
将删除Master节点的槽位分配至其他可用Master节点
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:1365-5460 (4096 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:12288-16383 (4096 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379 slots: (0 slots) slave replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots:0-1364,5461-6826,10923-12287 (4096 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:6827-10922 (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 1365What is the receiving node ID? 1c2fee65ee50cdf66c862d04710672286b305f55Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.Source node #1:0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aSource node #2:doneDo you want to proceed with the proposed reshard plan (yes/no)? yes[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:12288-16383 (4096 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379 slots: (0 slots) slave replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots:5461-6826,10923-12287 (2731 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:6827-10922 (4096 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 1366What is the receiving node ID? 337bf2a15dd9b0ae8caaee08c5ec41beb754b78ePlease enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.Source node #1:0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aSource node #2:doneDo you want to proceed with the proposed reshard plan (yes/no)? yes[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:12288-16383 (4096 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379 slots: (0 slots) slave replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots:10923-12287 (1365 slots) master 1 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.How many slots do you want to move (from 1 to 16384)? 1365What is the receiving node ID? 709b09b96f19209e7f0a1e3cfc5c035678491e4bPlease enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs.Source node #1:0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462aSource node #2:doneDo you want to proceed with the proposed reshard plan (yes/no)? yes
查看当前槽位分配结果
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 2 slaves.10.0.0.23:6379 (0e504f59...) -> 0 keys | 0 slots | 0 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 4 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 2 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379 slots: (0 slots) master 0 additional replica(s)S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#
删除下线节点
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb del-node 10.0.0.20:6379 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a>>> Removing node 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a from cluster 10.0.0.20:6379>>> Sending CLUSTER FORGET messages to the cluster...>>> SHUTDOWN the node.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb del-node 10.0.0.20:6379 6e761ee02c9c17838e4643b9030ddfe327f1c742>>> Removing node 6e761ee02c9c17838e4643b9030ddfe327f1c742 from cluster 10.0.0.20:6379>>> Sending CLUSTER FORGET messages to the cluster...>>> SHUTDOWN the node.[root@Redis-Ubuntu1804-node1:~]#
查看缩容后的集群状态
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.
4.3、集群维护之数据导入
当企业Redis服务由 原先的单机运行或主从架构 升级为Cluster模式后，需要将原数据迁移至当前Cluster集群中，需要使用集群数据导入
从历史Redis服务器node8 10.0.0.33 上将数据导入至当前集群中
[root@Redis-Ubuntu1804-node8:~]# redis-cli -a redis --no-auth-warning info keyspace# Keyspacedb0:keys=20000,expires=0,avg_ttl=0[root@Redis-Ubuntu1804-node8:~]# redis-cli -a redis --no-auth-warning get key_2931"value-2931"[root@Redis-Ubuntu1804-node8:~]#*集群导入数据需要确保新导入数据不能与Redis Cluster已有数据出现重复的key名称，否则将导致导入失败或中断（建议在导入数据前清空当前集群数据）
import host:port
–cluster-from ##外部redis的连接地址
–cluster-copy ##单独使用时，导入数据与当前集群中key值不可冲突
–cluster-replace ##强制替换key值
4.3.1、在 Redis 5 上执行数据导入清理集群和外部redis所有节点密码
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster call 10.0.0.20:6379 config set requirepass ''>>> Calling config set requirepass 10.0.0.20:6379: OK10.0.0.22:6379: OK10.0.0.21:6379: OK10.0.0.32:6379: OK10.0.0.31:6379: OK10.0.0.30:6379: OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.33 --no-auth-warning config set requirepass ''OK
执行数据导入
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --cluster info 10.0.0.20:6379Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.10.0.0.20:6379 (6b142fe3...) -> 1 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 1 keys | 5462 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 2 keys | 5461 slots | 1 slaves.[OK] 4 keys in 3 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli --cluster import 10.0.0.20:6379 --cluster-from 10.0.0.33:6379 --cluster-copy --cluster-replace ... ... ... Migrating key_7867 to 10.0.0.20:6379: OK Migrating key_6810 to 10.0.0.20:6379: OK Migrating key_1232 to 10.0.0.21:6379: OK Migrating key_10171 to 10.0.0.21:6379: OK Migrating key_15561 to 10.0.0.30:6379: OK Migrating key_13723 to 10.0.0.21:6379: OK Migrating key_18761 to 10.0.0.21:6379: OK Migrating key_14471 to 10.0.0.20:6379: OK Migrating key_12496 to 10.0.0.30:6379: OK Migrating key_10559 to 10.0.0.21:6379: OK Migrating key_15282 to 10.0.0.21:6379: OK Migrating key_19021 to 10.0.0.21:6379: OK Migrating key_12831 to 10.0.0.21:6379: OK Migrating key_13260 to 10.0.0.20:6379: OK Migrating key_15767 to 10.0.0.30:6379: OK
查看数据
[root@Redis-Ubuntu1804-node1:~]# redis-cli --cluster info 10.0.0.20:637910.0.0.20:6379 (6b142fe3...) -> 6662 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 6666 keys | 5462 slots | 1 slaves.10.0.0.30:6379 (5c9789e1...) -> 6676 keys | 5461 slots | 1 slaves.[OK] 20004 keys in 3 masters.1.22 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning get key_2981"value-2981"[root@Redis-Ubuntu1804-node1:~]#
4.3.2、在 Redis 3/4 上执行数据导入清理集群和外部redis所有节点密码
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb call 10.0.0.20:6379 config set requirepass ''>>> Calling CONFIG set requirepass 10.0.0.20:6379: OK10.0.0.31:6379: OK10.0.0.22:6379: OK10.0.0.32:6379: OK10.0.0.30:6379: OK10.0.0.21:6379: OK[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.33 config set requirepass ''OK[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379[ERR] Sorry, can't connect to node 10.0.0.20:6379
##清理 redis-trib.rb 工具配置的密码
[root@Redis-Ubuntu1804-node1:~]# sed -i 's/@r = Redis.new.*/@r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60)/' /usr/bin/redis-trib.rb[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]#
执行数据导入
由于gem redis 版本与客户端版本不一致出现数据导入错误Unsupported command argument type: NilClass (TypeError)
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb import --from 10.0.0.33:6379 --copy --replace 10.0.0.20:6379>>> Importing data from 10.0.0.33:6379 to cluster >>> Performing Cluster Check (using node 10.0.0.20:6379)M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379 slots:0-5460 (5461 slots) master 1 additional replica(s)S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379 slots: (0 slots) slave replicates 1c2fee65ee50cdf66c862d04710672286b305f55M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379 slots: (0 slots) slave replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78eS: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379 slots: (0 slots) slave replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4bM: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.>>> Connecting to the source Redis instance*** Importing 20000 keys from DB 0Traceback (most recent call last): 11: from /usr/bin/redis-trib.rb:1830:in `' 10: from /usr/bin/redis-trib.rb:1604:in `import_cluster_cmd' 9: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis/commands/keys.rb:26:in `scan' 8: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis/commands/keys.rb:433:in `_scan' 7: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis.rb:166:in `send_command' 6: from /usr/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize' 5: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis.rb:167:in `block in send_command' 4: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis/client.rb:73:in `call_v' 3: from /var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client.rb:223:in `call_v' 2: from /var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client/command_builder.rb:68:in `generate' 1: from /var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client/command_builder.rb:68:in `map!'/var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client/command_builder.rb:75:in `block in generate': Unsupported command argument type: NilClass (TypeError)
gem卸载当前版本 redis工具，重新安装4.0版本gem
[root@Redis-Ubuntu1804-node1:~]# gem uninstall redisSuccessfully uninstalled redis-5.0.6[root@Redis-Ubuntu1804-node1:~]# wget https://rubygems.org/downloads/redis-4.0.3.gem--2023-04-21 03:36:14-- https://rubygems.org/downloads/redis-4.0.3.gemResolving rubygems.org (rubygems.org)... 151.101.1.227, 151.101.65.227, 151.101.129.227, ...Connecting to rubygems.org (rubygems.org)|151.101.1.227|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 116224 (114K) [application/octet-stream]Saving to: ‘redis-4.0.3.gem’redis-4.0.3.gem 100%[===================================================================>] 113.50K --.-KB/s in 0.1s 2023-04-21 03:36:14 (1.11 MB/s) - ‘redis-4.0.3.gem’ saved [116224/116224][root@Redis-Ubuntu1804-node1:~]# gem install redis-4.0.3.gem Successfully installed redis-4.0.3Parsing documentation for redis-4.0.3Installing ri documentation for redis-4.0.3Done installing documentation for redis after 0 seconds1 gem installed[root@Redis-Ubuntu1804-node1:~]#
由于gem版本过高导致出现ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
Migrating key_2307 to 10.0.0.22:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)Migrating key_5080 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)Migrating key_15899 to 10.0.0.21:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)Migrating key_6538 to 10.0.0.22:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)Migrating key_1665 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)Migrating key_14837 to 10.0.0.21:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)Migrating key_1757 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)Migrating key_13190 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
卸载当前版本，安装gem-redis 3.5 版本
[root@Redis-Ubuntu1804-node1:~]# gem uninstall redisSuccessfully uninstalled redis-4.0.3[root@Redis-Ubuntu1804-node1:~]# wget https://rubygems.org/downloads/redis-3.3.5.gem--2023-04-21 03:46:33-- https://rubygems.org/downloads/redis-3.3.5.gemResolving rubygems.org (rubygems.org)... 151.101.193.227, 151.101.129.227, 151.101.65.227, ...Connecting to rubygems.org (rubygems.org)|151.101.193.227|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 92672 (90K) [application/octet-stream]Saving to: ‘redis-3.3.5.gem’redis-3.3.5.gem 100%[===================================================================>] 90.50K 497KB/s in 0.2s 2023-04-21 03:46:34 (497 KB/s) - ‘redis-3.3.5.gem’ saved [92672/92672][root@Redis-Ubuntu1804-node1:~]# gem install redis-3.3.5.gem Successfully installed redis-3.3.5Parsing documentation for redis-3.3.5Installing ri documentation for redis-3.3.5Done installing documentation for redis after 0 seconds1 gem installed
执行数据导入
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb import --from 10.0.0.33:6379 --copy --replace 10.0.0.20:6379.........Migrating key_19321 to 10.0.0.20:6379: OKMigrating key_3394 to 10.0.0.20:6379: OKMigrating key_11273 to 10.0.0.22:6379: OKMigrating key_15899 to 10.0.0.21:6379: OKMigrating key_14837 to 10.0.0.21:6379: OKMigrating key_1665 to 10.0.0.20:6379: OKMigrating key_5080 to 10.0.0.20:6379: OKMigrating key_13190 to 10.0.0.20:6379: OKMigrating key_1757 to 10.0.0.20:6379: OKMigrating key_6538 to 10.0.0.22:6379: OKMigrating key_2307 to 10.0.0.22:6379: OK[root@Redis-Ubuntu1804-node1:~]#
查看数据
[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:637910.0.0.20:6379 (1c2fee65...) -> 6661 keys | 5461 slots | 1 slaves.10.0.0.22:6379 (709b09b9...) -> 6674 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (337bf2a1...) -> 6665 keys | 5462 slots | 1 slaves.[OK] 20000 keys in 3 masters.1.22 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -c get key_12332"value-12332"[root@Redis-Ubuntu1804-node1:~]#
4.4、集群偏斜
Redis Cluser 多节点在运行一段时间以后，会出现倾斜现象，某个节点数据偏多，内存消耗更大，或者接收到的用户请求更多，发生偏移现象的可能性，如下：
- 节点和槽位分配不均
- 不同槽位对应的键值数量差异较大
- 包含bigkey（建议少用）
- 内存相关配置不一致
- 热点数据不均衡：一致性不高时，可使用本地缓存或者MQ
#查看指定槽位中的键值数[root@Redis-Ubuntu1804-node1:~]# redis-cli cluster countkeysinslot 0(integer) 3[root@Redis-Ubuntu1804-node1:~]# redis-cli cluster countkeysinslot 16383(integer) 0[root@Redis-Ubuntu1804-node1:~]# redis-cli cluster countkeysinslot 1(integer) 2[root@Redis-Ubuntu1804-node1:~]#
#槽位自动平衡分布[root@Redis-Ubuntu1804-node1:~]# redis-trib.rb rebalance 10.0.0.20:6379>>> Performing Cluster Check (using node 10.0.0.20:6379)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.*** No rebalancing needed! All nodes are within the 2.0% threshold.[root@Redis-Ubuntu1804-node1:~]#
#查找bigkeys,建议在slave上执行[root@Redis-Ubuntu1804-node1:~]# redis-cli --bigkeys# Scanning the entire keyspace to find biggest keys as well as# average sizes per key type. You can use -i 0.1 to sleep 0.1 sec# per 100 SCAN commands (not usually needed).[00.00%] Biggest string found so far 'key_13312' with 11 bytes-------- summary -------Sampled 6661 keys in the keyspace!Total key length in bytes is 56245 (avg len 8.44)Biggest string found 'key_13312' has 11 bytes6661 strings with 69567 bytes (100.00% of keys, avg size 10.44)0 lists with 0 items (00.00% of keys, avg size 0.00)0 sets with 0 members (00.00% of keys, avg size 0.00)0 hashs with 0 fields (00.00% of keys, avg size 0.00)0 zsets with 0 members (00.00% of keys, avg size 0.00)0 streams with 0 entries (00.00% of keys, avg size 0.00)[root@Redis-Ubuntu1804-node1:~]#
四、集群操作自动化1、初始环境自动化配置脚本-shell Ubuntu编译安装
[root@Client-Ubuntu-1804-250:~/script/redis]# cat redis_install.sh #!/bin/bash##********************************************************************#Author: janzen#Date: 2023-04-13#FileName： redis_install.sh#Description： Install Redis#Copyright (C): 2023 All rights reserved#********************************************************************DIR='/app/redis'DIR_S='\/app\/redis'PASSWD='redis'PORT=()echo -e "启用架构：\n 1、single node \n 2、Replicae_node \n 3、Cluster_node\n"read -p "选择节点类型：" node_typeread -p "创建的副本数：" numcase $num in 0) echo 不可创建0副本 exit ;; [1-9]*) echo 创建$num副本 ;; *) echo $num 不是数字 exit ;;esacfor i in $(seq $num); do rds_port=`echo $i-1+6379|bc` PORT+=($rds_port)donewget https://download.redis.io/releases/redis-5.0.14.tar.gztar xf redis-5.0.14.tar.gzuseradd redis -s /sbin/nologinmkdir $DIR/{etc,log,data,run} -pchown redis.redis $DIR/ -Rapt-get install build-essential tree -ycd redis-5.0.14make PREFIX=$DIR installln -s $DIR/bin/* /usr/bin/echo net.core.somaxconn=1024 >> /etc/sysctl.confecho vm.overcommit_memory=1 >> /etc/sysctl.confsysctl -pecho never > /sys/kernel/mm/transparent_hugepage/enabledfor i in ${PORT[*]};do case $node_type in 1) sed -e 's/^bind 127.0.0.1/bind 0.0.0.0/' -e 's/^port 6379/port '"$i"'/' -e 's/^pidfile \/var\/run\/redis_6379.pid/pidfile '"$DIR_S"'\/run\/redis_'"$i"'.pid/' -e 's/^logfile ""/logfile "'"$DIR_S"'\/log\/redis_'"$i"'.log"/g' -e 's/^always-show-logo yes/always-show-logo no/g' -e 's/^appendonly no/appendonly yes/' -e 's/^appendfilename "appendonly.aof"/appendfilename "appendonly_'"$i"'.aof"/g' -e 's/^dbfilename dump.rdb/dbfilename dump_'"$i"'.rdb/g' -e 's/^dir \.\//dir '"$DIR_S"'\/data/g' -e 's/^# masterauth $/masterauth '"$PASSWD"'/' -e 's/^# requirepass foobared/requirepass '"$PASSWD"'/' redis.conf > $DIR/etc/redis_$i.conf ;; 2) read -p "node $i Master： : " Master sed -e 's/^bind 127.0.0.1/bind 0.0.0.0/' -e 's/^port 6379/port '"$i"'/' -e 's/^pidfile \/var\/run\/redis_6379.pid/pidfile '"$DIR_S"'\/run\/redis_'"$i"'.pid/' -e 's/^logfile ""/logfile "'"$DIR_S"'\/log\/redis_'"$i"'.log"/g' -e 's/^always-show-logo yes/always-show-logo no/g' -e 's/^appendonly no/appendonly yes/' -e 's/^appendfilename "appendonly.aof"/appendfilename "appendonly_'"$i"'.aof"/g' -e 's/^dbfilename dump.rdb/dbfilename dump_'"$i"'.rdb/g' -e 's/^dir \.\//dir '"$DIR_S"'\/data/g' -e 's/^# masterauth $/masterauth '"$PASSWD"'/' -e 's/^# requirepass foobared/requirepass '"$PASSWD"'/' -e '/# replicaof/a replicaof '"$Master"'' redis.conf > $DIR/etc/redis_$i.conf ;; 3) sed -e 's/^bind 127.0.0.1/bind 0.0.0.0/' -e 's/^port 6379/port '"$i"'/' -e 's/^pidfile \/var\/run\/redis_6379.pid/pidfile '"$DIR_S"'\/run\/redis_'"$i"'.pid/' -e 's/^logfile ""/logfile "'"$DIR_S"'\/log\/redis_'"$i"'.log"/g' -e 's/^always-show-logo yes/always-show-logo no/g' -e 's/^appendonly no/appendonly yes/' -e 's/^appendfilename "appendonly.aof"/appendfilename "appendonly_'"$i"'.aof"/g' -e 's/^dbfilename dump.rdb/dbfilename dump_'"$i"'.rdb/g' -e 's/^dir \.\//dir '"$DIR_S"'\/data/g' -e 's/^# masterauth $/masterauth '"$PASSWD"'/' -e 's/^# requirepass foobared/requirepass '"$PASSWD"'/' -e '/# cluster-enabled yes/a cluster-enabled yes' -e '/# cluster-config-file/a cluster-config-file nodes-'"$i"'.conf' -e '/# cluster-require-full-coverage yes/a cluster-require-full-coverage yes' redis.conf > $DIR/etc/redis_$i.conf ;; *) exit ;; esac cat > /etc/systemd/system/redis_$i.service < /etc/init.d/disable_transparent_hugepage < /sys/kernel/mm/transparent_hugepage/enabledEOFchmod 755 /etc/init.d/disable_transparent_hugepagecd /etc/rcS.d/ln -s ../init.d/disable_transparent_hugepage disable_transparent_hugepagess -ntlps -ef | grep redis-server# reboot
-shell Ubuntu 二进制包安装
#!/bin/bash##********************************************************************#Author: janzen#Date: 2023-04-20#FileName： redis_install_apt.sh#Description： The test script#Copyright (C): 2023 All rights reserved#********************************************************************apt install redis -ysed -i.bak -e 's/bind 127.0.0.1/bind 0.0.0.0/' -e '/masterauth/a masterauth redis' -e '/# requirepass/a requirepass redis' -e '/# cluster-enabled yes/a cluster-enabled yes' -e '/# cluster-config-file/a cluster-config-file nodes-6379.conf' -e '/# cluster-require-full-coverage yes/a cluster-require-full-coverage yes' /etc/redis/redis.confsystemctl restart redisss -ntl
2、批量分配slot槽位
[root@Client-Ubuntu-1804-250:~/script/redis]# cat add_slots.sh #!/bin/bash##********************************************************************#Author: janzen#Date: 2023-04-20#FileName： add_slots.sh#Description： The test script#Copyright (C): 2023 All rights reserved#********************************************************************host=$1port=$2start=$3end=$4passwd=redisfor slot in `seq $start $end`; do redis-cli -h $host -p $port -a $passwd --no-auth-warning cluster addslots $slot && echo $slot add to node $host:$portdone
3、python脚本实现自动创建数据
pip 安装 redis-py-cluster 模块
[root@Client-Ubuntu-1804-250:~/script/redis]# pip3 install redis-py-clusterCollecting redis-py-cluster Downloading https://files.pythonhosted.org/packages/b2/96/153bbcf5dee29b52b2674e77a87ce864d381f72151737317529b7de4f337/redis_py_cluster-2.1.3-py2.py3-none-any.whl (42kB) 100% |████████████████████████████████| 51kB 403kB/s Collecting redis<4.0.0,>=3.0.0 (from redis-py-cluster) Downloading https://files.pythonhosted.org/packages/a7/7c/24fb0511df653cf1a5d938d8f5d19802a88cef255706fdda242ff97e91b7/redis-3.5.3-py2.py3-none-any.whl (72kB) 100% |████████████████████████████████| 81kB 1.5MB/s Installing collected packages: redis, redis-py-cluster Found existing installation: redis 2.10.6 Not uninstalling redis at /usr/lib/python3/dist-packages, outside environment /usrSuccessfully installed redis-3.5.3 redis-py-cluster-2.1.3
python脚本连接集群创建数据
#!/usr/bin/python3# -*- coding: UTF-8 -*-#********************************************************************#Author: janzen#Date: 2023-04-20#FileName： redis_cluster_newData.py#Description： The python3 script#Copyright (C): 2023 All rights reserved#********************************************************************from rediscluster import RedisClusterimport sysnum=int(sys.argv[1])passwd='redis'startup_nodes = [ {"host":"10.0.0.20","port":"6379"}, {"host":"10.0.0.21","port":"6379"}, {"host":"10.0.0.22","port":"6379"}, {"host":"10.0.0.30","port":"6379"}, {"host":"10.0.0.31","port":"6379"}, {"host":"10.0.0.32","port":"6379"} ]redis_conn = RedisCluster(startup_nodes = startup_nodes,password = passwd,decode_responses=True)for i in range(0,num): redis_conn.set("keys%s" % i,"value%s" % i) print("keys%s:" % i,redis_conn.get("keys%s" % i))
#!/usr/bin/python3# -*- coding: UTF-8 -*-#********************************************************************#Author: janzen#Date: 2023-04-20#FileName： redis_cluster_newData.py#Description： The python3 script#Copyright (C): 2023 All rights reserved#********************************************************************from rediscluster import RedisClusterimport sysnum=int(sys.argv[1])passwd='redis'startup_nodes = [ {"host":"10.0.0.20","port":"6379"}, {"host":"10.0.0.21","port":"6379"}, {"host":"10.0.0.22","port":"6379"}, {"host":"10.0.0.30","port":"6379"}, {"host":"10.0.0.31","port":"6379"}, {"host":"10.0.0.32","port":"6379"} ]redis_conn = RedisCluster(startup_nodes = startup_nodes,password = passwd,decode_responses=True)while 1: time.sleep(1) try: redis_conn.set("keys%s" % num,"value%s" % num) print("set data %s:%s" % (num,redis_conn.get("keys%s" % num))) except Exception as err: print(err) continue num+=1
python脚本连接集群读取数据
#!/usr/bin/python3# -*- coding: UTF-8 -*-#********************************************************************#Author: janzen#Date: 2023-04-20#FileName： redis_cluster_newData.py#Description： The python3 script#Copyright (C): 2023 All rights reserved#********************************************************************from rediscluster import RedisClusterimport sys,timenum=int(sys.argv[1])passwd='redis'startup_nodes = [ {"host":"10.0.0.20","port":"6379"}, {"host":"10.0.0.21","port":"6379"}, {"host":"10.0.0.22","port":"6379"}, {"host":"10.0.0.30","port":"6379"}, {"host":"10.0.0.31","port":"6379"}, {"host":"10.0.0.32","port":"6379"} ]redis_conn = RedisCluster(startup_nodes = startup_nodes,password = passwd,decode_responses=True)while 1: time.sleep(1) try: print("get data %s:%s" % (str(num),redis_conn.get("keys"+str(num)))) except Exception as err: print(err) continue
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c –no-auth-warning –cluster check 10.0.0.20:6379Could not connect to Redis at 10.0.0.22:6379: Connection refused*** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.10.0.0.20:6379 (6b142fe3…) -> 131 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8…) -> 126 keys | 5462 slots | 1 slaves.[OK] 257 keys in 2 masters.0.02 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s)M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379 slots: (0 slots) slave replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379 slots: (0 slots) slave replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379 slots: (0 slots) slave replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3[OK] All nodes agree about slots configuration.>>> Check for open slots…>>> Check slots coverage…[ERR] Not all 16384 slots are covered by nodes.
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c –no-auth-warning –cluster info 10.0.0.20:6379Could not connect to Redis at 10.0.0.22:6379: Connection refused*** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.10.0.0.20:6379 (6b142fe3…) -> 131 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8…) -> 126 keys | 5462 slots | 1 slaves.[OK] 257 keys in 2 masters.0.02 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c –no-auth-warning cluster nodes07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master,fail – 1682000024732 1682000022415 3 disconnecteda4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master – 0 1682000042000 2 connected 5461-109226a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682000040567 6 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682000042582 5 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master – 0 1682000041575 7 connected 10923-163836b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master – 0 1682000041000 1 connected 0-5460[root@Redis-Ubuntu1804-node1:~]#
[root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis –versionredis-cli 5.0.14[root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis –no-auth-warning cluster nodese603091426e2ce29629ca51fb5e7dafb2f0c9524 :6379@16379 myself,master – 0 0 0 connected[root@Redis-Ubuntu1804-node4:~]#