FAQ

Q: 無法啟動 node 服務 
 
 Application rabbit exited with reason: {{could_not_write_file,"/var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq982/cluster_nodes.config",enospc},{rabbit,start,[normal,[]]}} 
 
 Solution: 
 可能是磁碟空間使用爆了，移除目錄 /var/lib/rabbitmq/mnesia 底下 node 資料子目錄。 
 [root@tpeeaprmq981 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.8G 0 1.8G 0% /dev
tmpfs 1.8G 4.0K 1.8G 1% /dev/shm
tmpfs 1.8G 24M 1.8G 2% /run
tmpfs 1.8G 0 1.8G 0% /sys/fs/cgroup
/dev/mapper/rootvg-rootlv 9.0G 4.0G 5.1G 45% /
/dev/sda2 1014M 344M 671M 34% /boot
/dev/sda1 599M 5.8M 594M 1% /boot/efi
/dev/mapper/rootvg-mqdatalv 5.0G 5.0G 20K 100% /var/lib/rabbitmq
/dev/mapper/rootvg-homelv 507M 30M 478M 6% /home
/dev/mapper/rootvg-worktmp 507M 46M 462M 9% /worktmp
/dev/mapper/rootvg-optlv 2.0G 997M 1.1G 49% /opt
tmpfs 364M 0 364M 0% /run/user/0
[root@tpeeaprmq981 ~]#
[root@tpeeaprmq981 ~]#
[root@tpeeaprmq981 ~]# du -csh /var/lib/rabbitmq/mnesia/*
204K /var/lib/rabbitmq/mnesia/rabbit@rmq981
4.0K /var/lib/rabbitmq/mnesia/rabbit@rmq981-feature_flags
0 /var/lib/rabbitmq/mnesia/rabbit@rmq981-plugins-expand
300K /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq98
5.0G /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981
4.0K /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981-feature_flags
0 /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981-plugins-expand
4.0K /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq98-feature_flags
0 /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq98-plugins-expand
5.0G total

[root@tpeeaprmq981 ~]# rm -rf /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981
[root@tpeeaprmq981 ~]# rm /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981-feature_flags
rm: remove regular file '/var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981-feature_flags'? y
[root@tpeeaprmq981 ~]#
[root@tpeeaprmq981 ~]# rm /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981-plugins-expand
rm: cannot remove '/var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981-plugins-expand': Is a directory
[root@tpeeaprmq981 ~]# rm -rf /var/lib/rabbitmq/mnesia/rabbit@tpeeaprmq981-plugins-expand
 
 Q: 無法加入 Cluster 
 
 [error] Node rabbit@tpeeaprmq98 thinks it's clustered with node rabbit@tpeeaprmq982, but rabbit@tpeeaprmq982 disagrees 
 
 Solution: 
 到 node rabbit@tpeeaprmq98 執行 rabbitmqctl cluster_status ，如果有顯示 node rabbit@tpeeaprmq982 ，執行強制移除指令。 
 # On the node rabbit@tpeeaprmq98
rabbitmqctl forget_cluster_node rabbit@tpeeaprmq982 
 Q: Network partition detected 
 Web UI 出現告警訊息： 
 
 Node 執行 rabbitmqctl cluster_status 出現 Network Partitions 
 Network Partitions

Node rabbit@tpeeaprmq98 cannot communicate with rabbit@tpeeaprmq982
Node rabbit@tpeeaprmq981 cannot communicate with rabbit@tpeeaprmq982 
 原因：tpeeaprmq982 由於硬體或網路異常造成意外的離線，當 node 重新恢復網路連線後，Cluster 會觸發 Network Partition 事件(aka split-brain 腦裂事件)，且停止 quorum queue 的資料複寫，必須盡速完成修復。 
 解決方案：修復 Network Partition 異常事件 
 
 官方教學: Clustering and Network Partitions — RabbitMQ 
 
 重啟發生問題 Node tpeeaprmq982 的服務 
 rabbitmqctl stop
systemctl start rabbitmq-server