Overview
You may experience the issue where your cluster has been degraded and there's a failed CE. In the Grid system (version 5), the Cluster will be degraded if one of the Cache Engines fails.
Solution
To solve this issue please create a support ticket. And one of our agents will help you out.
<supportagent>
1) Login to management server with SSH
2) Change to root privilege with "su" command
3) Login to pang-cli
#su admin
4) Show the systemid
#show systemid
5) Enter enable mode
#enable
Note: use the systemid (*1234567) as password
6) Check System Status
#show status
Cluster state: r-degraded Server Slot Status Operational state Device state Administrative state ce-1 powered on enabled started unlocked ce-2 N/A N/A stopped N/A ce-3 powered on enabled started unlocked ce-4 powered on enabled started unlocked
CE-2 status is N/A
7) Exist from pang-cli
#quit
Note: In the following steps CE-2 is an example, you should replace it with the failed CE number)
8) Try to ping the failed CE, for example, if it was CE-2, try to ping it and the BMC of CE-2 from the management server (BMC IP is the iDRAC IP of CE-2)
Note: Internal IP address of CE and iDRAC can be found on /etc/hosts file
#ping CE-2
#ping BMC-2 (or) #ping 10.11.12.102
9) In this case, both pings fail, contact the customer's onsite engineer and let him remove the power cord and power supply from CE-2, then wait 5 mins and put them back and power on the machine
10) Try to ping both BMC and CE-2 from the management server again.
11) After ping is replied, Login to pang-cli (follow above step 3 to 5) and start the service on CE-2
#oper server 2 #start
12) Verify the service status
#show status
13) Verify the service status from UBview
https://management-ip/ui/status/logical-view
To check the log on iDRAC of the failed CE and find out why it happens.
- Launch the vnc session in the management server and connect the management server with vncviewer.
#vncserver :1
- Click Continue if you get an Encryption warning pop-up message
- Input the Root Access password for the Management Server
- After connecting to the management server with vncviewer, enter "opera" command to launch the opera browser
- On the Browser, open the failed CE iDRAC (in this example CE-2).
https://bmc-2 or https://10.11.12.102
- Login to IDRAC with root user and password
- Check the system Events log and verify the issue.
</supportagent>