Overview

You may experience the issue where your cluster has been degraded and there's a failed CE. In the Grid system (version 5), the Cluster will be degraded if one of the Cache Engines fails. 

 

Solution

To solve this issue please create a support ticket. And one of our agents will help you out.

 

<supportagent>

1) Login to management server with SSH

2) Change to root privilege with "su" command

3) Login to pang-cli

#su admin

4) Show the systemid

#show systemid

5) Enter enable mode

#enable 

Note: use the systemid (*1234567) as password 

6) Check System Status

#show status

Cluster state: r-degraded

  Server Slot   Status        Operational state    Device state   Administrative state
  ce-1          powered on    enabled              started        unlocked
  ce-2          N/A           N/A                  stopped        N/A
  ce-3          powered on    enabled              started        unlocked
  ce-4          powered on    enabled              started        unlocked

CE-2 status is N/A

7) Exist from pang-cli

#quit

Note: In the following steps CE-2 is an example, you should replace it with the failed CE number)

8) Try to ping the failed CE, for example, if it was CE-2, try to ping it and the BMC of CE-2 from the management server (BMC IP is the iDRAC IP of CE-2)

Note: Internal IP address of CE and iDRAC can be found on /etc/hosts file

#ping CE-2

#ping BMC-2 (or) #ping 10.11.12.102

9) In this case, both pings fail,  contact the customer's onsite engineer and let him remove the power cord and power supply from CE-2, then wait 5 mins and put them back and power on the machine

10) Try to ping both BMC and CE-2 from the management server again.

11) After ping is replied, Login to pang-cli (follow above step 3 to 5) and start the service on CE-2

#oper server 2
#start

12) Verify the service status

#show status

13) Verify the service status from UBview

https://management-ip/ui/status/logical-view

 

To check the log on iDRAC of the failed CE and find out why it happens.

• Launch the vnc session in the management server and connect the management server with vncviewer.

#vncserver :1

• Click Continue if you get an Encryption warning pop-up message

• Input the Root Access password for the Management Server

• After connecting to the management server with vncviewer, enter "opera" command to launch the opera browser
• On the Browser, open the failed CE iDRAC (in this example CE-2).

  https://bmc-2 or
  https://10.11.12.102

• Login to IDRAC with root user and password
• Check the system Events log and verify the issue.

 

</supportagent>