Collecting Logs for Cluster Issues

ICS can be deployed as a cluster pair in Active/Passive Mode or Active/Active Mode.

In Active/Passive Mode

•One ICS actively serves user requests

•The other ICS runs passively to synchronize state data

•Each node in a cluster is in constant communication with one another

•User requests to the cluster VIP are passed to the active ICS

•If the active ICS goes offline, the standby ICS automatically starts servicing user requests

•In Active/Active mode, all the machines in the cluster actively handle user requests sent by an external load balancer.

When there are issues with the cluster, use the following steps to enable and collect the required logs:

1.Enable the Node monitoring option on both the ICS nodes.

This option is available in ICS admin GUI under Maintenance > Troubleshooting > Monitoring > Nodemonitoring.

Log Settings:

•Select the Nodemonitoring enabled check box and save changes.

•Turn on the Debug log with the settings as mentioned in step 3.

2.Enable the Group Communication option in both the ICS nodes.

The option is available in ICS admin GUI under Maintenance > Troubleshooting > Monitoring > Clustering > Group Communication.

Enable the Access Group Communication option and configure the log settings as:

•Maximum statistics log size = 5 MB

•Monitoring Interval = 10 Seconds

3.Enable the Debuglog option on both the ICS nodes.

This option is available in ICS admin GUI under: Maintenance > Troubleshooting > Monitoring > Debuglog.

Log Settings (ICS):

•Select the Enable Debuglog check box.

•Save changes.

Debug log level is 50.

Following is the list of process names:

•dsnet

•dscluster

•dszserverd

•dssys

•bootcluster

•dsclusinfod

•gmi

4.Start the TCP Dump from the Internal port of ICS by clicking the Start Sniffing option.

Note: This option is available in ICS admin GUI under: Maintenance > Troubleshooting > Tools > TCP Dump.

After replicating the issue, disable the TCP Dump on both of the nodes by clicking the Stop Sniffing option.

5.Access System snapshot option.

Available in ICS admin GUI. After configuring the System Snapshot options, click Take Snapshot. This will generate snapshot with the name: Admin generated snapshot.

Important: Take snapshots from both the ICS nodes of the cluster by following the above procedure. Make sure you have logged on to the ICS using the physical IP of the respective nodes to enable logging and capture logs.

After taking the System Snapshot, disable the Monitoring options including debug log monitoring.

Data Collection

Admin can enable logs, tcpdump using Troubleshooting > Log Selection page.

Collect the following data:

•Logs

•System Snapshot taken from both nodes

•TCP dump captured from both nodes

•Event/User Access/Admin logs using the "save all logs" option in ICS/IPS admin GUI under System > Log/Monitoring

•Screenshot of the Clustering Status page available under System > Clustering

•Screenshot of the System Status page available under System > Status

•Time stamp of the Issue