Troubleshooting Clusters

When you have problems with cluster communication, you may be directed by your Ivanti Support representative to use the cluster node troubleshooting tools.

To use the cluster node troubleshooting tools:

From the admin console, select Maintenance > Troubleshooting > Monitoring > Node Monitor, in Maintenance > Troubleshooting > Clustering Network Connectivity, and in Maintenance > Troubleshooting > Clustering Group Communication.

You can use a built-in feature on the clustering Status page to identify the status of each cluster node. Pause the mouse pointer over the Status light icon and the system displays a tool tip containing a hexadecimal number. The hexadecimal number is a snapshot of the status of the Connect Secure. It is a bit mask indicating a number of states as shown in Table.

Cluster Status

Value

Meaning

0x000001

Connect Secure is in standalone mode.

0x000002

Connect Secure is in cluster disabled state.

0x000004

Connect Secure is in cluster enabled state.

0x000008

Unable to communicate (because it is offline, has wrong password, has different cluster definition, different version, or a related problem).

0x00002000

The node owns the VIPs (on) or not (off).

0x000100

Connect Secure is syncing state from another Ivanti Connect Secure (initial syncing phase).

0x000200

Connect Secure is transitioning from one state to another.

0x00020000

The group communication subsystems at the local and remote nodes are disconnected from each other.

0x00040000

Management interface (mgt0) appears disconnected.

0x00080000

Management gateway is unreachable for ARP ping.

0x000800

Connect Secure int0 appears disconnected (no carrier).

0x001000

This node is configured to be a cluster member.

0x002000

Connect Secure is syncing its state to another Connect Secure that is joining.

0x004000

Initial Synchronization as master or slave is taking place.

0x008000

This Connect Secure is the leader of the cluster.

0x010000

The group communication subsystem is functional.

0x020000

The gateway on int0 is unreachable for ARP pings (see log file).

0x040000

The gateway on int1 is unreachable for ARP pings (see log file).

0x080000

Leader election is taking place.

0x100000

Server life cycle process (dsmon) is busy.

0x200000

System performs post state synchronization activities.

0x30004

"The group communication subsystem is functional.

The gateway on int0 is unreachable for ARP pings (see log file).

Connect Secure is in cluster enabled state.

0x80000000

Cluster keystore or security world has not been associated with the FIPS card.

Each code, as you see it in the Connect Secure, may relate specifically to one state. However, each code may represent a combination of states, and so the actual code does not appear in Table. Instead, the code you see in the Connect Secure is the sum of several of the hexadecimal numbers shown in Table. You will need to factor out the codes, as in the following example:

0x38004 - The right-most digit (4) in this hexadecimal number corresponds to:

0x000004 - The Ivanti Connect Secure is in cluster enabled state.

0x038004 - The digit in the fourth position from the right (8) corresponds to:

0x008000 - This Connect Secure is the leader of the cluster.

0x38004 - The left-most digit (3) in this hexadecimal number does not exist in the table, which indicates that it corresponds to the sum of two other digits, in this case, 1 and 2, as shown in the following codes:

0x020000 - The gateway on int0 is unreachable for ARP pings (see log file).

0x010000 - The group communication subsystem is functional.

"Management IP Address Differs from the Management IP Address" Error Message

If you receive the following error when joining a standalone ISA node to a cluster even though the management port is configured and enabled:

If the Management IP address (x.x.x.x) for the local system differs from the Management IP address (not entered) configured for this system in the remote system, then perform the following steps to add the node:

1.From the admin console of the primary node, select System > Network > Management Port.

2.Select the node to add from the drop-down list next to the "Setting for" label.

3.Enable the management port and enter the IP address, netmask and default gateway for the joining node.

4.Click Save Changes.

5.From the admin console of the joining node, join the cluster again.

Fail-over Transactions

In the case of a fail-over (both in active/passive and active/active configurations), all transactions currently in progress (such as telnet or SSH sessions or large file downloads/uploads) must be restarted after the fail-over. There is no seamless fail-over for on-going transactions using sockets except for HTTP requests or non-stateful connections.