Verify Apigee mTLS installation

This section describes various ways to validate that the Apigee mTLS installation was successful. You can also use the techniques described in this section when troubleshooting issues with the cluster.

Validate the iptables configuration

You can validate that the apigee-mtls installation was successful by checking that the iptables routes are working and that the rules are valid.

WARNING: Do not flush your iptables rules after enabling mTLS.

Before validating an iptables configuration, be sure that:

You uninstalled firewalld from the node and replaced it with iptables, as described in Replace the default firewall.
You stopped all Apigee components on the node, including apigee-mtls.

To validate the apigee-mtls configuration was successful with iptables:

Log in to a node in your cluster. The order in which you do this does not matter.
Stop all components on the node, as the following example shows:
```
/opt/apigee/apigee-service/bin/apigee-all stop
```
Execute the validate command, as the following example shows:
```
/opt/apigee/apigee-mtls/lib/actions/iptables.sh validate
```
iptables sends messages to every port that either Consul or the local Apigee services use. If the script encounters an invalid rule or a failed route, it displays an error.

If any Apigee services or Consul servers are running on the node, then this command will fail.
Start the apigee-mtls component before all other components on the node by executing the following command:
```
/opt/apigee/apigee-service/bin/apigee-service apigee-mtls start
```
Start the remaining Apigee components on the node in the start order, as the following example shows:
```
/opt/apigee/apigee-service/bin/apigee-service component_name start
```
Repeat these steps on all nodes in the cluster. Ideally, do this on all nodes within 5 minutes of having started on the first node.

Verify the remote proxy status

You can use Consul on ZooKeeper nodes to check if the ingress and egress proxy services on all nodes are alive, healthy, and have joined the service mesh.

To check the proxy status of your nodes:

Log in to a node that is running ZooKeeper.
Execute the following command:
```
systemctl status consul_server
```

Verify the quorum status

The mTLS installation includes adding the Consul proxy services to all nodes. As a result, you should verify the quorum status of all ZooKeeper nodes.

To check the quorum status, log in to each node running ZooKeeper and execute the following command:

/opt/apigee/apigee-mtls-consul/bin/consul operator raft list-peers

This command displays a list of the Consul instances and their statuses, as the following example shows:

Node             ID                     Address            State     Voter  RaftProtocol
prc-test-0-1619  b59c1f44-6eb0-81d4-42  10.126.0.98:8300   leader    true   3
prc-test-1-1619  a4372a6e-8044-e587-43  10.126.0.146:8300  follower  true   3
prc-test-2-1619  71eb181f-4242-5353-44  10.126.0.100:8300  follower  true   3

For more information, see the following:

In addition, you can get information about the cluster's health, including whether the cluster's quorum has formed and if remote members are impairing functionality. To do this, use the following command:

/opt/apigee/apigee-service/bin/apigee-service apigee-mtls status