Enable Cassandra internode encryption

Internode (or node-to-node) encryption protects data traveling between nodes in a cluster using TLS. This page explains how to enable Cassandra internode encryption using TLS on Edge for Private Cloud. To perform these steps, you must be familiar with the details of your Cassandra ring.

Enable Cassandra internode encryption

Follow these steps to enable Cassandra internode encryption:

1. Generate server certificates by following the steps in the Appendix to create a self-signed key and certificate.

   The following steps assume you have created keystore.node0 and truststore.node0, as well as the keystore and truststore passwords, as explained in the Appendix. The keystore and truststore should be created as preliminary steps on each node before proceeding with next steps.

2. Add the following properties to the /opt/apigee/customer/application/cassandra.properties file. If the file does not exist, create it.

   conf_cassandra_internode_encryption=all
   conf_cassandra_keystore=/opt/apigee/data/apigee-cassandra/keystore.node0
   conf_cassandra_keystore_password=keypass
   conf_cassandra_truststore=/opt/apigee/data/apigee-cassandra/truststore.node0
   conf_cassandra_truststore_password=trustpass
   # Optionally set the following to enable 2-way TLS or mutual TLS
   # conf_cassandra_require_client_auth=true

3. Ensure that the file cassandra.properties is owned by the apigee user:

   chown apigee:apigee \
   /opt/apigee/customer/application/cassandra.properties

Execute the following steps on each Cassandra node, one at a time, so the changes take effect without causing any downtime for users:

3. Stop the Cassandra service:

   /opt/apigee/apigee-service/bin/apigee-service \
   apigee-cassandra stop

4. Restart the Cassandra service:

   /opt/apigee/apigee-service/bin/apigee-service \
   apigee-cassandra start

5. To determine if the TLS encryption service has started, check the system logs for the following message:

   Starting Encrypted Messaging Service on TLS port

Perform certificate rotation

To rotate certificates, follow these steps:

1. Add the certificate for each unique generated key pair (see Appendix) to an existing Cassandra node's truststore, such that both the old certificates and new certificates exist in the same truststore:

   keytool -import -v -trustcacerts -alias NEW_ALIAS \
   -file CERT -keystore EXISTING_TRUSTSTORE

   where NEW_ALIAS is a unique string to identify the entry, CERT is the name of the certificate file to add, and EXISTING_TRUSTSTORE is the name of the existing truststore on the Cassandra node.

2. Use a copy utility, such as scp, to distribute the truststore to all Cassandra nodes in the cluster replacing the existing truststore in use by each node.
3. Perform a rolling restart of the cluster to load the new truststore and establish trust for the new keys before they are in place:

   /opt/apigee/apigee-service/bin/apigee-service \
   apigee-cassandra restart

4. On each Cassandra node in the cluster, update the properties shown below to the new keystore values in the cassandra.properties file:

   conf_cassandra_keystore=NEW_KEYSTORE_PATH
   conf_cassandra_keystore_password=NEW_KEYSTORE_PASSOWRD
   
     
   where NEW_KEYSTORE_PATH is the path to the directory where the keystore file is
     located and NEW_KEYSTORE_PASSWORD is the keystore password set when the certificates
     were created, as explained in the Appendix.

5. Stop the Cassandra service:

   /opt/apigee/apigee-service/bin/apigee-service \
   apigee-cassandra stop

6. Restart the Cassandra service:

   /opt/apigee/apigee-service/bin/apigee-service \
   apigee-cassandra start

7. When communication is successfully established between all nodes, proceed to the next Cassandra node. Note: Only proceed to the next node if communication is successfully established between all nodes.

Appendix

The following example explains how to prepare server certificates needed to perform the internode encryption steps. The commands shown in the example use the following parameters:

1. Go to the following directory:

   cd /opt/apigee/data/apigee-cassandra

2. Run the following command to generate a file named keystore.node0 in the current directory:

   keytool -genkey -keyalg RSA -alias node0 -validity 3650 \
   -keystore keystore.node0 -storepass keypass \
   -keypass keypass -dname "CN=10.128.0.39, OU=None, \
   O=None, L=None, C=None"

   Important: Make sure that the key password is the same as the keystore password.

3. Export the certificate to a separate file:

   keytool -export -alias node0 -file node0.cer \
   -keystore keystore.node0

4. Ensure the file is readable by the apigee user only and by no one else:

   $ chown apigee:apigee \
   /opt/apigee/data/apigee-cassandra/keystore.node0
   $ chmod 400 /opt/apigee/data/apigee-cassandra/keystore.node0

5. Import the generated certificate node0.cer to the truststore of the node:

   keytool -import -v -trustcacerts -alias node0 \
   -file node0.cer -keystore truststore.node0

   The command above asks you to set a password. This is the truststore password and can be different from the keystore password you set earlier. If prompted to trust the certificate, enter yes.

6. Use openssl to generate a PEM file of the certificate with no keys. Note that cqlsh does not work with the certificate in the format generated.

   $ keytool -importkeystore -srckeystore keystore.node0 \
   -destkeystore node0.p12 -deststoretype PKCS12 -srcstorepass \
   keypass -deststorepass keypass
   $ openssl pkcs12 -in node0.p12 -nokeys -out node0.cer.pem \
   -passin pass:keypass
   $ openssl pkcs12 -in node0.p12 -nodes -nocerts -out node0.key.pem -passin pass:keypass

7. For node-to-node encryption, copy the node0.cer file to each node and import it to the truststore of each node.

   keytool -import -v -trustcacerts -alias node0 \
   -file node0.cer -keystore truststore.node1

8. Use keytool -list to check for certificates in the keystore and truststore files:

   $ keytool -list -keystore keystore.node0
   $ keytool -list -keystore truststore.node0