Edge for Private Cloud v4.18.05
This document describes how to add three new ZooKeeper nodes to an existing Edge for Private Cloud installation.
You can add one or two ZooKeeper nodes to an existing Edge installation, however, you must make sure that you always have an odd number of ZooKeeper voter nodes, as described below.
Existing Edge configuration
All the supported Edge topologies for a production system specify to use three ZooKeeper
nodes. The three nodes are specified to the ZK_HOSTS
and
ZK_CLIENT_HOSTS
properties in the config file as shown below:
IP1=10.10.0.1 IP2=10.10.0.2 IP3=10.10.0.3 HOSTIP=$(hostname -i) ADMIN_EMAIL=opdk@google.com APIGEE_ADMINPW=Secret123 LICENSE_FILE=/tmp/license.txt MSIP=$IP1 USE_LDAP_REMOTE_HOST=n LDAP_TYPE=1 APIGEE_LDAPPW=secret MP_POD=gateway REGION=dc-1 ZK_HOSTS="$IP1 $IP2 $IP3" ZK_CLIENT_HOSTS="$IP1 $IP2 $IP3" CASS_HOSTS="$IP1:1,1 $IP2:1,1 $IP3:1,1" SKIP_SMTP=n SMTPHOST=smtp.example.com SMTPUSER=smtp@example.com SMTPPASSWORD=smtppwd
Where:
ZK_HOSTS
specifies the IP addresses or DNS names of the ZooKeeper nodes. The IP addresses or DNS names must be listed in the same order on all ZooKeeper nodes. In a multi-data center environment, list all ZooKeeper nodes from both data centers.ZK_CLIENT_HOSTS
specifies the IP addresses or DNS names of the ZooKeeper nodes used by this data center. The IP addresses or DNS names must be listed in the same order on all ZooKeeper nodes.In a single data center installation, these are the same nodes as specified by
ZK_HOSTS
. In a multi-data center environment, list only the ZooKeeper nodes in this data center.
Modifying the config file to add the three new ZooKeeper nodes
In this example, the three new ZooKeeper nodes are at the following IP addresses:
- 10.10.0.14
- 10.10.0.15
- 10.10.0.16
You must first update Edge configuration file to add the new nodes:
IP1=10.10.0.1 IP2=10.10.0.2 IP3=10.10.0.3 # Add the new node IP addresses. IP14=10.10.0.14 IP15=10.10.0.15 IP16=10.10.0.16 HOSTIP=$(hostname -i) ADMIN_EMAIL=opdk@google.com ... # Update ZK_HOSTS to add each new node after an existing nodes. ZK_HOSTS="$IP1 $IP2 $IP3 $IP14 $IP15 $IP16:observer" # Update ZK_Client_HOSTS to add each new node after an existing nodes. ZK_CLIENT_HOSTS="$IP1 $IP2 $IP3 $IP14 $IP15 $IP16"
Mark the last node in ZK_HOSTS
with the with
:observer
modifier. Nodes without the :observer
modifier are called
"voters". You must have an odd number of "voters" in your configuration. Therefore, in this
configuration, you have 5 ZooKeeper voters and one observer.
Make sure to add the nodes to both ZK_HOSTS
and
ZK_CLIENT_HOSTS
in the same order. However, omit the :observer
modifier
when setting ZK_CLIENT_HOSTS
.
Configure Edge
After editing the config file, you must perform all of the following tasks.
Install ZooKeeper on the new nodes
- Install
apigee-setup
on the first node as described in Install the Edge apigee-setup utility. - Install ZooKeeper on the first node by using the following commands:
/opt/apigee/apigee-service/bin/apigee-service apigee-zookeeper install
/opt/apigee/apigee-service/bin/apigee-service apigee-zookeeper setup -f updatedConfigFile
- Repeat steps 1 and 2 for the remaining new ZooKeeper nodes.
Reconfigure the existing ZooKeeper nodes
On the existing ZooKeeper nodes:
- Rerun the setup command with the new config file:
/opt/apigee/apigee-service/bin/apigee-service apigee-zookeeper setup -f updatedConfigFile
Restart all Zookeeper nodes
On all ZooKeeper nodes:
- Restart the node:
/opt/apigee/apigee-service/bin/apigee-service apigee-zookeeper restart
You must restart all ZooKeeper nodes, but order of restart does not matter.
Reconfigure the Management Server node
On the Management Server node:
- Run the setup command:
/opt/apigee/apigee-service/bin/apigee-service edge-management-server setup -f updatedConfigFile
- Restart the Management Server:
/opt/apigee/apigee-service/bin/apigee-service edge-management-server restart
Reconfigure all the Routers
On all Router nodes:
- Run the setup command:
/opt/apigee/apigee-service/bin/apigee-service edge-router setup -f updatedConfigFile
- Restart the Router:
/opt/apigee/apigee-service/bin/apigee-service edge-router restart
Reconfigure all the Message Processors
On all Message Processor nodes:
- Run the setup command:
/opt/apigee/apigee-service/bin/apigee-service edge-message-processor setup -f updatedConfigFile
- Restart the Message Processor:
/opt/apigee/apigee-service/bin/apigee-service edge-message-processor restart
Reconfigure all Qpid nodes
On all Qpid nodes:
- Run the setup command:
/opt/apigee/apigee-service/bin/apigee-service edge-qpid-server setup -f updatedConfigFile
- Restart Qpid:
/opt/apigee/apigee-service/bin/apigee-service edge-qpid-server restart
Reconfigure all Postgres nodes
On all Postgres nodes:
- Run the setup command:
/opt/apigee/apigee-service/bin/apigee-service edge-postgres-server setup -f updatedConfigFile
- Restart Postgres:
/opt/apigee/apigee-service/bin/apigee-service edge-postgres-server restart
Validate the installation
You can validate the installation of the new ZooKeeper nodes by sending commands to port 2181 using netcat (nc) or telnet. For more info on ZooKeeper commands, see: http://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html#sc_zkCommands.
To validate:
- If it is not installed on the ZooKeeper node, install nc:
sudo yum install nc
- Run the following nc command:
echo stat | nc localhost 2181
- Repeat steps 1 and 2 on each ZooKeeper node. In the
Mode
line of the output for the nodes, one node should be designated as observer, one node as leader, and the rest as followers.