Multiple Data Center Installation for API BaaS

Edge for Private Cloud v4.18.01

You can install API BaaS across multiple data centers using an active/active configuration. This document describes how to add a BaaS data center to an existing BaaS data center.

Adding a data center when the Cassandra cluster is shared with Edge

All data centers must to have the same number of Cassandra nodes. An API BaaS installation can use its own Cassandra cluster, or it can share a Cassandra cluster with Edge.

If the BaaS installation shares a Cassandra cluster with Edge, you have to update the Management Server in the original data center to configure it to recognize the Cassandra nodes in the new data center.

Updating the existing data center

Adding a new BaaS data center (data center 2) requires you to perform the steps to install and configure the new data center, but it also requires you to update the original BaaS data center (data center 1) . These modifications are necessary because:

You are adding new Cassandra nodes in the new data center that have to be accessible by the existing data center.
You have to configure the replication information used by the BaaS Stack nodes in data center 1 to include information from data center 2. You cannot perform that configuration until data center 2 is installed.

To perform these updates on data center 1, you update the original configuration file used to install data center 1, and then rerun apigee-setup on the Cassandra nodes of data center 1. If the Cassandra cluster for data center 1 is shared with an Edge installation, then you also have to update the Management Server as well.

Creating the configuration files

For installing data center 1, use the configuration shown at API BaaS Installation. The configuration file does not need to include any information about data center 2 at the time you install data center 1.You add information to the configuration file after installing data center 2.

Shown below is the configuration files for data center 2. This configuration file assumes you are installing dc-2 on 10 nodes, as described at Installation Topologies. Modify this file accordingly if you are installing on 7 nodes.

Notice that the config file for data center 2 (dc-2) contains information about data center 1 (dc-1):

# Specify IP address or DNS name of node for dc-2.
IP1=192.168.1.1   # ElasticSearch
IP2=192.168.1.2   # ElasticSearch
IP3=192.168.1.3   # ElasticSearch
IP4=192.168.1.4   # API BaaS Stack
IP5=192.168.1.5   # API BaaS Stack
IP6=192.168.1.6   # API BaaS Stack
IP7=192.168.1.7   # API BaaS Portal
IP8=192.168.1.8   # Cassandra dc-2 (shared with Edge or standalone)
IP9=192.168.1.9   # Cassandra dc-2 (shared with Edge or standalone)
IP10=192.168.1.10  # Cassandra dc-2 (shared with Edge or standalone)

# Specify node information for dc-1
IP11=192.168.1.11   # Cassandra dc-1 (shared with Edge or standalone)
IP12=192.168.1.12   # Cassandra dc-1 (shared with Edge or standalone)
IP13=192.168.1.13   # Cassandra dc-1 (shared with Edge or standalone)
IP14=192.168.1.14   # API BaaS Stack
IP15=192.168.1.15   # API BaaS Stack

# Must resolve to IP address or DNS name of host - not to 127.0.0.1 or localhost.
HOSTIP=$(hostname -i)

# Define the API BaaS administrator account.  
AS_ADMIN="superuser"    # User name - default is "superuser".
AS_ADMIN_EMAIL=stackAdmin@email.com
AS_PASSWD=stackAdminPWord

# Specify Cassandra data center and rack suffix.
# List dc-2 nodes first, then dc-1.
# Must use IP addresses for CASS_HOSTS, not DNS names.
# CASS_HOSTS="$IP8:2,1 $IP9:2,1 $IP10:2,1 $IP11:1,1 $IP12:1,1 $IP13:1,1"

# Specify the Cassandra region.
REGION=dc-2

# Cassandra uname/pword.
# Even if Cassandra authentication is disabled,
# you must still pass values for these properties.
CASS_USERNAME=cassandra    # Default value
CASS_PASSWORD=cassandra    # Default value

# Specify BaaS Cassandra connection information.
# Specify the data center name as dc-2.
BAAS_CASS_LOCALDC=dc-2    

# Specify both data centers.
BAAS_CASS_DC_LIST=dc-1,dc-2

# Replication is in the form "dataCenterName:#CassandraNodes". 
# Specify both data centers.
BAAS_CASS_REPLICATION=dc-1:3,dc-2:3

# Defines the initial contact points for members of the BaaS cluster. 
# Specify the IP address of no more than two Stack nodes per data center.
# Specify both data centers. 
BAAS_CLUSTER_SEEDS="dc-1:$IP14,dc-1:$IP15,dc-2:$IP4,dc-2:$IP5"

# ElasticSearch IPs or DNS names, separated by spaces, for dc-2.
ES_HOSTS="$IP1 $IP2 $IP3"

# API BaaS Stack information.
# Default cluster name is "apigee_baas"
BAAS_USERGRID_CLUSTERNAME="apigee_baas" 

# URL and port of the load balancer for the API BaaS Stack nodes, 
# or IP/DNS and port 8080 of a single Stack node with no load balancer.
BAAS_USERGRID_URL=http://myloadbalancer:8443

# API BaaS Portal information.
# URL and port number of load balancer, if there is one in front of the Portal,
# or the URL and port of the Portal node.  
BAAS_PORTAL_URL="http://$IP7:9000"

# Portal port. Default value is 9000.
BAAS_PORTAL_LISTEN_PORT=9000 

# SMTP information. BaaS requires an SMTP server.
SMTPHOST=smtp.gmail.com
SMTPPORT=465
SMTPUSER=your@email.com
SMTPPASSWORD=yourEmailPassword
SMTPSSL=y
SMTPMAILFROM="My Company <myco@company.com>"

Add a BaaS data center

Install the first data center as described at API BaaS Installation. That document includes the configuration file that you use to install data center 1.
Create the config file for dc-2 as described above.
Install Cassandra, either as part of an existing Edge installation or as a standalone cluster for BaaS:
1. Install the Edge apigee-setup utility on the first Cassandra node of dc-2, machine 8, using the internet or non-internet procedure. See Install the Edge apigee-setup utility for more.
2. At the command prompt, run the setup script to install Cassandra on the first node:
  > /opt/apigee/apigee-setup/bin/setup.sh -p c -f configFile
  
  The “-p c” option specifies to install Cassandra.
  
  The configuration file must be accessible or readable by the "apigee" user.
3. Repeat steps 3 and 4 for the remaining Cassandra nodes, machines 9 and 10, in dc-2.
On the Cassandra nodes of dc-1, edit the configuration file to add the Cassandra nodes from data center 2 based on how you installed Cassandra:

# List dc-1 nodes first, then dc-2,
# including Cassandra data center and rack suffix
CASS_HOSTS="$IP11:1,1 $IP12:1,1 $IP13:1,1 $IP8:2,1 $IP9:2,1 $IP10:2,1"
On the first Cassandra node of dc-1, run setup.sh with the new dc-1 config file that includes the Cassandra nodes from dc-2:
> /opt/apigee/apigee-setup/bin/setup.sh -p c -f configFile
Repeat step 6 and 7 for the remaining Cassandra nodes in dc-1.
If you are connecting to a Cassandra cluster for dc-1 shared with Edge, add the dc-2 Cassandra nodes to the config file for the Management Server node of dc-1 and run setup.sh:
> /opt/apigee/apigee-setup/bin/setup.sh -p ms -f configFile
Update Cassandra keyspaces with correct replication factor for the two data centers. You only have to run this step once on any Cassandra server in either data center:

Note: The commands below all set the replication factor to "3", indicating three Cassandra nodes in the cluster. Modify this value as necessary for your installation.
1. Start the Cassandra cqlsh utility:
  > /opt/apigee/apigee-cassandra/bin/cqlsh cassandraIP
2. Execute the following CQL commands at the "cqlsh>" prompt to set the replication levels for Cassandra keyspaces:
  1. cqlsh> ALTER KEYSPACE "Apigee_Baas_dc_1" WITH replication = { 'class': 'NetworkTopologyStrategy', 'dc-1': '3','dc-2': '3' };
  2. cqlsh> ALTER KEYSPACE "Apigee_Baas" WITH replication = { 'class': 'NetworkTopologyStrategy', 'dc-1': '3','dc-2': '3' };
  3. cqlsh> ALTER KEYSPACE "Apigee_Baas_Locks" WITH replication = { 'class': 'NetworkTopologyStrategy', 'dc-1': '3','dc-2': '3' };
  4. cqlsh> ALTER KEYSPACE "system_traces" WITH replication = { 'class': 'NetworkTopologyStrategy', 'dc-1': '3','dc-2': '3' };
  5. cqlsh> ALTER KEYSPACE "system_auth" WITH replication = { 'class': 'NetworkTopologyStrategy', 'dc-1': '3','dc-2': '3' };
  6. View the keyspaces by using the command:
    cqlsh> select * from system.schema_keyspaces;
  7. Exit cqlsh:
    cqlsh> exit
On all Cassandra nodes in dc-2, run the rebuild command, specifying the region name of dc-1:
> /opt/apigee/apigee-cassandra/bin/nodetool -h cassandraIP rebuild dc-1
Run "nodetool status" on every Cassandra node and verify that all the Cassandra nodes have ‘100%’ for the ‘Owns’ value:
> /opt/apigee/apigee-cassandra/bin/nodetool status
Install BaaS in dc-2:
1. Edit the config file to list only the Cassandra nodes in dc-2:
  # List dc-2 Cassandra nodes,
  # omitting Cassandra data center and rack suffix
  CASS_HOSTS="$IP8 $IP9 $IP10"
2. Install ElasticSearch on nodes 4, 5, and 6 of dc-2:
  > /opt/apigee/apigee-setup/bin/setup.sh -p e -f configFile
3. Install the BaaS Stack on nodes 4, 5, and 6 of dc-2:
  > /opt/apigee/apigee-setup/bin/setup.sh -p b -f configFile
4. Install the BaaS Portal on machine 7:
  > /opt/apigee/apigee-setup/bin/setup.sh -p p -f configFile
Update BaaS Stack nodes in data center 1:
1. On the first BaaS Stack node in data center 1, edit /opt/apigee/customer/application/usergrid.properties in an editor. If the file does not exist, create it.
2. Add the following properties to usergrid.properties:
  # Same value as BAAS_CLUSTER_SEEDS in the dc-2 config file,
  # without double quotes.
  usergrid-deployment_usergrid.cluster.seeds=dc-1:dc1StackIP1,dc-1:dc1StackIP2,dc-2:dc2StackIP1,dc-2:dc2StackIP2
  
  # Same value as BAAS_CASS_DC_LIST in the dc-2 config file.
  usergrid-deployment_usergrid.cluster.region.list=dc-1,dc-2
3. Repeat steps a and b on the remaining BaaS Stack nodes.
4. Restart all BaaS Stack nodes.
  
  Note: When restarting BaaS Stack nodes, restart them in the same order that they are listed in BAAS_CLUSTER_SEEDS. BAAS_CLUSTER_SEEDS lists at most two of the Stack nodes. After restarting those two nodes in order, you can restart the remaining nodes in any order.
  
  > /opt/apigee/apigee-service/bin/apigee-service baas-usergrid restart
Update BaaS keyspace values. These keyspace have to be set for replication at install time, but do not have to be at run time. Removing replication also saves Cassandra memory.

?You only have to run this step once on any Cassandra server in either data center:
1. Start the Cassandra cqlsh utility:
  > /opt/apigee/apigee-cassandra/bin/cqlsh cassandraIP
2. Execute the following CQL commands to set the replication levels for Cassandra keyspaces:
  1. cqlsh> ALTER KEYSPACE "Apigee_Baas_dc_1" WITH replication = { 'class': 'NetworkTopologyStrategy', 'dc-1': '3' };
  2. cqlsh> ALTER KEYSPACE "Apigee_Baas_dc_2" WITH replication = { 'class': 'NetworkTopologyStrategy', 'dc-2': '3' };
  3. View the keyspaces by using the command:
    cqlsh> select * from system.schema_keyspaces;
  4. Exit cqlsh:
    cqlsh> exit

This installation of the second data center is now complete.

After you complete the installation and configuration of the two data centers, you can validate the installation by using the following procedure:

On all BaaS Stack nodes, check the status:
> curl 0:8080/status
Check token API call is working:
> curl -X POST "http://localhost:8080/management/token" -d '{"grant_type":"password", "username":"adminEmail","password":"pWord"}'
Try logging in to the BaaS Portal for dc-2. Verify, all the data is replicated in dc-2, such as collections created on dc-1.