Update Apigee Edge 4.17.0x to 4.18.01

Edge for Private Cloud v4.18.01

Which Edge versions can you update to 4.18.01

You can update Apigee Edge version 4.17.0x using this procedure.

If you have a version of Edge previous to version 4.16.01, then you must first migrate to version 4.16.01.x and then update to version 4.18.01.

Who can perform the update

The user running the update should be the same as the user who originally installed Edge, or a user running as root.

After you install the Edge RPMs, any user can configure them.

Which components must you update

You must update all Edge components. Edge does not support a setup that contains components from multiple versions.

Maintain compatibility

When upgrading Apigee Edge for Private Cloud to 4.18.01, you must ensure that the resulting implementation only references supported properties in its configuration files.

Property	Location	Description
`conf_load_balancing_load.balancing. driver.nginx.server.retry`	`router.properties`	Remove this property from your `router.properties` file. To set retry options in 4.18.01, use the virtual host configuration properties.

Automatic propagation of property settings

If you have set any properties by editing .properties files in /opt/apigee/customer/application then these values are retained by the update.

Required addition of SMTPMAILFROM configuration parameter

Edge 4.17.05 added a new required parameter to the configuration file used when you enable an SMTP server. If you are updating from 4.17.01, then you must set SMTPMAILFROM in the configuration file when enabling the SMTP server.

API BaaS version 4.17.09 also added support for the SMTPMAILFROM property. When updating API BaaS from either 4.17.01 or 4.17.05, you must set SMTPMAILFROM in the configuration file.

This parameter has the form:

SMTPMAILFROM="My Company <myco@company.com>"

Required upgrade to Postgres 9.6

This release of Edge includes an upgrade to Postgres 9.6. As part of that upgrade, all Postgres data is migrated to the Postgres 9.6.

Most Edge production systems use two Postgres nodes configured for master-standby replication. During the update process, while the Postgres nodes are down for update, analytics data is still written to the Qpid nodes. After the Postgres nodes are updated and back online, analytics data is then pushed to the Postgres nodes.

The way you perform the Postgres update depends on how you configured data storage for your Postgres nodes:

If you use local data storage for your Postgres nodes, you must install a new Postgres standby node for the duration of the upgrade. After the upgrade completes, you can decommission the new Postgres standby node.

The additional Postgres standby node is required if you have to roll back the update for any reason. If you have to roll back the update, the new Postgres standby node becomes the master Postgres node after the rollback. Therefore, when you install the new Postgres standby node, it should be on a node that meets all the hardware requirements of a Postgres server, as defined in the Edge Installation Requirements.

In a 1-node and 2-node installation of Edge, topologies used for prototyping and testing, you only have a single Postgres node. You can update these Postgres nodes directly without having to create a new Postgres node.
If you use network storage for your Postgres nodes, as recommended by Apigee, you do not have to install a new Postgres node. In the procedures below, you can skip the steps that specify to install and later decommission a new Postgres standby node.

Before you begin the update process, take a network snapshot of the data store used by Postgres. Then, if any errors occur during update and you are forced to perform a roll back, you can restore the Postgres node from that snapshot.

Installing a new Postgres standby node

This procedure creates a Postgres standby server on a new node. Ensure that you install a new Postgres standby server for your existing version of Edge (4.17.0x), not for version 4.18.01.

To perform the install, use the same config file that you used to install your current version of Edge.

To create a new Postgres standby node:

On the current Postgres master, edit the /opt/apigee/customer/application/postgresql.properties file to set the following token:
```
conf_pg_hba_replication.connection=host replication apigee existing_slave_ip/32
trust\ \nhost replication apigee new_slave_ip/32 trust
```
Where existing_slave_ip is the IP address of the current Postgres standby server and new_slave_ip is the IP address of the new standby node.

If the postgresql.properties file does not exist, create it and use chown to give the apigee user access to it, as the following example shows:
```
chown apigee:apigee /opt/apigee/customer/application/postgresql.properties
```

Restart apigee-postgresql on the Postgres master:

/opt/apigee/apigee-service/bin/apigee-service apigee-postgresql restart

Verify that the new standby node was added by viewing the /opt/apigee/apigee-postgresql/conf/pg_hba.conf file on the master. You should see the following lines in that file:
```
host replication apigee existing_slave_ip/32 trust
host replication apigee new_slave_ip/32 trust
```
Install the new Postgres standby server:
1. Edit the config file that you used to install your current version of Edge to specify the following:
```
# IP address of the current master:
PG_MASTER=192.168.56.103
# IP address of the new standby node
PG_STANDBY=192.168.56.102 
```
2. Disable SELinux as described at Install the Edge apigee-setup utility.
3. Download the Edge bootstrap_4.17.0x.sh file (where 4.17.0x is 4.17.01, 4.17.05, or 4.17.09) to /tmp/bootstrap_4.17.0x.sh :
```
curl https://software.apigee.com/bootstrap_4.17.0x.sh -o /tmp/bootstrap_4.17.0x.sh
```
4. Install the Edge apigee-service utility and dependencies:
```
sudo bash /tmp/bootstrap_4.17.0x.sh apigeeuser=uName
  apigeepassword=pWord
```
5. Use apigee-service to install the apigee-setup utility:
```
/opt/apigee/apigee-service/bin/apigee-service apigee-setup install
```
6. Install Postgres:
```
/opt/apigee/apigee-setup/bin/setup.sh -p ps -f configFile
```
7. On the new standby server, execute the following command:
```
/opt/apigee/apigee-service/bin/apigee-service apigee-postgresql postgres-check-standby
```
  Validate that it says it is the standby.

Decommissioning a Postgres node

After the update completes, decommission the new standby node:

Make sure Postgres is running:
> /opt/apigee/apigee-service/bin/apigee-all status

If Postgres is not running, start it:
> /opt/apigee/apigee-service/bin/apigee-all start
Get the UUID of the new standby node by running the following cURL command on the new standby node:
> curl -u sysAdminEmail:password http://<node_IP>:8084/v1/servers/self

You should see the UUID of the node at the end of the output, in the form:
"type" : [ "postgres-server" ],
"uUID" : "599e8ebf-5d69-4ae4-aa71-154970a8ec75"
Stop the new standby node by running the following command on the new standby node:
> /opt/apigee/apigee-service/bin/apigee-all stop
On the Postgres master node, edit /opt/apigee/customer/application/postgresql.properties to remove the new standby node from conf_pg_hba_replication.connection:
conf_pg_hba_replication.connection=host replication apigee existing_slave_ip/32 trust
Restart apigee-postgresql on the Postgres master:
> /opt/apigee/apigee-service/bin/apigee-service apigee-postgresql restart
Verify that the new standby node was removed by viewing the /opt/apigee/apigee-postgresql/conf/pg_hba.conf file on the master. You should see only the following line in that file:
host replication apigee existing_slave_ip/32 trust
Delete the UUID of the standby node from ZooKeeper by making the following Edge management API call on the Management Server node:
> curl -u sysAdminEmail:password -X DELETE http://<ms_IP>:8080/v1/servers/<new_slave_uuid>

Update prerequisites

Take care of following prerequisites before upgrading Apigee Edge:

Backup all nodes
Before you update, it is recommended to perform a complete backup of all nodes for safety reasons. Use the procedure for your current version of Edge to perform the backup.

This allows you to have a backup plan, in case the update to a new version doesn’t function properly. For more information on backup, see Backup and Restore.
Ensure Edge is running
Ensure that Edge is up and running during update process by using the command:

> /opt/apigee/apigee-service/bin/apigee-all status

Handling a failed update

In the case of an update failure, you can try to correct the issue, and then run update.sh again. You can run the update multiple times and it continues the update from where it last left off.

If the failure requires that you roll back the update to your previous version, see 4.18.01 Rollback Process for more.

Logging update information

By default, the update.sh utility writes log information to:

/opt/apigee/var/log/apigee-setup/update.log

If the user running the update.sh utility does not have access to that directory, it writes the log to the /tmp directory as a file named update_username.log.

If the user does not have access to /tmp, the update.sh utility fails.

Zero-downtime update

A zero-downtime update, or rolling update, lets you update your Edge installation without bringing down Edge.

Zero-downtime update is only possible with a 5-node configuration and larger.

The key to zero-downtime upgrading is to remove each Router, one at a time, from the load balancer. You then update the Router and any other components on the same machine as the Router, and then add the Router back to the load balancer.

Update the machines in the correct order for your installation as described below in "Order of machine update".
When it is time to update the Routers, select any one Router and make it unreachable, as described in Enabling/Disabling Server (Message Processor/Router) Reachability.
Update the selected Router and all other Edge components on the same machine as the Router. All Edge configurations show a Router and Message Processor on the same node.
Make the Router reachable again.
Repeat steps 2 through 4 for the remaining Routers.
Continue the update for any remaining machines in your installation.

Take care of the following before/after update:

On combined Router and Message Processor node:
- Before update – perform the following:
  1. Make the Router unreachable.
  2. Make the Message Processor unreachable.
- After update - perform the following:
  1. Make the Message Processor reachable.
  2. Make the Router reachable.
On single Router node:
- Before update, make the Router unreachable.
- After update, make the Router reachable.
On single Message Processor node:
- Before update, make the Message Processor unreachable.
- After update, make the Message Processor reachable.

Using a silent configuration file

You must pass a silent configuration file to the update command. The silent configuration file should be the same one you used to install Edge 4.17.0x.

Procedure for updating to 4.18.01 on a node with an external internet connection

Use the following procedure to update the Edge components on a node:

If you are currently using Postgres master-standby replication with local storage, install a new Postgres standby node as described above in Installing a new Postgres standby node.

If you are using network storage, you do not have to install the new node. Instead, take a network snapshot of the data store used by Postgres. See Required upgrade to Postgres 9.6 for more.
If present, disable any CRON jobs configured to perform a repair operation on Cassandra until after the update completes.
Log in to your node as root to install the Edge RPMs.
Note: While RPM installation requires root access, you can perform Edge configuration without root access.
Install yum-utils and yum-plugin-priorities:
> sudo yum install yum-utils
> sudo yum install yum-plugin-priorities
Disable SELinux as described in Install the Edge apigee-setup utility.
If you are installing on Oracle 7.x, run the following command:
> sudo yum-config-manager --enable ol7_optional_latest
If you are installing on AWS, run the following yum-configure-manager commands:
> yum update rh-amazon-rhui-client.noarch
> sudo yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional
Download the Edge 4.18.01 bootstrap_4.18.01.sh file to /tmp/bootstrap_4.18.01.sh:
> curl https://software.apigee.com/bootstrap_4.18.01.sh -o /tmp/bootstrap_4.18.01.sh
Install the Edge 4.18.01 apigee-service utility and dependencies:
> sudo bash /tmp/bootstrap_4.18.01.sh apigeeuser=uName apigeepassword=pWord

where uName:pWord are the username and password you received from Apigee. If you omit pWord, you will be prompted to enter it.

By default, the installer checks to see that you have Java 1.8 installed. If you do not, it installs it for you. Use the JAVA_FIX option to specify how to handle Java installation. JAVA_FIX takes the following values:

I = Install OpenJDK 1.8 (default)
C = Continue without installing Java
Q = Quit. For this option, you have to install Java yourself.
Use apigee-service to update the apigee-setup utility:
> /opt/apigee/apigee-service/bin/apigee-service apigee-setup update
Update the apigee-validate utility on the Management Server:
> /opt/apigee/apigee-service/bin/apigee-service apigee-validate update
Update the apigee-provision utility:
> /opt/apigee/apigee-service/bin/apigee-service apigee-provision update
Run the update utility on your nodes in the order described below in "Order of machine update" below:
> /opt/apigee/apigee-setup/bin/update.sh -c component -f configFile

The only requirement on the config file is that the configuration file must be accessible or readable by the "apigee" user.

Use the “-c” option to specify the component to update. The list of possible components includes:
ldap = OpenLDAP
cs = Cassandra
zk = Zookeeper
qpid = qpidd
ps = postgresql
edge =All Edge components except Edge UI: Management Server, Message Processor, Router, QPID Server, Postgres Server
ui = Edge UI
all = update all components on machine (only use for an Edge aio installation profile or an API BaaS asa installation profile)
e = ElasticSearch
b = API BaaS Stack
p = API BaaS Portal
ebp = ElasticSearch, API BaaS Stack, and API BaaS Portal on the same node
Test the update by running the apigee-validate utility on the Management Server, as described in Test the install.
If you installed a new Postgres standby node, decommission the node as described above in Decommissioning a Postgres node.

If you are using network storage, you do not have to install the new node. See Required upgrade to Postgres 9.6 for more.

To later rollback the update, use the procedure described in 4.18.01 Rollback Process.

Procedure for updating to 4.18.01 from a local repo

If your Edge nodes are behind a firewall, or in some other way are prohibited from accessing the Apigee repository over the Internet, then you can perform the update from a local repository, or mirror, of the Apigee repo.

After you create a local Edge repository, you have two options for updating Edge from the local repo:

Create a .tar file of the repo, copy the .tar file to a node, and then update Edge from the .tar file.
Install a webserver on the node with the local repo so that other nodes can access it. Apigee provides the Nginx webserver for you to use, or you can use your own webserver.

To update from a local 4.18.01 repo:

If you are currently using Postgres master-standby replication, install a new Postgres standby node as described above in Installing a new Postgres standby node.

If you are using network storage, you do not have to install the new node. Instead, take a network snapshot of the data store used by Postgres. See Required upgrade to Postgres 9.6 for more.
Create a local 4.18.01 repo as described in "Create a local Apigee repository" at Install the Edge apigee-setup utility.
Note: If you already have an existing 4.17.0x repo, you can add the 4.18.01 repo to it as described in "Update a local Apigee repository" at Install the Edge apigee-setup utility.
To install apigee-service from a .tar file:
1. On the node with the local repo, use the following command to package the local repo into a single .tar file named /opt/apigee/data/apigee-mirror/apigee-4.18.01.tar.gz:
  > /opt/apigee/apigee-service/bin/apigee-service apigee-mirror package
2. Copy the .tar file to the node where you want to update Edge. For example, copy it to the /tmp directory on the new node.
3. On the new node, untar the file to the /tmp directory:
  > tar -xzf apigee-4.18.01.tar.gz
  
  This command creates a new directory, named repos, in the directory containing the .tar file. For example /tmp/repos.
4. Install the Edge apigee-service utility and dependencies from /tmp/repos:
  > sudo bash /tmp/repos/bootstrap_4.18.01.sh apigeeprotocol="file://" apigeerepobasepath=/tmp/repos
  
  Notice that you include the path to the repos directory in this command.
To install apigee-service using the Nginx webserver:
1. Configure the Nginx web server as described in "Install from the repo using the Nginx webserver" at Install the Edge apigee-setup utility.
2. On the remote node, download the Edge bootstrap_4.18.01.sh file to /tmp/bootstrap_4.18.01.sh:
  > /usr/bin/curl http://uName:pWord@remoteRepo:3939/bootstrap_4.18.01.sh -o /tmp/bootstrap_4.18.01.sh
  
  where uName:pWord are the username and password you set above for the repo, and remoteRepo is the IP address or DNS name of the repo node.
3. On the remote node, install the Edge apigee-service utility and dependencies:
  > sudo bash /tmp/bootstrap_4.18.01.sh apigeerepohost=remoteRepo:3939 apigeeuser=uName apigeepassword=pWord apigeeprotocol=http://
  
  where uName:pWord are the repo username and password.
Use apigee-service to update the apigee-setup utility:
> /opt/apigee/apigee-service/bin/apigee-service apigee-setup update
Update the apigee-validate utility on the Management Server.
> /opt/apigee/apigee-service/bin/apigee-service apigee-validate update
Update the apigee-provision utility:
> /opt/apigee/apigee-service/bin/apigee-service apigee-provision update
Run the update utility on your nodes in the order described below in "Order of machine update" below:
> /opt/apigee/apigee-setup/bin/update.sh -c component -f configFile

The only requirement on the config file is that the configuration file must be accessible or readable by the "apigee" user.

Use the “-c” option to specify the component to update. The list of possible components includes:
ldap = OpenLDAP
cs = Cassandra
zk = Zookeeper
qpid = qpidd
ps = postgresql
edge =All Edge components except Edge UI: Management Server, Message Processor, Router, QPID Server, Postgres Server
ui = Edge UI
all = update all components on machine (only use for an Edge aio installation profile or an API BaaS asa installation profile)
e = ElasticSearch
b = API BaaS Stack
p = API BaaS Portal
ebp = ElasticSearch, API BaaS Stack, and API BaaS Portal on the same node
Test the update by running the apigee-validate utility on the Management Server, as described in Test the install.
If you installed a new Postgres standby node, decommission the node as described above in Decommissioning a Postgres node.

If you are using network storage, you do not have to install the new node. See Required upgrade to Postgres 9.6 for more.

To later rollback the update, use the procedure described in 4.18.01 Rollback Process.

Order of machine update

The order that you update the machines in an Edge installation is important. The most important considerations to an update are:

You must update all Cassandra and ZooKeeper nodes before you update any other nodes.
For any machine with multiple Edge components (Management Server, Message Processor, Router, QPID Server but not Postgres Server), use the "-c edge" option to update them all at the same time.
If a step specifies that it should be performed on multiple machines, perform it in the specified machine order.
There is no separate step to update Monetization. It is updated when you specify the "-c edge" option.