Edge for Private Cloud v4.18.05
The need to manage the firewall goes beyond just the virtual hosts; both VM and physical host firewalls must allow traffic for the ports required by the components to communicate with each other.
The following images show the port requirements for both a single data center and multiple data center configuration:
Single Data Center
The following image shows the port requirements for each Edge component in a single data center configuration:
Notes on this diagram:
- The ports prefixed by "M" are ports used to manage the component and must be open on the component for access by the Management Server.
- The Edge UI requires access to the Router, on the ports exposed by API proxies, to support the Send button in the trace tool.
- Access to JMX ports can be configured to require a username/password. See How to Monitor for more information.
- You can optionally configure TLS/SSL access for certain connections, which can use different ports. See TLS/SSL for more.
- You can configure the Management Server and Edge UI to send emails through an external SMTP server. If you do, you must ensure that the Management Server and UI can access the necessary port on the SMTP server (not shown). For non-TLS SMTP, the port number is typically 25. For TLS-enabled SMTP, it is often 465, but check with your SMTP provider.
Multiple Data Centers
If you install the 12-node clustered configuration with two data centers, ensure that the nodes in the two data centers can communicate over the ports shown below:
- All Management Servers must be able to access all Cassandra nodes in all other data centers.
- All Message Processors in all data centers must all be able to access each other over port 4528.
- The Management Server must be able to access all Message Processors over port 8082.
- All Management Servers and all Qpid nodes must be able to access Postgres in all other data centers.
- For security reasons, other than the ports shown above and any others required by your own network requirements, no other ports should be open between data centers.
The table below describes the ports that need to be opened in firewalls, by Edge component:
|Standard HTTP ports||80, 443||HTTP plus any other ports you use for virtual hosts|
|Cassandra||7000, 9042, 9160||Apache Cassandra ports for communication between Cassandra nodes and for access by other Edge components.|
|7199||JMX port. Must be open for access by the Management Server.|
|Management Server||1099||JMX port|
|4526||Port for distributed cache and management calls. This port is configurable.|
|8080||Port for Edge management API calls. These components require access to port 8080 on the Management Server: Router, Message Processor, UI, Postgres, and Qpid.|
|Management UI||9000||Port for browser access to management UI|
|Message Processor||1101||JMX port|
|4528||For distributed cache and management calls between Message Processors, and for
communication from the Router and Management Server.
A Message Processor must open port 4528 as its management port. If you have multiple Message Processors, they must all be able to access each other over port 4528 (indicated by the loop arrow in the diagram above for port 4528 on the Message Processor). If you have multiple data centers, the port must be accessible from all Message Processors in all data centers.
Default management port for Message Processor and must be open on the component for access by the Management Server.
If you configure TLS/SSL between the Router and Message Processor, used by the Router to make health checks on the Message Processor.
Port 8082 on the Message Processor only has to be open for access by the Router when you configure TLS/SSL between the Router and Message Processor. If you do not configure TLS/SSL between the Router and Message Processor, the default configuration, port 8082 still must be open on the Message Processor to manage the component, but the Router does not require access to it.
|8443||When TLS is enabled, you must open port 8443 on the Message Processor for the Router. This is in addition to port 8082, which must also be open, whether or not TLS is enabled.|
|8998||Message Processor port for communications from Router|
|Postgres||22||If configuring two Postgres nodes to use master-standby replication, you must open port 22 on each node for SSH access.|
|4530||For distributed cache and management calls|
|5432||Used for communication from Qpid/Management Server to Postgres|
|8084||Default management port on Postgres serverand must be open on the component for access by the Management Server.|
|4529||For distributed cache and management calls|
|8083||Default management port on Qpid server and must be open on the component for access by the Management Server.|
|Router||4527||For distributed cache and management calls.
A Router must open port 4527 as its management port. If you have multiple Routers, they must all be able to access each other over port 4527 (indicated by the loop arrow in the diagram above for port 4527 on the Router).
While it is not required, you can open port 4527 on the Router for access by any Message Processor. Otherwise, you might see error messages in the Message Processor log files.
|8081||Default management port for Router and must be open on the component for access by the Management Server.|
Health check port. A load balancer uses this port to determine if the Router is available.
To get the status of a Router, the load balancer makes a request to port 15999 on the Router:
curl -v http://routerIP:15999/v1/servers/self/reachable
If the Router is reachable, the request returns HTTP 200.
|59001||Port used for testing the Edge installation by the
|SmartDocs||59002||The port on the Edge router where SmartDocs page requests are sent.|
|ZooKeeper||2181||Used by other components like Management Server, Router, Message Processor and so on|
|2888, 3888||Used internally by ZooKeeper for ZooKeeper cluster (known as ZooKeeper ensemble) communication|
The next table shows the same ports, listed numerically, with the source and destination components:
|Port Number||Purpose||Source Component||Destination Component|
|virtual_host_port||HTTP plus any other ports you use for virtual host API call traffic. Ports 80 and 443 are most commonly used; the Message Router can terminate TLS/SSL connections.||External client (or load balancer)||Listener on Message Router|
|1099 through 1103||JMX Management||JMX Client||Management Server (1099)
Message Processor (1101)
Qpid Server (1102)
Postgres Server (1103)
|2181||Zookeeper client communication||Management Server
|2888 and 3888||Zookeeper internode management||Zookeeper||Zookeeper|
|4526||RPC Management port||Management Server||Management Server|
|4527||RPC Management port for distributed cache and management calls, and for communications between Routers||Management Server
|4528||For distributed cache calls between Message Processors, and for communication from the Router||Management Server
|4529||RPC Management port for distributed cache and management calls||Management Server||Qpid Server|
|4530||RPC Management port for distributed cache and management calls||Management Server||Postgres Server|
|5432||Postgres client||Qpid Server||Postgres|
||Qpid Server||Qpid Server|
|7000||Cassandra inter-node communications||Cassandra||Other Cassandra node|
|7199||JMX management. Must be open for access on the Cassandra node by the Management Server.||JMX client||Cassandra|
|8080||Management API port||Management API clients||Management Server|
|8081 through 8084||
Component API ports, used for issuing API requests directly to individual components. Each component opens a different port; the exact port used depends on the configuration but must be open on the component for access by the Management Server
|Management API clients||Router (8081)
Message Processor (8082)
Qpid Server (8083)
Postgres Server (8084)
|8443||Communication between Router and Message Processor when TLS is enabled||Router||Message Processor|
|8998||Communication between Router and Message Processor||Router||Message Processor|
|9000||Default Edge management UI port||Browser||Management UI Server|
|9042||CQL native transport||Router
|9160||Cassandra thrift client||Router
|10389||LDAP port||Management Server||OpenLDAP|
|15999||Health check port. A load balancer uses this port to determine if the Router is available.||Load balancer||Router|
|59001||Port used by the
|59002||The router port where SmartDocs page requests are sent||SmartDocs||Router|
A Message Processor keeps a dedicated connection pool open to Cassandra, which is configured to never timeout. When a firewall is between a Message Processor and Cassandra server, the firewall can time out the connection. However, the Message Processor is not designed to re-establish connections to Cassandra.
To prevent this situation, Apigee recommends that the Cassandra server, Message Processor, and Routers be in the same subnet so that a firewall is not involved in the deployment of these components.
If a firewall is between the Router and Message Processors, and has an idle TCP timeout set, our recommendations is to do the following:
net.ipv4.tcp_keepalive_time = 1800in sysctl settings on Linux OS, where 1800 should be lower than the firewall idle tcp timeout. This setting should keep the connection in an established state so that the firewall does not disconnect the connection.
- On all Message Processors, edit
/opt/apigee/customer/application/message-processor.propertiesto add the following property. If the file does not exist, create it.
- Restart the Message Processor:
/opt/apigee/apigee-service/bin/apigee-service edge-message-processor restart
- On all Routers, edit
/opt/apigee/customer/application/router.propertiesto add the following property. If the file does not exist, create it.
- Restart the Router:
/opt/apigee/apigee-service/bin/apigee-service edge-router restart