Load balancing across backend servers

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

Apigee Edge enhances the availability of your API by providing built-in support for load balancing and failover across multiple backend server instances.

TargetServer configurations decouple concrete endpoint URLs from TargetEndpoint configurations. Each TargetServer is referenced by name in a TargetEndpoint HTTPConnection. Instead of defining a concrete URL in the configuration, you can configure one or more named TargetServers as described in the section TargetEndpoint.

A TargetServer definition consists of a name, a host and a port, with an additional element to indicate whether the TargetServer is enabled or disabled.

Videos

Watch the following videos to learn more about API routing and load balancing using target servers

Video Description
Load balancing using target servers Load balancing APIs across target servers.
API routing based on environment using target servers Route an API to a different target server based on the environment.
API routing and load balancing using target servers (Classic Edge) Route an API to a different target server based on the environment and load balance your API across target servers in the Classic Edge UI.

Sample TargetServer Configuration

The following code defines a target server:

<TargetServer  name="target1">
  <Host>1.mybackendservice.com</Host>
  <Port>80</Port>
  <IsEnabled>true</IsEnabled>
</TargetServer >

TargetServer Configuration Elements

The following table describes the elements used to create and configure a TargetServer:

Name Description Default Required?
name The name of the TargetServer configuration, which must be unique within the environment. The TargetServer name can contain only alphanumeric characters. N/A Yes
Host

The host URL of the backend service (without the protocol).

N/A Yes
Port The port on which the backend service is listening N/A Yes
IsEnabled A boolean that indicates whether the TargetServer configuration is enabled or disabled. This enables you to take TargetServers out of rotation without modifying the API proxy configuration. A common usage would be to write an app or script that enables or disables TargetServers automatically based on expected capacity requirements, maintenance schedules, etc. true Yes

Managing target servers using the UI

Manage target servers, as described below..

Edge

To manage target servers using the Edge UI:

  1. Sign in to apigee.com/edge.
  2. Select Admin > Environments > Target Servers in the left navigation bar.
  3. Select the desired environment, such as test or prod.
  4. To create a target server:
    1. Click + Target server.
    2. Enter a name, host, and port for the target server.

      For example:

      • Name: target1
      • Host: 1.mybackendservice.com
      • Port: 80
    3. Select SSL, if required.
    4. Select Enabled to enable the target server.
    5. Click Add.
  5. To edit the target server:
    1. Position your cursor over the target server that you want to edit to display the actions menu.
    2. Click .
    3. Edit the targer server values.
    4. Click Update.
  6. To delete the target server:
    1. Position your cursor over the target server that you want to delete to display the actions menu.
    2. Click .
    3. Click Delete to confirm the operation.

Classic Edge (Private Cloud)

To access the Create Proxy wizard using the Classic Edge UI:

  1. Sign in to http://ms-ip:9000, where ms-ip is the IP address or DNS name of the Management Server node.
  2. Select APIs > Environment Configuration > Target Servers in the left navigation bar.
  3. Select the desired environment, such as test or prod.
  4. To create a target server:
    1. Click Edit.
    2. Click + Target server.
    3. Enter a name, host, and port for the target server.

      For example:

      • Name: target1
      • Host: 1.mybackendservice.com
      • Port: 80
    4. Select Enabled to enable the target server.
    5. Click Save.
  5. To edit the target server:
    1. Click Edit.
    2. Edit the targer server values.
    3. Click Save.
  6. To delete the target server:
    1. Click Edit.
    2. Click Delete.

Managing target servers using the API

You can use Edge API to create, delete, update, get, and list target servers. For more information, see TargetServers.

Use the following API call to create a target server:

$ curl -H "Content-Type:text/xml" -X POST -d \
'<TargetServer name="target1">
   <Host>1.mybackendservice.com</Host>
   <Port>80</Port>
   <IsEnabled>true</IsEnabled>
 </TargetServer>' \
-u email:password https://api.enterprise.apigee.com/v1/o/{org_name}/environments/test/targetservers

Sample Response:

{
  "host" : "1.mybackendservice.com",
  "isEnabled" : true,
  "name" : "target1",
  "port" : 80
}

After you create the first TargetServer, then use the following API call to create a second TargetServer. By defining two TargetServers, you provide two URLs that a TargetEndpoint can use for load balancing:

$ curl -H "Content-type:text/xml" -X POST -d \
'<TargetServer  name="target2">
  <Host>2.mybackendservice.com</Host>
  <Port>80</Port>
  <IsEnabled>true</IsEnabled>
</TargetServer >' \
-u email:password https://api.enterprise.apigee.com/v1/o/{org_name}/environments/test/targetservers

Sample Response:

{
  "host" : "2.mybackendservice.com",
  "isEnabled" : true,
  "name" : "target2",
  "port" : 80
}

Use the following API call to retrieve a list of TargetServers in an environment:

$ curl -u email:password https://api.enterprise.apigee.com/v1/o/{org_name}/environments/test/targetservers

Sample response:

[ "target2", "target1" ]

There are now two TargetServers available for use by API proxies deployed in the test environment. To load balance traffic across these TargetServers, you configure the HTTP connection in an API proxy's target endpoint to use the TargetServers.

There is a limit of 500 TargetServers per environment, as documented in the Limits topic.

Configuring a TargetEndpoint to load balance across named TargetServers

Now that you have two TargetServers available, you can modify the TargetEndpoint HTTP connection setting to reference those two TargetServers by name:

<TargetEndpoint name="default">
  <HTTPTargetConnection>
    <LoadBalancer>
      <Server name="target1" />
      <Server name="target2" />
    </LoadBalancer>
    <Path>/test</Path>
  </HTTPTargetConnection>
</TargetEndpoint>

The configuration above is the most basic load balancing configuration possible. The load balancer supports three load balancing algorithms, Round Robin, Weighted, and Least Connection. Round Robin is the default algorithm. Since no algorithm is specified in the configuration above, outbound requests from the API proxy to the backend servers will alternate, one for one, between target1 and target 2.

The <Path> element forms the basepath of the TargetEndpoint URI for all target servers. It is only used when <LoadBalancer> is used. Otherwise, it is ignored. In the above example, a request reaching to "target1" will be http://target1/test and so for other target servers.

Setting load balancer options

You can tune availability by using options for load balancing and failover at the load balancer and TargetServer level. This section describes these options.

Algorithm

Sets the algorithm used by <LoadBalancer>. The available algorithms are RoundRobin, Weighted, and LeastConnections, each of which is documented below.

Round robin

The default algorithm, round robin, forwards a request to each TargetServer in the order in which the servers are listed in the target endpoint HTTP connection. For example:

<TargetEndpoint name="default">
  <HTTPTargetConnection>
      <LoadBalancer>
        <Algorithm>RoundRobin</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
      </LoadBalancer>
      <Path>/test</Path>
  </HTTPTargetConnection>
</TargetEndpoint>

Weighted

The weighted load balancing algorithm enables you to configure proportional traffic loads for your TargetServers. The weighted LoadBalancer distributes request to your TargetServers in direct proportion to each TargetServer 's weight. Therefore, the weighted algorithm requires you to set a weight attribute for each TargetServer. For example:

<TargetEndpoint name="default">
  <HTTPTargetConnection>
    <LoadBalancer>
      <Algorithm>Weighted</Algorithm>
      <Server name="target1">
        <Weight>1</Weight>
      </Server>
      <Server name="target2">
        <Weight>2</Weight>
      </Server>
    </LoadBalancer>
    <Path>/test</Path>
  </HTTPTargetConnection>
</TargetEndpoint>

In this example, two requests will be routed to target2 for every one request routed to target1.

Least Connection

LoadBalancers configured to use the least connection algorithm route outbound requests to the TargetServer with fewest open HTTP connections. For example:

<TargetEndpoint name="default">
  <HTTPTargetConnection>
      <LoadBalancer>
        <Algorithm>LeastConnections</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
      </LoadBalancer>
  </HTTPTargetConnection>
  <Path>/test</Path>
</TargetEndpoint>

Maximum failures

The maximum number of failed requests from the API proxy to the TargetServer that results in the request being redirected to another TargetServer.

A response failure means Apigee doesn't receive any response from a target server. When this happens, the failure counter increments by one.

However, when Apigee does receive a response from a target, even if the response is an HTTP error (such as 500), that counts as a response from the target server, and the failure counter is reset. To help ensure that bad HTTP responses (such as 500) also increment the failure counter to take an unhealthy server out of load balancing rotation as soon as possible, you can add the <ServerUnhealthyResponse> element with <ResponseCode> child elements to your load balancer configuration. Edge will also count responses with those codes as failures.

In the following example, target1 will be removed from rotation after five failed requests, including some 5XX responses from the target server.

<TargetEndpoint name="default">
  <HTTPTargetConnection>
      <LoadBalancer>
        <Algorithm>RoundRobin</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
        <MaxFailures>5</MaxFailures>
        <ServerUnhealthyResponse>
            <ResponseCode>500</ResponseCode>
            <ResponseCode>502</ResponseCode>
            <ResponseCode>503</ResponseCode>
        </ServerUnhealthyResponse>
      </LoadBalancer>
      <Path>/test</Path>
  </HTTPTargetConnection>
</TargetEndpoint>

The MaxFailures default is 0. This means that Edge always tries to connect to the target for each request and never removes the target server from the rotation.

It is best to use MaxFailures > 0 with a HealthMonitor. If you configure MaxFailures > 0, the TargetServer is removed from rotation when the target fails the number of times you indicate. When a HealthMonitor is in place, Apigee automatically puts the TargetServer back in rotation after the target is up and running again, according to the configuration of that HealthMonitor. See Health monitoring for more information.

Alternatively, if you configure MaxFailures > 0, and you do not configure a HealthMonitor, Apigee will not re-include the TargetServer into the rotation automatically after Apigee detects a failure. In this case, you must redeploy the API proxy before Apigee puts the TargetServer back into rotation. See Deploying an API proxy.

Retry

If retry is enabled, a request will be retried whenever a response failure (I/O error or HTTP timeout) occurs or the response received matches a value set by <ServerUnhealthyResponse>. See Maximum failures above for more on setting <ServerUnhealthyResponse>.

By default <RetryEnabled> is set to true. Set to false to disable retry. For example:

<RetryEnabled>false</RetryEnabled>

IsFallback

One (and only one) TargetServer can be set as the 'fallback' server. The fallback TargetServer is not included in load balancing routines until all other TargetServers are identified as unavailable by the load balancer. When the load balancer determines that all TargetServers are unavailable, all traffic is routed to the fallback server. For example:

<TargetEndpoint name="default">
  <HTTPTargetConnection>
      <LoadBalancer>
        <Algorithm>RoundRobin</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
        <Server name="target3">
          <IsFallback>true</IsFallback>
        </Server>
      </LoadBalancer>
      <Path>/test</Path>
  </HTTPTargetConnection>
</TargetEndpoint>

The configuration above results in round robin load balancing between targets 1 and 2 until both targets 1 and 2 are unavailable. When targets 1 and 2 are unavailable, all traffic is routed to target 3.

Path

Path defines a URI fragment that will be appended to all requests issued by the TargetServer to the backend server.

This element accepts a literal string path or a message template. A message template lets you perform variable string substitution at runtime. For example, in the following target endpoint definition, the value of {mypath} is used for the path:

<HTTPTargetConnection>
    <SSLInfo>
      <Enabled>true</Enabled>
    </SSLInfo>
    <LoadBalancer>
      <Server name="testserver"/>
    </LoadBalancer>
    <Path>{mypath}</Path>
</HTTPTargetConnection>

Configuring a target server for TLS/SSL

If you are using a TargetServer to define the backend service, and the backend service requires the connection to use the HTTPS protocol, then you must enable TLS/SSL in the TargetServer definition. This is necessary because the <Host> tag does not let you specify the connection protocol. Shown below is the TargetServer definition for one-way TLS/SSL where Edge makes HTTPS requests to the backend service:

<TargetServer name="target1">
  <Host>mocktarget.apigee.net</Host>
  <Port>443</Port>
  <IsEnabled>true</IsEnabled>
  <SSLInfo>
      <Enabled>true</Enabled>
  </SSLInfo> 
</TargetServer>

If the backend service requires two-way, or mutual, TLS/SSL, then you configure the TargetServer by using the same TLS/SSL configuration settings as TargetEndpoints:

<TargetServer  name="TargetServer 1">
    <IsEnabled>true</IsEnabled>
    <Host>www.example.com</Host>
    <Port>443</Port>
    <SSLInfo>
        <Ciphers/>
        <ClientAuthEnabled>true</ClientAuthEnabled>
        <Enabled>true</Enabled>
        <IgnoreValidationErrors>false</IgnoreValidationErrors>
        <KeyAlias>keystore-alias</KeyAlias>
        <KeyStore>keystore-name</KeyStore>
        <Protocols/>
        <TrustStore>truststore-name</TrustStore>
    </SSLInfo>
</TargetServer >

For information on the <SSLInfo> properties, such as <Ciphers> and <ClientAuthEnabled>, see the information on setting those properties for a Virtual Host at Configuring TLS access to an API for the Private Cloud.

For complete instructions on configuring outbound TLS/SSL, see Configuring TLS from Edge to the backend (Cloud and Private Cloud).

TargetServer schema

See the schema for TargetServer and other entities on GitHub.

Health monitoring

Health monitoring enables you to enhance load balancing configurations by actively polling the backend service URLs defined in the TargetServer configurations. With health monitoring enabled, a failed TargetServer is automatically put back into rotation when the HealthMonitor determines that the TargetServer is active.

Health monitoring works with <MaxFailures>. Without health monitoring enabled, <MaxFailures> specifies the number of failed requests from the API proxy to the TargetServer that results in the request being redirected to another TargetServer. The failing TargetServer is then taken out of rotation until you redeploy the proxy.

With health monitoring enabled, a failed TargetServer is automatically put back into rotation and no proxy re-deployments are required.

The HealthMonitor acts as a simple client that invokes a backend service over TCP or HTTP:

  • A TCP client simply ensures that a socket can be opened.
  • You configure the HTTP client to submit a valid HTTP request to the backend service. You can define HTTP GET, PUT, POST, or DELETE operations. The response of the HTTP monitor call must match the configured settings in the <SuccessResponse> block.

Successes and failures

When you enable health monitoring, Edge begins sending health checks to your target server. A health check is a request sent to the target server that determines whether the target server is healthy or not.

A health check can have one of two possible results:

  • Success: The target server is considered healthy when a successful health check occurs. This is typically the result of one or more of the following:
    • Target server accepts a new connection to the specified port, responds to a request on that port, and then closes the port within the specified timeframe. The response from the target server contains “Connection: close”
    • Target server responds to a health check request with a 200 (OK) or other HTTP status code that you determine is acceptable.
    • Target server responds to a health check request with a message body that matches the expected message body.

    When Edge determines that a server is healthy, Edge continues or resumes sending requests to it.

  • Failure: The target server can fail a health check in different ways, depending on the type of check. A failure can be logged when the target server:
    • Refuses a connection from Edge to the health check port.
    • Does not respond to a health check request within a specified period of time.
    • Returns an unexpected HTTP status code.
    • Responds with a message body that does not match the expected message body.

    When a target server fails a health check, Edge increments that server’s failure count. If the number of failures for that server meets or exceeds a predefined threshold (<MaxFailures>), Edge stops sending requests to that server.

Enabling a HealthMonitor

To create a HealthMonitor, you add the <HealthMonitor> element to the TargetEndpoint's HTTPConnection configuration for a proxy. You cannot do this in the UI. Instead, you create a proxy configuration and upload it as a ZIP file to Edge. A proxy configuration is a structured description of all aspects of an API proxy. Proxy configurations consist of XML files in a pre-defined directory structure. For more information, see API Proxy Configuration Reference.

A simple HealthMonitor defines an IntervalInSec combined with either a TCPMonitor or an HTTPMonitor. The <MaxFailures> element specifies maximum number of failed requests from the API proxy to the TargetServer that results in the request being redirected to another TargetServer. By default <MaxFailures> is 0, which means Edge performs no corrective action. When configuring a health monitor, ensure that you set <MaxFailures> in the <HTTPTargetConnection> tag of the <TargetEndpoint> tag to a non-zero value.

TCPMonitor

The configuration below defines a HealthMonitor that polls each TargetServer by opening a connection on port 80 every five seconds. (Port is optional. If not specified, the TCPMonitor port is the TargetServer port.)

  • If the connection fails or takes more than 10 seconds to connect, then the failure count increments by 1 for that TargetServer.
  • If the connection succeeds, then the failure count for the TargetServer is reset to 0.

You can add a HealthMonitor as a child element of the TargetEndpoint's HTTPTargetConnetion element, as shown below:

<TargetEndpoint name="default">
  <HTTPTargetConnection>
      <LoadBalancer>
        <Algorithm>RoundRobin</Algorithm>
        <Server name="target1" />
        <Server name="target2" />
        <MaxFailures>5</MaxFailures>
      </LoadBalancer>
      <Path>/test</Path>
      <HealthMonitor>
        <IsEnabled>true</IsEnabled>
        <IntervalInSec>5</IntervalInSec>
        <TCPMonitor>
            <ConnectTimeoutInSec>10</ConnectTimeoutInSec>
            <Port>80</Port>
        </TCPMonitor>
      </HealthMonitor>
  </HTTPTargetConnection>
. . .

HealthMonitor with TCPMonitor configuration elements

The following table describes the TCPMonitor configuration elements:

Name Description Default Required?
IsEnabled A boolean that enables or disables the HealthMonitor. false No
IntervalInSec The time interval, in seconds, between each polling TCP request. 0 Yes
ConnectTimeoutInSec Time in which connection to the TCP port must be established to be considered a success. Failure to connect in the specified interval counts as a failure, incrementing the load balancer's failure count for the TargetServer. 0 Yes
Port Optional. The port on which the TCP connection will be established. If not specified, the TCPMonitor port is the TargetServer port. 0 No

HTTPMonitor

A sample HealthMonitor that uses an HTTPMonitor will submit a GET request to the backend service once every five seconds. The sample below adds an HTTP Basic Authorization header to the request message. The Response configuration defines settings that will be compared against actual response from the backend service. In the example below, the expected response is an HTTP response code 200 and a custom HTTP header ImOK whose value is YourOK. If the response does not match, then the request will treated as a failure by the load balancer configuration.

The HTTPMonitor supports backend services configured to use HTTP and one-way HTTPS protocols. However, it does not support the following:

  • Two-way HTTPS (also called two-way TLS/SSL)
  • Self-signed certificates.

Note that all of the Request and Response settings in an HTTP monitor will be specific to the backend service that must be invoked.

    <HealthMonitor>
      <IsEnabled>true</IsEnabled>
      <IntervalInSec>5</IntervalInSec>
      <HTTPMonitor>
        <Request>
          <IsSSL>true</IsSSL>
          <ConnectTimeoutInSec>10</ConnectTimeoutInSec>
          <SocketReadTimeoutInSec>30</SocketReadTimeoutInSec>
          <Port>80</Port>
          <Verb>GET</Verb>
          <Path>/healthcheck</Path>
          <Header name="Authorization">Basic 12e98yfw87etf</Header>
          <IncludeHealthCheckIdHeader>true</IncludeHealthCheckIdHeader>
        </Request>
        <SuccessResponse>
          <ResponseCode>200</ResponseCode>
          <Header name="ImOK">YourOK</Header>
        </SuccessResponse>
      </HTTPMonitor>
    </HealthMonitor>

HealthMonitor with HTTPMonitor configuration elements

The following table describes the HTTPMonitor configuration elements:

Name Description Default Required?
IsEnabled A boolean that enables or disables the HealthMonitor. false No
IntervalInSec The time interval, in seconds, between each polling request. 0 Yes
Request

Configuration options for the outbound request message sent by the HealthMonitor to the TargetServers in the rotation.

The Path does not support variables.

 N/A Yes
IsSSL Specifies whether to use HTTPS (secure HTTP) for monitoring connections.

Potential values:
  • true: HTTPs is used.
  • false: HTTP is used.
  • Unspecified: Uses the target server configuration.
 false No
ConnectTimeoutInSec Time, in seconds, in which the TCP connection handshake to the HTTP service must complete to be considered a success. Failure to connect in the specified interval counts as a failure, incrementing the LoadBalancer's failure count for the TargetServer. 0 No
SocketReadTimeoutInSec Time, in seconds, in which data must be read from the HTTP service to be considered a success. Failure to read in the specified interval counts as a failure, incrementing the LoadBalancer's failure count for the TargetServer. 0 No
Port The port on which the HTTP connection to the backend service will be established. N/A No
Verb The HTTP verb used for each polling HTTP request to the backend service . N/A No
Path The path appended to the URL defined in the TargetServer. Use the path element to configure a 'polling endpoint' on your HTTP service. N/A No

IncludeHealthCheckIdHeader

 Allows you to track the healthcheck requests on upstream systems. The IncludeHealthCheckIdHeader takes a Boolean value, and defaults to false. If you set it to true, then there is a Header named X-Apigee-Healthcheck-Id which gets injected into the healthcheck request. The value of the header is dynamically assigned, and takes the form ORG/ENV/SERVER_UUID/N, where ORG is the organization name, ENV is the environment name, SERVER_UUID is a unique ID identifying the MP, and N is the number of milliseconds elapsed since January 1, 1970.

Example resulting request header:

X-Apigee-Healthcheck-Id: orgname/envname/E8C4D2EE-3A69-428A-8616-030ABDE864E0/1586802968123 		false No
Payload The HTTP body generated for each polling HTTP request. Note that this element is not required for GET requests. N/A No
SuccessResponse Matching options for the inbound HTTP response message generated by the polled backend service. Responses that do not match increment the failure count by 1. N/A No
ResponseCode The HTTP response code expected to be received from the polled TargetServer. A code different than the code specified results in a failure, and the count being incremented for the polled backend service. You can define multiple ResponseCode elements. N/A No
Headers A list of one or more HTTP headers and values expected to be received from the polled backend service. Any HTTP headers or values on the response that are different from those specified result in a failure, and the count for the polled TargetServer is incremented by 1. You can define multiple Header elements. N/A No