Google is committed to advancing racial equity for Black communities. See how.

503 Service Unavailable - NoActiveTargets

Videos

See the following videos for more information on 503 errors:

Video Description
Troubleshoot and resolve 503 Service Unavailable - NoActiveTargets Learn about the following:
  • Importance of Target Servers and Health Monitors
  • Troubleshooting and resolving a real-time 503 Service Unavailable - NoActiveTargets error

Symptom

The client application receives the HTTP response status code 503 with the message Service Unavailable and the error code NoActiveTargets for the API proxy requests.

Error message

You will see the following error response:

HTTP/1.1 503 Service Unavailable
  

You will see the following error message in the HTTP response:

{
   "fault": {
      "faultstring": "The Service is temporarily unavailable",
      "detail": {
           "errorcode": "messaging.adaptors.http.flow.NoActiveTargets"
       }
    }
}
  

Possible causes

The HTTP response 503 Service Unavailable with the error code NoActiveTargets is typically observed when you use one or more target servers in the target endpoint configuration in your API Proxy.

The following table lists the possible causes for the 503 Service Unavailable with error code NoActiveTargets response:

Cause Description Who can perform the troubleshooting steps
Target Server is Disabled The target server specified in the target endpoint configuration is disabled. Edge Public and Private Cloud users
Connection Errors due to incorrect DNS resolution The DNS resolution of the target server resulted in bad IP addresses that lead to connection errors. Edge Private Cloud users
Connection Errors Network or connectivity issues prevent the client from connecting to the server. Edge Private Cloud users
Incorrect Target Host Alias The target server host specified is incorrect or has unwanted characters (such as a space). Edge Public and Private Cloud users
SSL Handshake Failures The TLS/SSL handshake failed between the client and server. Edge Public and Private Cloud users
Health Check Failures Health checks configured to check the health of the target server may fail due to some reason. Edge Private Cloud users

Cause: Target server is disabled

If all the target servers specified in the target endpoint configuration are disabled, then you get 503 Service Unavailable response with the error code NoActiveTargets.

Diagnosis

  1. Determine the name of the target server being used in the specific Target Endpoint Configuration of the failing API Proxy using one of the following ways:
    1. If there’s a single target endpoint, then check that particular target endpoint.
    2. If there are multiple target endpoints and if you are unsure which one has disabled target server, then follow the steps below:
      1. Enable the trace session, make the API call and reproduce the issue - 503 Service Unavailable.
      2. From the trace, navigate to Target Request Flow Started and determine the name of the target endpoint as shown below:
      3. Determining target endpoint name from trace

  2. Once you’ve identified the target endpoint, get the target server name used from the target endpoint configuration as shown in the example below:
    <TargetEndpoint name="default">>
      <HTTPTargetConnection>
        <LoadBalancer>
          <Server name="demo-target" />
        </LoadBalancer>
        <Path>/test</Path>
      </HTTPTargetConnection>
    </TargetEndpoint>
          

    In the example above, there is a single target server named demo-target.

  3. Get the definition for each of the target servers used in the target endpoint using Edge UI or Edge API call.

    Edge UI

    To get the definition using the Edge UI:

    1. Navigate to Admin > Environments > Target Servers.
    2. Select the specific environment in which you are seeing the failure.
    3. Search for the specific target server name to get the target server definition.

      For example, type in the target server name demo-target and you will see its definition as shown below:

      Target server name and enabled/disabled status

      Notice here that the target server demo-target has a host alias, port # and SSL is enabled. However, the target server itself is disabled, which is indicated by the element ENABLED being greyed out.

    Edge API

    To get the definition using the Edge API:

    Use the Get Target Server API to get the target server definition.

    Target Server Definition Output

    <TargetServer name="demo-target">
      <Host>demo-target.apigee.net</Host>
      <Port>443</Port>
      <IsEnabled>false</IsEnabled>
      <SSLInfo>
          <Enabled>true</Enabled>
      </SSLInfo>
    </TargetServer>
              

    The Apigee API output shows that the target server demo-target is disabled as the element IsEnabled is set to false.

    Since the target server is disabled, the Message Processor will send 503 Service Unavailable with the error code NoActiveTargets immediately as a response to the client.

Resolution

Ensure that the specific target servers used in the target endpoint configuration of your API Proxy are always enabled.

Edge UI

  1. Navigate to Admin > Environments > Target Servers.
  2. Select the specific environment in which you are seeing the failure.
  3. Search for the specific target server name to get its definition.
  4. Select the specific target server and then click Edit.
  5. Select the Enabled checkbox.
  6. Click Update.

Edge API

Use the Update a Target Server API to update the target server definition and ensure that IsEnabled is set to true in the request payload of the API as shown below:

<TargetServer name="demo-target">
  <Host>demo-target.apigee.net</Host>
  <Port>443</Port>
  <IsEnabled>true</IsEnabled>
  <SSLInfo>
      <Enabled>true</Enabled>
  </SSLInfo>
</TargetServer>
        

If the problem still persists, go to Must Gather Diagnostic Information.

Diagnose issues using API monitoring

API Monitoring enables you to isolate problem areas quickly to diagnose error, performance, and latency issues and their source, such as developer apps, API proxies, backend targets, or the API platform.

Step through a sample scenario that demonstrates how to troubleshoot 5xx issues with your APIs using API Monitoring. For example, you may want to set up an alert to be notified when the number of messaging.adaptors.http.flow.NoActiveTargets faults exceeds a particular threshold.

Must gather diagnostic information

If the problem persists even after following the instructions above, please gather the following diagnostic information. Contact and share them to Apigee Support:

  1. If you are a Public Cloud user, provide the following information:
    1. Organization Name
    2. Environment Name
    3. API Proxy Name
    4. Complete curl command to reproduce the error
    5. Trace file containing the requests with 503 Service Unavailable with error code NoActiveTargets
  2. If you are a Private Cloud user, provide the following information:
    1. Complete Error Message observed
    2. Environment Name
    3. API Proxy bundle
    4. Trace file containing the requests with 503 Service Unavailable with error code NoActiveTargets
    5. Nginx Access Logs

      (/opt/apigee/var/log/edge-router/nginx/<org>~<env>.<port#>_access_log)

    6. Message Processor Logs

      (/opt/apigee/var/log/edge-message-processor/logs/system.log)