You're viewing **Apigee Edge** documentation.  
[Go to the
**Apigee X** documentation](http://cloud.google.com/apigee/docs/api-platform/errorcatalog/mp-runtime-errorcatalog). info

> [!NOTE]
> Was this troubleshooting playbook helpful? Please let us know by clicking [Send Feedback]().

## Symptom

The client application receives a **502 Bad Gateway** error. The Message Processor
returns this error to the client application when it does not receive a response
from a backend server.

## Error message

The client application receives the following response code:

    HTTP/1.1 502 Bad Gateway

In addition, you may observe the following error message:

    {
     "fault": {
        "faultstring":"Bad Gateway",
        "detail":{
            "errorcode":"messaging.adaptors.http.flow.BadGateway"
        }
     }
    }

## Possible Cause

The possible cause for this issue is listed in the following table:

|---|---|---|
| **Cause** | **Description** | **Troubleshooting steps can be performed by** |
| [TLS/SSL handshake timeout](https://docs.apigee.com/api-platform/troubleshoot/runtime/502-bad-gateway-timeout.md.txt#cause_tlsssl_handshake_timeout) | A timeout occurs during the TLS/SSL Handshake between the Message Processor and backend server. | Edge Private and Public Cloud users |

## Cause: TLS/SSL handshake timeout

In Apigee Edge, you can set up a [TLS/SSL connection to the backend server](https://docs.apigee.com/api-platform/system-administration/configuring-ssl-edge-backend-service.md.txt) to enable TLS communication
between the Edge Message Processor and a backend server.

A TLS/SSL handshake involves multiple steps. This error typically happens when
the TLS/SSL handshake between the Message Processor and a backend server times out.

## Diagnosis

This section explains how to correctly diagnose a TLS/SSL handshake timeout.
Instructions for Edge Private Cloud and Public Cloud are listed.

### Investigate Trace session output

The following steps explain how to do a preliminary diagnosis of the problem
using the Apigee Edge Trace tool.

1. In the Edge UI, enable a [Trace session](https://docs.apigee.com/api-platform/debug/using-trace-tool-0.md.txt) for the affected API proxy.
2. If the trace for the failing API request shows the following, it is likely that a TLS/SSL handshake timeout error occurred. The probable cause of the error is that the backend server firewall is blocking traffic from Apigee.

   1. Determine if the **502 Bad Gateway** error occurs after **55 seconds**, which is the default timeout period set on the Message Processor. If you see that the error occurred after 55 seconds, this tells you that a timeout was the likely cause of the problem.
   2. Determine if the error shows the fault: **messaging.adaptors.http.BadGateway**. Again, this error usually indicates a timeout occurred.
   3. If you are on **Edge Private Cloud** , note the value of the
      **X-Apigee.Message-ID** field in the trace output as shown below. A Private
      Cloud user can use this ID value to do further troubleshooting, as explained later.

      1. Click the **Analytics Data Recorded** icon in the trace path:

         ![](https://docs.apigee.com/static/api-platform/images/502-bad-gateway-timout-2.png)
      2. Scroll down and note the value of the field called **X-Apigee.Message-ID**.

To confirm that TLS/SSL Handshake timeout was the cause of the error, follow the steps in the following sections depending on whether you are on Public Cloud or Private Cloud.

### Additional diagnostic steps for Edge Private Cloud users only

If you are on Apigee Edge Private Cloud, you can perform the following steps to help verify the cause of the handshake error. In this step, you inspect the Message Processor log file for relevant information. If you are on Edge Public Cloud you can skip this section and go to [Further Diagnostic Steps for Private and Public Cloud Users](https://docs.apigee.com/api-platform/troubleshoot/runtime/502-bad-gateway-timeout.md.txt#further_diagnostic_steps_for_edge_private_and_public_cloud_users).

1. Check if you can connect to the specific backend server directly from each of the Message Processors using the `telnet` command:

   1. If the backend server resolves to a single IP address, then use this command:

      ```
      telnet BackendServer-IPaddress 443
      ```
   2. If the backend server resolves to multiple IP addresses, then use the hostname of the backend server in the telnet command as shown below:

      ```
      telnet BackendServer-HostName 443
      ```

   If you are able to connect to the backend server without any errors, go on to the next step.

   If the `telnet` command fails, you need to work with your network team to check the connectivity between the message processor and the backend server.
2. Check the Message Processor log file for evidence of a handshake failure. Open the file:

   `/opt/apigee/var/log/edge-message-processor/system.log`

   and search for the unique message ID (the value of **X-Apigee.Message-ID** that you
   found in the trace file). Determine if you see a handshake error message associated
   with the message ID as shown below:

       org:xxx env:xxx api:xxx rev:x messageid:<MESSAGE_ID> NIOThread@1 ERROR HTTP.CLIENT -
       HTTPClient$Context.handshakeTimeout() : SSLClientChannel[Connected: Remote:X.X.X.X:443
       Local:X.X.X.X]@739028 useCount=1 bytesRead=0 bytesWritten=0 age=55221ms lastIO=55221ms
       isOpen=true handshake timeout

If you see this error in the message processor's log file, continue further investigation. Go to [Further diagnostic steps for Edge Private and Public Cloud users](https://docs.apigee.com/api-platform/troubleshoot/runtime/502-bad-gateway-timeout.md.txt#further_diagnostic_steps_for_edge_private_and_public_cloud_users).

If you do not see the handshake message in the log file, go to [Must Gather Diagnostic Information](https://docs.apigee.com/api-platform/troubleshoot/runtime/502-bad-gateway-timeout.md.txt#must_gather_diagnostic_information)

### Further diagnostic steps for Edge Private and Public Cloud users

To further pinpoint the problem, you can use the [tcpdump](https://docs.apigee.com/api-platform/troubleshoot/diagnostic-tools-and-logs.md.txt#tcpippacketsniffertcpdumputility) tool to analyze TCP/IP packets to confirm if a timeout occurred during the TLS/SSL handshake.

1. If you are a **Private Cloud user**, you can capture the TCP/IP packets on the backend server or Message Processor. Preferably, capture them on the backend server, because the packets are decrypted on the backend server.
2. If you are a **Public Cloud user**, you do not have access to the Message Processor; however, capturing the TCP/IP packets on the backend server may help pinpoint a problem.
3. After deciding where to capture the TCP/IP packets, use the following
   [tcpdump](https://docs.apigee.com/api-platform/troubleshoot/diagnostic-tools-and-logs.md.txt#tcpippacketsniffertcpdumputility)
   command to capture TCP/IP packets.

       tcpdump -i any -s 0 host <IP address> -w <File name>

   - If you are taking the TCP/IP packets on the backend server, then use the public
     IP address of the Message Processor in the `tcpdump` command. For help using the
     command to examine backend server traffic, see [tcpdump](https://docs.apigee.com/api-platform/troubleshoot/diagnostic-tools-and-logs.md.txt#tcpippacketsniffertcpdumputility).

   - If you are taking the TCP/IP packets on the Message Processor, then use the public
     IP address of the backend server in the `tcpdump` command. For help using the command
     to examine Message Processor traffic, see [tcpdump](https://docs.apigee.com/api-platform/troubleshoot/diagnostic-tools-and-logs.md.txt#tcpippacketsniffertcpdumputility).

   - If there are multiple IP addresses for backend server/Message Processor, then
     you need to try another `tcpdump` command usage. Refer to [tcpdump](https://docs.apigee.com/api-platform/troubleshoot/diagnostic-tools-and-logs.md.txt#tcpippacketsniffertcpdumputility) for more information about this tool and
     for other variants of this command.

4. Analyze the TCP/IP packets using the [Wireshark tool](https://www.wireshark.org/)
   or a similar tool. The following screenshot shows TCP/IP packets in Wireshark.

   ![](https://docs.apigee.com/static/api-platform/images/502-bad-gateway-timout-1.png)
5. Notice in the Wireshark output, that the three-way TCP handshake completes
   successfully in the first 3 packets.

6. The Message Processor then sends the "Client Hello" message in packet #4.

7. Because there is no acknowledgement from the backend server, the Message
   Processor retransmits the "Client Hello" message multiple times in packets 5,
   6 and 7 after waiting for a predefined interval of time.

8. When the Message Processor doesn't receive any acknowledgement after 3 retries,
   it sends the **FIN, ACK** message to backend server to indicate that it is closing the connection.

9. As you shown in the example Wireshark session, the connection to the backend
   is successful (step #1), however, the SSL handshake timed out because the backend
   server never responded.

If you performed the troubleshooting steps in this playbook and determined that a
timeout caused the TLS/SSL handshake error, go to the [Resolution](https://docs.apigee.com/api-platform/troubleshoot/runtime/502-bad-gateway-timeout.md.txt#resolution) section.

### Using API Monitoring to identify a problem

> [!NOTE]
> **Note:** The Apigee Edge API Monitoring feature is available to Public Cloud users only.

[API Monitoring](https://docs.apigee.com/api-monitoring.md.txt) enables you to isolate
problem areas quickly to diagnose error, performance, and latency issues and their
source, such as developer apps, API proxies, backend targets, or the API platform.

[Step through a sample scenario](https://docs.apigee.com/api-monitoring/sample-scenarios.md.txt)
that demonstrates how to troubleshoot 5xx issues with your APIs using API Monitoring.
For example, you may want to set up an alert to be notified when the number of
**messaging.adaptors.http.BadGateway** faults exceeds a particular threshold.

## Resolution

Typically the SSL handshake timeouts occur because of firewall restrictions on
the backend server that blocks the traffic from Apigee Edge. If you followed
the diagnostic steps and determined that the cause of the handshake error is a
timeout, you need to engage your network team to identify the cause and fix the
firewall restrictions.

Note that the firewall restrictions could be imposed at different network layers.
It's important to make sure that the restrictions at all network layers are removed pertaining
to the Message Processor IPs to ensure smooth traffic flow between Apigee Edge
and the backend server.

If there are no firewall restrictions and/or the problem still persists, go to
[Must Gather Diagnostic Information](https://docs.apigee.com/api-platform/troubleshoot/runtime/502-bad-gateway-timeout.md.txt#must_gather_diagnostic_information).

## Must Gather Diagnostic Information

If the problem persists even after following the above instructions, please gather the following diagnostic information. Contact and share them to [Apigee Edge Support](https://docs.apigee.com/api-platform/troubleshoot/support.md.txt):

1. If you are a Public Cloud user, then provide the following information:
   1. Organization Name
   2. Environment Name
   3. API Proxy Name
   4. Complete curl command to reproduce the error
   5. Trace File showing the error
   6. TCP/IP packets captured on the backend server
2. If you are a Private Cloud user, provide the following information:
   1. Complete Error Message observed
   2. API Proxy bundle
   3. Trace File showing the error
   4. Message Processor logs /opt/apigee/var/log/edge-message-processor/logs/system.log
   5. TCP/IP packets captured on the backend server or Message Processor.
3. Details about what sections in this Playbook you have tried and any other insights that will help us to fastrack resolution of this issue.