502 Bad Gateway - Self-signed certificate in chain

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

Symptom

The client application receives an HTTP response code of 502 with the message Bad Gateway as a response for API calls in Edge Microgateway.

Alternatively, the administrator will get a self signed certificate in certificate chain error when running the edgemicro configure command.

Error message

The client will see the following response message: 

HTTP/1.1 502 Bad Gateway

Two common examples of error responses are: 

{"message":"self signed certificate in certificate chain","code":"SELF_SIGNED_CERT_IN_CHAIN"}

{"message":"self signed certificate","code":"DEPTH_ZERO_SELF_SIGNED_CERT"}

Alternatively, this error can occur when running edgemicro configure: 

{ Error: self signed certificate in certificate chain
at TLSSocket.onConnectSecure (_tls_wrap.js:1051:34)
at TLSSocket.emit (events.js:189:13)
at TLSSocket._finishInit (_tls_wrap.js:633:8) code: 'SELF_SIGNED_CERT_IN_CHAIN' }

Possible causes

Cause Description Troubleshooting instructions applicable for
Target server presents a self-signed certificate Edge Microgateway will verify the target server's certificate, and if it's not trusted it will raise a runtime error. Edge Public and Private Cloud users
Apigee Edge Management Server uses a self-signed certificate When configuring Edge Microgateway for the first time, it will connect to Apigee Edge over TLS to bootstrap. If Edge presents a self-signed certificate, this will fail. Edge Private Cloud users

Cause: Target server presents a self-signed certificate

If a self-signed certificate is presented by the target server at the southbound connection, then Edge Microgateway would, by default, raise this error because it does not trust self-signed certificates.

Diagnosis

You may observe the following error in the logs (/var/tmp/edgemicro-`hostname`- *.log): 

2021-05-18T10:52:46.425Z [error][0:8000][1][gsc][test][edgemicro_badtargethost][][][2db53f80-
b7c7-11eb-9abe-05b6297863f1][microgateway-core][][GET][502][self signed certificate in certificate
chain][SELF_SIGNED_CERT_IN_CHAIN][]

The error code SELF_SIGNED_CERT_IN_CHAIN indicates that the Edge Microgateway has very likely received a self-signed certificate from the target server. To confirm this, perform the following steps:

  1. Run the following openssl command to verify the target server’s certificate chain: 
    echo | openssl s_client -connect TARGET_SERVER_HOSTNAME:PORT -servername TARGET_SERVER_HOSTNAME | openssl x509 -noout

  2. If the target server’s certificate chain is indeed self-signed, then this is the cause of the issue.

    In the following example, notice that the target server presents a self-signed certificate: 

    echo | openssl s_client -connect untrusted-root.badssl.com:443 -servername untrusted-root.badssl.com | openssl x509 -noout

    depth=1 C = US, ST = California, L = San Francisco, O = BadSSL, CN = BadSSL Untrusted Root Certificate Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
DONE

Resolution

  1. Work with the team owning the target server to procure a proper TLS certificate signed by a trusted Certificate Authority (CA).

  2. If that is not possible, then consider one of the following options to allow self-signed certificates in Edge Microgateway.

    Option #1: Set a system property to allow Edge Microgateway to trust all certificates

    1. If you are using docker, then refer to Using a CA that is not trusted by Node.js

    2. Otherwise, export an environment variable called NODE_EXTRA_CA_CERTS, pointing to the root CA file.

      This is documented on the official Node.js website.

    Option #2: Configure the Edge Microgateway YAML config file to trust that specific certificate for that target server

    1. Ensure that you have the target server's certificate (or chain) in PEM format. To convert other certificate formats to PEM, follow the instructions in Converting certificates to supported format.

    2. If there is a certificate chain, ensure that the certificates are in the correct order. The leaf certificate should always be first, followed by the intermediate certificate, then the root certificate. There is more explanation on this in Validating certificate chain.

      In the following example, we have configured the trusted CA file for untrusted-root.badssl.com. 

      edgemicro:
...
targets:
  - host: 'untrusted-root.badssl.com'
    ssl:
      client
        ca: /opt/apigee/certs/untrusted-root.pem

    The instructions for configuring this are also covered in the Edge Microgateway Module - Configure 1-way and 2-way Southbound TLS video. Refer to Configuring SSL on the Edge Microgateway server for further information.

If the problem still persists, go to Must gather diagnostic information.

Cause: Apigee Edge Management Server uses a self-signed certificate

When Edge Microgateway is set up for the first time, one of the commands you will need to run is edgemicro configure or edgemicro private configure. This command will bootstrap the cluster, and it will contact Apigee Edge to download the required information.

For Edge Private Cloud, the Management Server URL is determined by the -m argument. If you have enabled TLS for the Management Server, then Edge Microgateway would attempt to verify the certificate presented by the Management Server.

An example edgemicro configure command for Edge Private Cloud is as follows: 

edgemicro private configure -u <username> -p <password> -o apigee -e dev -v secure -r https://apigee-dev.net -m https://management.apigee-dev.net:8443

If the Management Server is configured with a self-signed certificate, then you will get the following error in the console output. 

{ Error: self signed certificate in certificate chain
at TLSSocket.onConnectSecure (_tls_wrap.js:1051:34)
at TLSSocket.emit (events.js:189:13)
at TLSSocket._finishInit (_tls_wrap.js:633:8) code: 'SELF_SIGNED_CERT_IN_CHAIN' }

Diagnosis

  1. In this case, the Management Server (management.apigee-dev.net) may be returning a self-signed TLS certificate.
  2. It is likely that your Apigee Edge system administrator has provided the certificate and has a copy of it.
  3. Otherwise, run the following command to get information about the certificate: 
    echo | openssl s_client -connect management.apigee-dev.net:8443 -servername management.apigee-dev.net | openssl x509 -noout
  4. If the Management Server has a self-signed certificate, then that’s the cause of this issue.

Resolution

  1. Work with the team owning the target server to procure a proper TLS certificate signed by a trusted Certificate Authority (CA).

  2. If that is not possible, then do the following to allow self-signed certificates in Edge Microgateway.

  3. Set a system property to allow Edge Microgateway to trust all certificates.
  4. If you are using docker, then refer to Using a CA that is not trusted by Node.js.
  5. Otherwise, export an environment variable called NODE_EXTRA_CA_CERTS, pointing to the root CA file.This is documented on the official Node.js website.

Must gather diagnostic information

If the problem persists even after following the above instructions, gather the following diagnostic information and then contact Apigee Edge Support:

  • Log files: The default folder is /var/tmp but it may be overridden in the main config.yaml file (logging > dir parameter). It is recommended to change the log > level to info before providing the log files to Apigee Edge Support.
  • Configuration file: The main configuration of Edge Microgateway resides in the YAML file in the default Edge Microgateway folder, $HOME/.edgemicro. There is a default config file called default.yaml and then one for each environment ORG-ENV-config.yaml. Upload this file in full for the impacted org and env.

    Reference documents

    Configure the Edge UI to use TLS to access the Edge API