404 Unable to identify proxy for host: <virtual host name> and url: <path>

Symptom

The client application gets an HTTP status code of 404 with the message "Not Found" and the error message as "Unable to identify proxy for host: and url: " as a response to the API calls.

This error means that Edge could not find the API Proxy for the specified virtual host and path.

Error Message

You will get the following HTTP status code:

HTTP/1.1 404 Not Found

You will also observe an error message something similar to the one shown below:

{
   "fault":{
      "faultstring":"Unable to identify proxy for host: default and url: \/oauth2\/token",
      "detail":{
         "errorcode":"messaging.adaptors.http.flow.ApplicationNotFound"
      }
   }
}

The above error message indicates that Edge could not find the API Proxy for the "default" virtual host and "/oauth2/token" path.

Possible Causes

Some of the possible causes for this error are listed below:

Cause Description Troubleshooting Instructions Applicable for
API Proxy not associated with the specific VirtualHost If the specific API Proxy is not configured to accept requests on the VirtualHost specified in the error message. Edge Public and Private Cloud Users
Path not associated with any API Proxy If the specific API Proxy is not configured to accept requests on the path specified in the error message. Edge Public and Private Cloud Users
API Proxy not deployed in an environment If the specific API Proxy is not deployed in the specific environment in which you are trying to make the API requests. Edge Public and Private Cloud Users
Environment not loaded on the Message Processor If the specific environment (in which you are trying to make the API requests) has not been loaded on the Message Processor (s) due to an error. Edge Private Cloud Users
API Proxy not deployed on one or more Message Processors The API Proxy may not be deployed on one or more Message Processors due to missing event notification during deployment. Edge Private Cloud Users

Cause: API Proxy not associated with the specific Virtual Host

If the API Proxy is not configured to accept the requests for the specific Virtual Host, then we can get a 404 Not Found response with the error message "Unable to identify proxy for host: and url: ".

Diagnosis

  1. Check the Proxy Endpoint configuration for the API Proxy and see if the API Proxy is configured to accept the requests for the VirtualHost specified in the error. This is indicated by the VirtualHost element. Let's look at a sample Proxy Endpoint configuration to understand this. Sample Proxy Endpoint configuration showing that the API Proxy accepts requests on a secure VirtualHost

  2. Let's say the Virtual Hosts are defined in the specific environment as follows:
    Name Port Host Alias
    default 80 myorg-prod.apigee.net
    secure 443 myorg-prod.apigee.net
  3. You make an API request to the default VirtualHost using the URL http://myorg-prod.apigee.net/weather
  4. Since the Proxy Endpoint does not have "default" VirtualHost as shown in the example above, you get the 404 response code with the below error message:
    {"fault":{"faultstring":"Unable to identify proxy for host: default and url: \/weather","detail":{"errorcode":"messaging.adaptors.http.flow.ApplicationNotFound"}}}
  5. Move to the Resolution section below to know how to address this issue.
  6. If the Proxy Endpoint is configured to accept the requests on the "default" VirtualHost, move to the next cause - Path not associated with any API Proxy.

Resolution

  1. Add the missing VirtualHost to the Proxy Endpoint configuration to address the issue.
    1. In the Example shown above, you can add the default VirtualHost to the Proxy Endpoint configuration as follows:
      <VirtualHost>default</VirtualHost>
      

      Sample Proxy Endpoint configuration showing the "default" VirtualHost being added

  2. Alternatively, in the example referred to above, if you intended to use only the secure VirtualHost for this specific API Proxy, then make the API requests only to the secure VirtualHost using the https protocol:
    https://myorg-prod.apigee.net/weather

Cause: Path not associated with any API Proxy

If the API Proxy is not configured to accept the requests for the specific path used in the API Request URL, then we can get 404 Not Found response with the error message "Unable to identify proxy for host: and url: ".

Diagnosis

  1. Look at the Proxy Endpoint configuration for the specific API Proxy for which you intended to make the API requests.
  2. Check if the API Proxy is configured to accept the requests for the specific path indicated in the error message. You can do this by following the below steps:

Scenario #1: Path does not match the basepath of the API Proxy

  1. If the path indicated in the error message is not same as the basepath of the specific API Proxy and/or it does not start with the basepath, then that could be the cause for the error.
  2. Let's take an example to explain this:
    1. Basepath of the intended API Proxy is /weather
    2. API Request URL is https://myorg-prod.apigee.net/climate. This means that path used in the API request URL is /climate.
  3. In this example, the path is neither same as the basepath or does not start with the basepath. Hence you get the error:
    {
       "fault":{
          "faultstring":"Unable to identify proxy for host: secure and url: \/climate",
          "detail":{
             "errorcode":"messaging.adaptors.http.flow.ApplicationNotFound"
          }
       }
    }

Resolution

  1. Ensure the path used in your API request URL is same as the basepath of the specific API Proxy.
  2. In the example above, the API Request URL should be as follows: https://myorg-prod.apigee.net/weather

Scenario #2: Path does not match any of the available Conditional Flows

  1. If the path used in the API Request URL starts with the basepath, then it's possible that the path suffix (i.e., the part that comes after the basepath) indicated in the error message does not match any of the conditional flows, then that could cause the 404 error.
  2. Let's take an example to explain this:
    1. Basepath of the intended API Proxy is /weather
    2. API Request URL is https://myorg-prod.apigee.net/weather/Delhi. This means that path used in the API request URL is /weather/Delhi.
  3. In this example, the path starts with the basepath /weather. In addition, it has path suffix which /Delhi.
  4. Now check if there are any Conditional flows in the Proxy Endpoint.
  5. If there are no Conditional flows or there are a few non Conditional flows, then move to next cause - API Proxy not deployed in an environment.
  6. If the Proxy Endpoint has got all the flows to be only Conditional flows, then check for the following:
    1. If the conditions in all these Conditional flows check for a specific proxy.pathsuffix (i.e., the path after the basepath).
    2. And if the path suffix specified in the API Request URL does not match any of the conditions, then that's the cause for the error.
  7. Let's say we have two flows in the Proxy Endpoint and both of them are Conditional flows as shown below:
    <Condition>(proxy.pathsuffix MatchesPath "/Bangalore") and (request.verb = "GET")</Condition>
    
    <Condition>(proxy.pathsuffix MatchesPath "/Chennai") and (request.verb = "GET")</Condition>
    
    1. In the example shown above, we have two Conditional flows, one that matches the proxy.pathsuffix (path after the basepath) to /Bangalore and the other one matches /Chennai. But there's none that matches to /Delhi which is the path suffix passed in the API Request URL
    2. This is the cause for the 404 error. Hence you would get the error
      {
         "fault":{
            "faultstring":"Unable to identify proxy for host: secure and url: \/weather\/Delhi",
            "detail":{
               "errorcode":"messaging.adaptors.http.flow.ApplicationNotFound"
            }
         }
      }

Resolution

  1. Ensure the path suffix matches at least one of the conditional flows in your proxy endpoint.
  2. In the example above, you can use of the following approaches to resolve the error:
    1. If you want to execute any specific set of Policies for the path /Delhi, then add a separate flow with the required set of Policies and ensure there's a condition that matches the proxy.pathsuffix /Delhi as shown below:
      <Condition>(proxy.pathsuffix MatchesPath "/Delhi") and (request.verb = "GET")</Condition>
      
    2. If you want to execute common set of Policies for the path /Delhi, then in the common Flow, ensure that there's a condition that allows generic proxy.pathsuffix. That is it would allow any path after the basepath /weather as shown below
      <Condition>(proxy.pathsuffix MatchesPath "/**") and (request.verb = "GET")</Condition>

If the Proxy Endpoint has the correct basepath and the path suffix specified in the API URL does match one of the conditional flows, then move to next cause - API Proxy not deployed in an environment.

Cause: API Proxy not deployed in an environment

Diagnosis

  1. Determine the environment to which the host alias used in your API request URL exists. This can be done by checking the details of all virtual hosts in each of the environments of your organization in the Edge UI.
    1. For example, assume the following configuration:
      1. If http://myorg-prod.apigee.net/weather is your URL, then myorg-prod.apigee.net is the host alias.
      2. The host alias myorg-prod.apigee.net is configured as part of one of the virtual host(s) of in "prod" environment of your organization.
  2. Check if the specific API Proxy is deployed in the specific environment determined in the step #1 above.
  3. If the API Proxy is not deployed in the specific environment, then that's the cause for the 404 error.
    1. So in the example used in step #1 above, let's say the API Proxy is not deployed in the"prod" environment, then that's the cause for the error.
    2. Move to the Resolution section below.
  4. If the API Proxy is deployed in the specific environment, then move to next cause - Environment not loaded on Message Processor(s).

Resolution

  1. Deploy the API Proxy in the specific environment in which you intend to make API requests.

Cause: Environment not loaded on the Message Processor(s)

(For Edge Private Cloud users only)

Diagnosis

  1. Log in to each of the Message Processor(s) and check if the specific environment in which you are making the API request is loaded on the Message Processor using the following command:
    curl -v 0:8082/v1/runtime/organizations/<orgname>/environments
    
  2. If the specific environment is listed as part of the above command, then move to next cause - API Proxy not deployed on one or more Message Processors.
  3. If the specific environment is not listed, then check the /opt/apigee/var/log/edge-message-processor/logs/system.log and /opt/apigee/var/log/edge-message-processor/logs/startupruntimeerrors.log on Message Processors for any errors during loading of Environments.
  4. There could be many different errors that could lead to failure of loading an environment on the Message Processor. Resolution depends on the error that occurred.

Resolution

The Environment may not be loaded on Message Processor due to many reasons. This section illustrates a couple of possible reasons that can lead to this issue and explains how to resolve the issue.

  1. If you see one of the following errors in the Message Processor log, then it is caused due to an issue found with the certificates/keys that have been added to the specified keystore/truststore in the specified environment.

    Error #1: java.security.KeyStoreException: Cannot overwrite own certificate

    2018-01-30 12:04:38,248 pool-47-thread-4 ERROR MESSAGING.RUNTIME - AbstractConfigurator.propagateEvent() : Error while handling the update for the Configurator
    com.apigee.kernel.exceptions.spi.UncheckedException: Failed to add certificate : mycert in key store : mytruststore in environment : test
    at com.apigee.entities.configurators.KeyStore.setCertificateEntry(KeyStore.java:156) ~[config-entities-1.0.0.jar:na]
    at com.apigee.entities.configurators.KeyStore.handleUpdate(KeyStore.java:101) ~[config-entities-1.0.0.jar:na]
    at com.apigee.entities.AbstractConfigurator.propagateEvent(AbstractConfigurator.java:85) ~[config-entities-1.0.0.jar:na]
    at com.apigee.messaging.runtime.Environment.handleUpdate(Environment.java:238) [message-processor-1.0.0.jar:na]
    …
    Caused by: java.security.KeyStoreException: Cannot overwrite own certificate
    at com.sun.crypto.provider.JceKeyStore.engineSetCertificateEntry(JceKeyStore.java:355) ~[sunjce_provider.jar:1.8.0_151]
    at java.security.KeyStore.setCertificateEntry(KeyStore.java:1201) ~[na:1.8.0_151]
    at com.apigee.entities.configurators.KeyStore.setCertificateEntry(KeyStore.java:153) ~[config-entities-1.0.0.jar:na]
    

    ... 20 common frames omitted

    2018-01-30 12:04:38,250 pool-47-thread-4 ERROR MESSAGING.RUNTIME - AbstractConfigurator.rollbackTransaction() : Error in processing the changes : Unknown resource type cert
    

    Error #2: java.security.KeyStoreException: Cannot overwrite secret key

    2017-11-01 03:28:47,560 pool-21-thread-7 ERROR MESSAGING.RUNTIME - AbstractConfigurator.propagateEvent() : Error while handling the update for the Configurator
    com.apigee.kernel.exceptions.spi.UncheckedException: Failed to add certificate : mstore in key store : myTruststore in environment : dev
    at com.apigee.entities.configurators.KeyStore.setCertificateEntry(KeyStore.java:156) ~[config-entities-1.0.0.jar:na]
    at com.apigee.entities.configurators.KeyStore.handleUpdate(KeyStore.java:101) ~[config-entities-1.0.0.jar:na]
    ...
    Caused by: java.security.KeyStoreException: Cannot overwrite secret key
    at com.sun.crypto.provider.JceKeyStore.engineSetCertificateEntry(JceKeyStore.java:354) ~[sunjce_provider.jar:1.8.0_144]
    at java.security.KeyStore.setCertificateEntry(KeyStore.java:1201) ~[na:1.8.0_144]
    at com.apigee.entities.configurators.KeyStore.setCertificateEntry(KeyStore.java:153) ~[config-entities-1.0.0.jar:na]
    ... 20 common frames omitted
    
    2017-11-01 03:28:47,562 pool-21-thread-7 ERROR MESSAGING.RUNTIME - AbstractConfigurator.rollbackTransaction() : Error in processing the changes : Unknown resource type cert
  2. Get the details of the keystore/truststore specified in the error message shown in the previous step by using the following management API call:

    curl -v "http://<management-IPaddress>:8080/v1/organizations/<org-name>/environments/<env-name>/keystores/myTruststore" -u <user> 


    Example output:
    {
       "certs":[
          "mycert",
          "mycert-new"
       ],
       "keys":[
          "mycert"
       ],
       "name":"myTruststore"
    }
    
  3. The example output shows that there are two certificates and a key in the truststore myTruststore. The truststore generally does not contain a key. If it does, it is better to have a single certificate and a single key.
  4. Get the details about the two certificates using the following API:
    curl -s http://<management-IPaddress>:8080/v1/runtime/organizations/<org-name>/environments/<env-name>/keystores/<keystore-name>/certs/<cert-name>
    
  5. Check the expiry date of each of the certificates and determine the expired/older certificate.
  6. Delete the expired or unwanted certificate from the truststore "myTruststore".

If the problem still persists or if you see any error other than the ones mentioned in Step #1 above, go to Must Gather Diagnostic Information.

Cause: API Proxy not deployed on one or more Message Processors

(For Edge Private Cloud users only)

The API Proxy may not be deployed on one or more Message Processors. This issue occurs very rarely and happens mostly due to a missing event notification from the Management Server to Message Processor during the deployment of the specific API Proxy. In this case also, you will not be able to create the trace session in the Edge UI.

Diagnosis

  1. Login to each of the Message Processors and check if the specific revision of the API Proxy is deployed or not using the following command:
    curl -v 0:8082/v1/runtime/organizations/<orgname>/environments/<envname>/apis/<apiname>/revisions
    
  2. If the specific revision of the API Proxy does not show up as the output of the command mentioned in Step #1 above, then restart the specific Message Processor as explained in the Resolution below.
  3. Repeat the steps 1-2 for all the Message Processors.
  4. If the specific revision of the API Proxy is deployed on all the Message Processors, then this is not the cause for this issue. Move to Must Gather Diagnostic Information.

Resolution

  1. Restart the specific Message Processor(s) on which the specific revision of the API Proxy is not deployed.
    /opt/apigee/apigee-service/bin/apigee-service edge-message-processor restart

Diagnose issues using API Monitoring

Note: The steps in this section can be performed by Public Cloud users only.

API Monitoring enables you to isolate problem areas quickly to diagnose error, performance, and latency issues and their source, such as developer apps, API proxies, backend targets, or the API platform.

Step through a sample scenario that demonstrates how to troubleshoot issues with your APIs using API Monitoring. For example, you may want to set up an alert to be notified when the number of 404 status codes exceeds a particular threshold.

Must Gather Diagnostic Information

If the problem persists even after following the above instructions, please gather the following diagnostic information. Contact and share this information with Apigee Support.

  1. If you are a Public Cloud user, provide the following information:
    1. Organization Name
    2. Environment Name
    3. API Proxy Name
    4. Complete curl command to reproduce the error
  2. If you are a Private Cloud user, provide the following information:
    1. Complete Error Message observed
    2. Environment name
    3. API Proxy bundle
    4. Message Processor logs /opt/apigee/var/log/edge-message-processor/logs/system.log
    5. Output of the following commands on each of the Message Processors.
    6.       curl -v 0:8082/v1/runtime/organizations/<orgname>/environments
            curl -v 0:8082/v1/runtime/organizations/<orgname>/environments/<envname>/apis/<apiname>/revisions
            
  3. Details about what sections in this Playbook you have tried and any other insights that will help us to fastrack resolution of this issue.