404 Multiple virtual hosts with the same host alias

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

Symptom

The client application gets an HTTP status code of 404 with the message Not Found and the error message Unable to identify proxy for host: VIRTUAL_HOST and url: PATH as a response to the API calls.

This error means that Edge could not find the API proxy for the specified virtual host and path.

Error message

The client application gets the following response code:

HTTP/1.1 404 Not Found

In addition, you may observe an error message similar to the one shown below:

{
   "fault":{
      "faultstring":"Unable to identify proxy for host: default and url: \/oauth2\/token",
      "detail":{
         "errorcode":"messaging.adaptors.http.flow.ApplicationNotFound"
      }
   }
}

Possible causes

Cause	Description	Troubleshooting instructions applicable for
Virtual host with duplicate host alias	Multiple virtual hosts have the same host alias and port number.	Edge Public and Private Cloud users

Common diagnosis steps

NGINX and Message Processor logs will be helpful in troubleshooting the 404 error. Use the following steps to check the logs:

View the NGINX logs using the following command:

/opt/apigee/var/log/edge-router/nginx/ORG~ENV.PORT#_access_log

Check for the following fields in the log entries:

Field	Value
`Upstream_status, status`	`404`
`X-Apigee-fault-code`	`messaging.adaptors.http.flow.ApplicationNotFound`

Make a note of the message ID from the logs.

Check the Message Processor logs (/opt/apigee/var/log/edge-message-processor/logs/system.log) to see if you have messaging.adaptors.http.flow.ApplicationNotFound for the specific API or if you have the unique message ID from step 2 for the API request.
Sample error message from Message Processor log

NIOThread@1 ERROR ADAPTORS.HTTP.FLOW - AbstractRequestListener.onException() : Request:POST,
uri:/weather, message Id:null, exception:com.apigee.rest.framework.ResourceNotFoundException{
code = messaging.adaptors.http.flow.ApplicationNotFound, message = Unable to identify proxy for
host: vh1 and url: /weather, associated contexts = []}, context:Context@342ea86b
input=ClientInputChannel(SSLClientChannel[Accepted: Remote:10.123.123.123:8443
Local:10.135.33.68:62092]@1206954 useCount=1 bytesRead=0 bytesWritten=0 age=1ms
lastIO=0ms  isOpen=true)

The log above shows the error code and error message is as follows:

code = messaging.adaptors.http.flow.ApplicationNotFound,
message = Unable to identify proxy for host: vh1 and url: /weather

Note: The url and host above may vary depending on the API proxy for which you are having the issue.

Cause: Multiple virtual hosts with the same host alias and port number

Apigee Edge Routers and Message Processors use both the host header, port number and URI paths to route traffic to the correct API proxy. Having ambiguous definitions such as multiple virtual hosts with the same host alias and port number is a documented anti-pattern and can lead to unexpected behaviours. One of the common errors that you will observe is a 404 error with the message Unable to identify proxy for host: VIRTUAL_HOST and url: PATH.

Typically if multiple virtual hosts with the same host alias exist, then you will observe intermittent 404 errors. This is because the specific API Proxy may be configured to accept the requests on only one of the virtual hosts. When the API requests are routed to the specific virtual host that is configured in the API Proxy, you will get a successful response. However, when the API requests are routed to the other virtual hosts to which the API Proxy is not configured to accept the requests, then the APIs will fail with these 404 errors.

Follow the instructions given in 404 Unable to identify proxy for host: <virtual host name> and url: <path> and troubleshoot this error. If none of the causes are leading to this error, then use the steps below to determine if virtual hosts with duplicate host aliases are causing the 404 errors.

Diagnosis

Use one of the following methods to determine if there are multiple virtual hosts having the same host alias/port # leading to 404 errors:

Edge UI
Management APIs

Edge UI

Use these instructions to determine if there are multiple virtual hosts having the same host alias/port # using the Edge UI.

For example, if you observed the 404 error with the url http://example.com:9001/proxy1, then you need to find which virtual hosts have the host alias example.com and port 9001.

In Public Cloud and new Edge UI on Private Cloud:
1. Select Admin.
2. Select Virtual Hosts.
3. For each Environment use the search filter to determine the Virtual Hosts that match the specific Host Alias with which the API requests were invoked.
4. If you find multiple Virtual Hosts using the same Host Alias go to Resolution to resolve this issue.
For example:
In Classic UI on Private Cloud:
1. Select APIs tab.
2. Select Environment Configuration.
3. Select Virtual Hosts.
4. For each Environment view the list of Virtual Hosts to see if any match the specific Host Alias with which the API requests were invoked.
5. If you find multiple Virtual Hosts matching the same Host Alias go to Resolution to resolve this issue.
For Example:

Management APIs

Use these instructions to determine if there are multiple virtual hosts having the same host alias/port # using the Management APIs.

Get the definition of each of the virtual hosts in each of the environments in your organizations to see which virtual hosts have the same host alias and port number:

For example, if you observed the 404 error with the url http://example.com:9001/proxy1, then you need to find which virtual hosts have the host alias example.com and port 9001.

Get the list of environments

Public Cloud user:

curl -v -X GET https//api.enterprise.apigee.com/v1/organizations/ORGANIZATION_NAME/environments -u USERNAME

Private Cloud user:

curl -v -X GET http://MANAGEMENT_HOST:PORT#/v1/organizations/ORGANIZATION_NAME/environments -u USERNAME

Where:

ORGANIZATION_NAME is the name of the organization

Example:

curl http://127.0.0.1:8080/v1/organizations/myorg/environments -u USERNAME

[ "prod", "test", "dev" ]

Get the list of virtual hosts in an environment

Public Cloud user:

curl -v -X GET https//api.enterprise.apigee.com/v1/organizations/ORGANIZATION_NAME/environments/ENVIRONMENT_NAME/virtualhosts -u USERNAME

Private Cloud user:

curl -v -X GET http://MANAGEMENT_HOST:PORT#/v1/organizations/ORGANIZATION_NAME/environments/ENVIRONMENT_NAME/virtualhosts -u USERNAME

Where:

ORGANIZATION_NAME is the name of the organization

ENVIRONMENT_NAME is the name of the environment

Example:

curl http://127.0.0.1:8080/v1/organizations/myorg/environments/test/virtualhosts -u USERNAME

[ "default" ]

Get the definition of each of the virtual hosts in the environment.

Public Cloud user:

curl -v -X GET https//api.enterprise.apigee.com/v1/organizations/ORGANIZATION_NAME/environments/ENVIRONMENT_NAME/virtualhosts/VIRTUAL_HOST_NAME  -u USERNAME

Private Cloud user:

curl -v -X GET http://MANAGEMENT_HOST:PORT#/v1/organizations/ORGANIZATION_NAME/environments/ENVIRONMENT_NAME/virtualhosts/VIRTUAL_HOST_NAME -u USERNAME

Where:

ORGANIZATION_NAME is the name of the organization

ENVIRONMENT_NAME is the name of the environment

VIRTUAL_HOST_NAME is the name of the virtual host

Example:

curl http://127.0.0.1:8080/v1/organizations/myorg/environments/test/virtualhosts/default -u USERNAME

{
  "hostAliases" : [ "example.com" ],
  "interfaces" : [ ],
  "listenOptions" : [ ],
  "name" : "default",
  "port" : "9001",
  "retryOptions" : [ ]
}

Repeat the above two steps for the other environments in your organization.

In this example, repeat the steps for the dev environment:

curl http://127.0.0.1:8080/v1/organizations/myorg/environments/dev/virtualhosts -u USERNAME

[ "default" ]

curl http://127.0.0.1:8080/v1/organizations/myorg/environments/dev/virtualhosts/default -u USERNAME

{
  "hostAliases" : [ "example.com" ],
  "interfaces" : [ ],
  "listenOptions" : [ ],
  "name" : "default",
  "port" : "9001",
  "retryOptions" : [ ]
}

In this example you can see that the two virtual hosts default in two different environments, test and dev, both contain the same host alias example.com and port number 9001. This is the cause for 404 errors.

If you find multiple Virtual Hosts matching the same Host Alias go to Resolution to resolve this issue.

Resolution

Ensure that each virtual host only contains unique host alias and port combinations.
If you have identified multiple virtual hosts with the same host alias and port combinations then you must update these with a unique host alias.
You can update these using the Edge UI or the Management API, instructions can be found under Modifying a virtual host.
Ensure that each host alias has a proper DNS entry.
Note: If the duplicate host alias was found in a virtual host configured on secure port such as 443, then you need to procure a certificate that matches the new host alias name or has the new host alias in SAN (Subject Alternative Name) in an existing certificate used in the keystore.

In the example discussed above if our configuration looked like this:

curl -X GET http://localhost:8080/v1/organizations/myorg/environments -u user

[ "prod", "test", "dev" ]

curl -X GET http://localhost:8080/v1/organizations/myorg/environments/test/virtualhosts/default -u user

{
  "hostAliases" : [ "example.com" ],
  "interfaces" : [  ],
  "listenOptions" : [  ],
  "name" : "default",
  "port" : "9001",
  "retryOptions" : [  ]
}

curl -X GET http://localhost:8080/v1/organizations/myorg/environments/dev/virtualhosts/default -u user

{
  "hostAliases" : [ "example.com" ],
  "interfaces" : [  ],
  "listenOptions" : [  ],
  "name" : "default",
  "port" : "9001",
  "retryOptions" : [  ]
}

You can update the incorrect virtual host such that they are not overlapping.
That is update the host alias as example2.com.

Ensure the new host alias has a similar DNS entry as the previous host alias.

curl -X GET http://localhost:8080/v1/organizations/myorg/environments/dev/virtualhosts/default -u user -H 'Content-Type: application/json' -d '{
  "hostAliases" : [ "example2.com" ],
  "interfaces" : [  ],
  "listenOptions" : [  ],
  "name" : "default",
  "port" : "9001",
  "retryOptions" : [  ]
}' -i

HTTP/1.1 200 OK
Date: Tue, 02 Feb 2021 20:54:29 GMT
Content-Type: application/json
X-Apigee.user: user
X-Apigee.organization: myorg
X-Apigee.environment: dev
X-Apigee.backends: management-server
Date: Tue, 02 Feb 2021 20:54:29 GMT
Content-Length: 152

{
  "hostAliases" : [ "example2.com" ],
  "interfaces" : [  ],
  "listenOptions" : [  ],
  "name" : "default",
  "port" : "9001",
  "retryOptions" : [  ]
}

Make the API calls again to the proxy and verify that you are getting successful responses consistently:

curl http://example.com:9001/proxy1

{
    "slideshow": {
	    "author": "Yours Truly",
		"date": "date of publication",
		"slides": [
		    {
			    "title": "Wake up to WonderWidgets!",
				"type:": "all"
			},
			{
			    "items": [
				    "Why WonderWidgets are great",
					"Who buys WonderWidgets"
				],
				"title": "Overview",
				"type": "all"
			}
		],
		"title": "Sample Slide Show"
    }

}

If the problem still persists, go to Must gather diagnostic information.

Must gather diagnostic information

If the problem persists even after following the above instructions, gather the following diagnostic information, and then contact Apigee Edge Support:

If you are a Public Cloud user, provide the following information:

Organization name
Environment name
API Proxy name
Complete curl command to reproduce the 404 error
If the 404 errors are not occurring currently, provide the time period with the timezone information when 404 errors occurred in the past.

If you are a Private Cloud user, provide the following information:

Complete error message observed for the failing requests
Organization, Environment name and API Proxy name for which you are observing 404 errors
API Proxy bundle
NGINX access logs
/opt/apigee/var/log/edge-router/nginx/ORGANIZATION_NAME~ENVIRONMENT_NAME.PORT#_access_log
Message Processor logs
/opt/apigee/var/log/edge-message-processor/logs/system.log
The time period with the timezone information when the 404 errors occurred