All Edge for Private Cloud customers are required to submit to Apigee statistics about API proxy traffic. Apigee recommends that customers upload that information once a day, possibly by creating a cron job.
You must submit the data for your production API deployments, but not for APIs in development or testing deployments. In most Edge installations, you will define specific organizations or environments for your production APIs. The data that you submit is only for those production organizations and environments.
To assist in uploading this data, Apigee provides the Beta release of the
apigee-analytics-collector command-line
utility. This utility sends the API call volume report back to Apigee. Every Edge for the
Private Cloud installation can use this utility to retrieve and report traffic data to
Apigee.
Required: Contact Apigee Support before uploading data
Before you can upload data to Apigee, you must contact Apigee Edge Support to complete the onboarding process.
Install apigee-analytics-collector
The apigee-analytics-collector
utility is an RPM that you install by using the apigee-service utility.
Where to install
The node on which you install the apigee-analytics-collector utility can be any
node that can access the Edge management API on the Edge Management Server. You can install it
directly on the Management Server, on another node of the Edge installation, or on a separate
node as long as that node can make API requests to the Management Server.
Internet access requirements
Install the apigee-analytics-collector utility on a
machine with external internet access. Then the apigee-analytics-collector utility
can upload data directly to Apigee.
If there is no node with both access to the Edge management API on the Edge Management Server
and access to the external internet, then you can use the Edge management API to save the traffic
data locally. You must then transfer the data to a machine with internet access for upload to
Apigee. In this scenario, you do not need to use the apigee-analytics-collector
utility. See
Manually uploading data to
Apigee for more.
Installation
Use the following command to install the apigee-analytics-collector utility. Because
you are installing an RPM file, this command must be run by the root user or by a user that has
full sudo access. For full sudo access, that means the user has sudo access to perform the same
operations as root.
/opt/apigee/apigee-service/bin/apigee-service apigee-analytics-collector install
Running apigee-analytics-collector
This section describes how to run the apigee-analytics-collector utility.
Configuring a user to run apigee-analytics-collector
You must run apigee-analytics-collector as a non-root
user. This user must have full sudo access to the "apigee" user.
To configure a user to have full sudo access to the "apigee" user, use the "visudo" command to edit the sudoers file to add:
analyticsUser ALL=(apigee) NOPASSWD: ALL
Where analyticsUser is the username of the person running
the apigee-analytics-collector utility.
After installing the apigee-analytics-collector utility
and configuring the user, you can test the utility by running the help command for
the apigee-analytics-collector utility:
/opt/apigee/apigee-service/bin/apigee-service apigee-analytics-collector export traffic --help
Required information to run apigee-analytics-collector
You need the following information to run the apigee-analytics-collector command
and to forward data to Apigee:
apigee_mgmt_api_uri: The base URL of the Edge API on your Management Server. This URL is typically in the following form:http://ms_IP:8080/v1
Where ms_IP is the IP address or your Management Server and 8080 is the port used by the Edge API. If you created a DNS entry for the Edge API, then the URL is in the form:
http://ms_DNS/v1
If you enable TLS on the Edge management API, then it is in the form:
https://ms_IP:8080/v1 https://ms_DNS/v1
apigee_mgmt_api_email: The email address of an account with access to the Edge/statsAPIs. Often this will be the Edge system administrator's email, or the email of the organization administrator for your production organizations.apigee_mgmt_api_password: The Edge password for the account specified byapigee_mgmt_api_email.apigee_analytics_client_idand apigee_analytics_secret: Your credentials for uploading data to Apigee. Please submit a ticket with Apigee Edge Support to obtainapigee_analytics_client_idandapigee_analytics_secret.
Example commands
Shown below is an example command to retrieve traffic data for all organizations and
environments in the Edge installation and upload that data to Apigee. Notice how you
use apigee-service to
run the apigee-analytics-collector command:
/opt/apigee/apigee-service/bin/apigee-service apigee-analytics-collector export traffic \ --apigee_mgmt_api_uri http://192.168.56.103:8080/v1 \ --apigee_mgmt_api_email $ae_username \ --apigee_mgmt_api_password $ae_password \ --apigee_analytics_client_id $apigee_analytics_client_id \ --apigee_analytics_secret $apigee_analytics_secret
Notice that the command contains all required information, such as your apigee_analytics_client_id and apigee_analytics_secret.
You should see results in the form:
[
{
"org": "myOrg",
"env": "prod",
"time_range_start": "08/27/2016 00:00",
"time_range_end": "08/30/2016 00:00",
"response": [
{
"store_org_env_metrics_hourly_v4": 1
}
]
},
{
"org": "VALIDATE",
"env": "test",
"time_range_start": "08/27/2016 00:00",
"time_range_end": "08/30/2016 00:00",
"response": [
{
"store_org_env_metrics_hourly_v4": 1
}
]
}
]
Use command-line options to the command to control its actions. Use the following options to specify the organizations and environments to include in the generated data:
-i, --include_orgs comma-separated list of items-x, --exclude_orgs comma-separated list of items-n, --include_envs comma-separated list of items-e, --exclude_envs comma-separated list of items
For example, to specify only the production organizations and environments, use the
-i (or --include_orgs) and -n (or
--include_envs) options:
/opt/apigee/apigee-service/bin/apigee-service apigee-analytics-collector export traffic -i myOrg -n prod \ --apigee_mgmt_api_uri http://192.168.56.103:8080/v1 \ --apigee_mgmt_api_email $ae_username \ --apigee_mgmt_api_password $ae_password \ --apigee_analytics_client_id $apigee_analytics_client_id \ --apigee_analytics_secret $apigee_analytics_secret
In this example, you only collect data from the prod environment of
myOrg.
To dump the data to a screen to examine it before sending it to Apigee, use the -S
option:
/opt/apigee/apigee-service/bin/apigee-service apigee-analytics-collector export traffic -i myOrg -n prod -S \ --apigee_mgmt_api_uri http://192.168.56.103:8080/v1 \ --apigee_mgmt_api_email $ae_username \ --apigee_mgmt_api_password $ae_password \ --apigee_analytics_client_id $apigee_analytics_client_id \ --apigee_analytics_secret $apigee_analytics_secret
The -S option omits the data
upload to Apigee. You can then rerun the command without the -S option to send the
data to Apigee.
One reason to use the -S
option is so that you can display different types of data locally. Apigee only requires that you
upload API traffic data, but the -D option lets you display data about API
products, developers, apps, or API proxies. The example below uses the -D and
-S options to display developer data
locally:
/opt/apigee/apigee-service/bin/apigee-service apigee-analytics-collector export traffic -i myOrg -n prod -S -D devs \ --apigee_mgmt_api_uri http://192.168.56.103:8080/v1 \ --apigee_mgmt_api_email $ae_username \ --apigee_mgmt_api_password $ae_password \ --apigee_analytics_client_id $apigee_analytics_client_id \ --apigee_analytics_secret $apigee_analytics_secret
Include the -v option to get
verbose output, and the -R option
to see the curl commands generated by apigee-analytics-collector:
/opt/apigee/apigee-service/bin/apigee-service apigee-analytics-collector export traffic -i myOrg -n prod -S -R -v \ --apigee_mgmt_api_uri http://192.168.56.103:8080/v1 \ --apigee_mgmt_api_email $ae_username \ --apigee_mgmt_api_password $ae_password \ --apigee_analytics_client_id $apigee_analytics_client_id \ --apigee_analytics_secret $apigee_analytics_secret
The next section contains a complete list of command-line options.
Command parameters
The following table lists the complete set of options to the
apigee-analytics-collector utility:
| Command | Description |
|---|---|
-h, --help |
Output usage information |
-D, --dimension dimension |
The traffic dimension to collect. Valid values are:
|
-d, --days days
|
The number of past days of data to collect, starting from the current date. The default is 3. If you specify |
-m, --apigee_mgmt_api_uri apigee_mgmt_api_uri |
URL to Edge management API. |
-u, --apigee_mgmt_api_email apigee_mgmt_api_email |
The email address of an account with access to the Edge |
-p, --apigee_mgmt_api_password apigee_mgmt_api_password |
The password associated with the Edge management API email account specified by
|
-i, --include_orgs items
|
A comma-separated list of organizations to include in the output. |
-x, --exclude_orgs items
|
A comma-separated list of orgs to exclude from the output. |
-n, --include_envs items
|
A comma-separated list of environments to include in the output. |
-e, --exclude_envs items
|
A comma-separated list of environments to include from the output. |
-o, --output path
|
The path and filename to save output. |
-s, --time_range_start time_range_start
|
Time range to start for querying traffic stats, in the form: "MM/DD/YYYY HH:MM". If you specify |
-z, --time_range_end time_range_end
|
Time range end for querying traffic stats, in the form: "04/01/2016 24:00". If you specify |
-t, --time_unit time_unit
|
Time unit for traffic data. The default value is
If you set |
-S, --standard_output
|
Write output to the terminal (stdout), instead of uploading it to Apigee. |
-c, --apigee_analytics_client_id apigee_analytics_client_id
|
Your ID for uploading data to Apigee. Please submit a ticket with Apigee Edge Support to obtain. |
-r, --apigee_analytics_secret apigee_analytics_secret
|
Your secret for upload data to Apigee. Please submit a ticket with Apigee Edge Support to obtain. |
-R, --include_curl_commands
|
Include the generated |
-v, --verbose
|
Display verbose output. |
Manually uploading data to Apigee
Apigee recommends that you install the apigee-analytics-collector utility on a
machine with external internet access. Then the apigee-analytics-collector utility
can upload data directly to Apigee.
However, if the machine does not have external internet access, use the Edge management API to
collect traffic data, and then use curl commands to upload it to Apigee from a
machine with
internet access. You must repeat this process for every production organization and
environment in your Edge installation.
Use the following curl command to gather traffic data for a specific organization and
environment for a specified time interval:
curl -X GET -u apigee_mgmt_api_email:apigee_mgmt_api_password \ "http://ms_IP:8080/v1/organizations/org_name/environments/env_name/stats/apiproxy?select=sum(message_count)&timeRange=MM/DD/YYYY%20HH:MM~MM/DD/YYYY%20HH:MM&timeUnit=hour"
This command uses the Edge Get API message count API. In this command:
- apigee_mgmt_api_email:apigee_mgmt_api_password specifies the email address of an account with access to the Edge /stats APIs.
- ms_IP is the IP address or DNS name of the Edge Management Server.
- org_name and env_name specifies the org and environment.
- apiproxy is the dimenesion that groups metrics by API proxies.
MM/DD/YYYY%20HH:MM~MM/DD/YYYY%20HH:MM&timeUnit=hourspecifies the time range divided into time units of the metrics to gather. Notice that thecurlcommand uses the hex code%20for spaces in the time range.
For example, to gather API proxy message counts hour by hour over a 24 hour period, use the following API call.
curl -X GET -u apigee_mgmt_api_email:apigee_mgmt_api_password \ "http://192.168.56.103:8080/v1/organizations/myOrg/environments/prod/stats/apiproxy?select=sum(message_count)&timeRange=01%2F01%2F2018%2000%3A00~01%2F02%2F2018%2000%3A00&timeUnit=hour"
(Note that timeRange contains URL-encoded characters.)
You should see a response in the form:
{
"environments" : [ {
"dimensions" : [ {
"metrics" : [ {
"name" : "sum(message_count)",
"values": [
{
"timestamp": 1514847600000,
"value": "35.0"
},
{
"timestamp": 1514844000000,
"value": "19.0"
},
{
"timestamp": 1514840400000,
"value": "58.0"
},
{
"timestamp": 1514836800000,
"value": "28.0"
},
{
"timestamp": 1514833200000,
"value": "29.0"
},
{
"timestamp": 1514829600000,
"value": "33.0"
},
{
"timestamp": 1514826000000,
"value": "26.0"
},
{
"timestamp": 1514822400000,
"value": "57.0"
},
{
"timestamp": 1514818800000,
"value": "41.0"
},
{
"timestamp": 1514815200000,
"value": "27.0"
},
{
"timestamp": 1514811600000,
"value": "47.0"
},
{
"timestamp": 1514808000000,
"value": "66.0"
},
{
"timestamp": 1514804400000,
"value": "50.0"
},
{
"timestamp": 1514800800000,
"value": "41.0"
},
{
"timestamp": 1514797200000,
"value": "49.0"
},
{
"timestamp": 1514793600000,
"value": "35.0"
},
{
"timestamp": 1514790000000,
"value": "89.0"
},
{
"timestamp": 1514786400000,
"value": "42.0"
},
{
"timestamp": 1514782800000,
"value": "47.0"
},
{
"timestamp": 1514779200000,
"value": "21.0"
},
{
"timestamp": 1514775600000,
"value": "27.0"
},
{
"timestamp": 1514772000000,
"value": "20.0"
},
{
"timestamp": 1514768400000,
"value": "12.0"
},
{
"timestamp": 1514764800000,
"value": "7.0"
}
]
}
],
"name" : "proxy1"
} ],
"name" : "prod"
} ],
"metaData" : {
"errors" : [ ],
"notices" : [ "query served by:53dab80c-e811-4ba6-a3e7-b96f53433baa", "source pg:6b7bab33-e732-405c-a5dd-4782647ce096", "Table used: myorg.prod.agg_api" ]
}
}
To then upload that data to Apigee from a machine with internet access, use the following
curl command:
curl -X POST -H 'Content-Type:application/json' \ -u apigee_analytics_client_id:apigee_analytics_secret \ https://nucleus-api-prod.apigee.com/v1/apigee-analytics-cli-api/traffic/orgs/org_name/apis -d '"environments"...'
Where:
- apigee_analytics_client_id:apigee_analytics_secret specify your credentials for uploading data to Apigee that you obtained from Apigee.
- org_name specifies the organization name.
- "environments"... contains the results of the
curlcommand that you used to gather stats above.