Managing alerts and notifications using the API

You're viewing Apigee Edge documentation.
Go to the Apigee X documentation. info

The following sections describe how to manage alerts using the API.

See Alerts API for more on the alerts API.

Set up alerts and notifications using the API

Set up alerts and notifications by issuing a POST request to the following resource: https://apimonitoring.enterprise.apigee.com/alerts.

The following sections provide examples of setting up alerts and notifications using the API:

Set up a 5xx status code alert for an API proxy using the API

The following example shows how to set up an alert that is triggered when 5xx status codes occur at a rate that is greater than 100 transactions per second (TPS) for 10 minutes for the hotels API proxy in the prod environment for any region. A notification is sent to the specified email address when the alert is triggered.

curl 'https://apimonitoring.enterprise.apigee.com/alerts' \
  -X POST \
  -H 'Accept: application/json, text/plain, */*' -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{
     "organization":"myorg",
     "name":"5xx Alert",
     "description":"My 5xx alert",
     "environment":"prod",
     "enabled":true,
     "conditions":[
     {
        "description":"",
        "dimensions":{
            "org":"myorg", 
            "env":"prod", 
            "proxy":"hotels",
            "region":"ANY",
            "statusCode":"5xx"
        },
        "metric":"tps",
        "threshold":100,
        "durationSeconds":600,
        "comparator":">"
     }
     ],
     "notifications":[{ 
         "channel":"email", 
         "destination":"ops@acme.com"
     }],
     "playbook":"http://acme.com/myplaybook.html",
     "throttleIntervalSeconds":3600,
     "reportEnabled":true
    }'

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

Set up a 95th percentile latency alert for an API proxy using the API

The following example shows how to set up an alert that is triggered if total response latency for the 95th percentile is greater than 100 ms for 5 minutes for the hotel API proxy in the prod environment for any region.

A notification is sent to the specified Webhook in the event the alert is triggered.

curl 'https://apimonitoring.enterprise.apigee.com/alerts' \
  -X POST \
  -H 'Accept: application/json, text/plain, */*' -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{
     "organization":"myorg",
     "name":"My Alert",
     "description":"My first alert",
     "environment":"prod",
     "enabled":true,
     "conditions":[
     {
        "description":"",
        "dimensions":{
            "org":"myorg", 
            "env":"prod",
            "proxy" : "hotels",
            "region":"ANY",
            "percentile":"95"
        },
        "metric":"totalLatency",
        "threshold":100,
        "durationSeconds":300,
        "comparator":">"
     }
     ],
     "notifications":[{ "channel":"webhook", "destination":"https://apigee.com/test-webhook"}],
     "playbook":"http://acme.com/myplaybook.html",
     "throttleIntervalSeconds":3600,
     "reportEnabled":true
}'

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

Set up a 404 (Application Not Found) status code alert for all API proxies using the API

The following example shows how to set up an alert that is triggered when HTTP 404 status codes occur at a rate that is greater than 10% for 5 minutes for all API proxies in the prod environment for any region.

A notification is sent to the specified Slack channel when the alert is triggered.

curl 'https://apimonitoring.enterprise.apigee.com/alerts' \
  -X POST \
  -H 'Accept: application/json, text/plain, */*' -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{
     "organization":"myorg",
     "name":"404 Application Not Found Alert",
     "description":"My 404 alert",
     "environment":"prod",
     "enabled":true,
     "conditions":[
     {
        "description":"",
        "dimensions":{"org":"myorg", 
            "env":"prod", 
            "proxy":"ALL",
            "region":"ANY",
            "statusCode":"404"},
        "metric":"rate",
        "threshold":0.05,
        "durationSeconds":300,
        "comparator":">"
     }],
     "notifications":[{ "channel":"slack", "destination":"https://hooks.slack.com/services/T00000000/B00000000/XXXXX"}],
     "playbook":"http://acme.com/myplaybook.html",
     "throttleIntervalSeconds":3600,
     "reportEnabled":true
}'

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

Set up an API proxy count alert for APIs using the API

The following provides an example of how to set up an alert that is triggered when the 5xx code count for APIs exceeds 200 for 5 minutes for any region.

In this example, the APIs are captured in the Critical API Proxies collection (with UUID aeff4394-86b7-11e8-83d7-42010a840040). To obtain the UUID for a collection, see View all collections using the API.

curl 'https://apimonitoring.enterprise.apigee.com/alerts' \
  -X POST \
  -H 'Accept: application/json, text/plain, */*' -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{
     "organization":"myorg",
     "name":"Proxy Count Alert",
     "description":"My proxy count alert",
     "environment":"prod",
     "enabled":true,
     "conditions":[
     {
        "description":"",
        "dimensions":{
            "collection":"aeff4394-86b7-11e8-83d7-42010a840040",
            "org":"myorg", 
            "env":"prod",
            "proxy" : "ANY",
            "region":"ANY",
            "statusCode":"5xx"
        },
        "metric":"count",
        "threshold":200,
        "durationSeconds":300,
        "comparator":">"
     }
     ],
     "notifications":[{ 
         "channel":"email", 
         "destination":"ops@acme.com"
     }],
     "playbook":"http://acme.com/myplaybook.html",
     "throttleIntervalSeconds":3600,
     "reportEnabled":true
    }'

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

Set up an error rate alert for target services using the API

The following provides an example of how to set up an alert that is triggered when the 500 code rate for target services exceeds 10% for 1 hour for any region.

In this example, the target services are captured in the Critical targets collection (with UUID aeff4394-86b7-11e8-83d7-42010a840040). To obtain the UUID for a collection, see View all collections using the API.

curl 'https://apimonitoring.enterprise.apigee.com/alerts' \
  -X POST \
  -H 'Accept: application/json, text/plain, */*' -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{
     "organization":"myorg",
     "name":"Error rate Alert",
     "description":"My error rate alert",
     "environment":"prod",
     "enabled":true,
     "conditions":[
     {
        "description":"",
        "dimensions":{
            "collection":"aeff4394-86b7-11e8-83d7-42010a840040",
            "org":"myorg", 
            "env":"prod",
            "proxy" : "ANY",
            "region":"ANY",
            "statusCode":"500"
        },
        "metric":"rate",
        "threshold":0.1,
        "durationSeconds":3600,
        "comparator":">"
     }
     ],
     "notifications":[{ 
         "channel":"email", 
         "destination":"ops@acme.com"
     }],
     "playbook":"http://acme.com/myplaybook.html",
     "throttleIntervalSeconds":3600,
     "reportEnabled":true
    }'

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

Set up an error rate alert for the Service Callout policy using the API

The following provides an example of how to set up an alert that is triggered when the 500 code rate for the service specified by the ServiceCallout policy exceeds 10% for 1 hour for any region.

curl 'https://apimonitoring.enterprise.apigee.com/alerts' \
  -X POST \
  -H 'Accept: application/json, text/plain, */*' -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{
     "organization":"myorg",
     "name":"Error rate Alert",
     "description":"My error rate alert",
     "environment":"prod",
     "enabled":true,
     "conditions":[
     {
        "description":"",
        "dimensions":{
            "target": "sc://docstore-api",
            "org":"myorg", 
            "env":"prod",
            "proxy" : "ANY",
            "region":"ANY",
            "statusCode":"500"
        },
        "metric":"rate",
        "threshold":0.1,
        "durationSeconds":3600,
        "comparator":">"
     }
     ],
     "notifications":[{ 
         "channel":"email", 
         "destination":"ops@acme.com"
     }],
     "playbook":"http://acme.com/myplaybook.html",
     "throttleIntervalSeconds":3600,
     "reportEnabled":true
    }'

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

Set up a policy fault code alert using the API

The following example shows how to set up an alert that is triggered when if one of the following conditions are met:

  • SpikeArrestViolation fault code count is greater than 10 for 5 minutes for APIs in the prod environment for any region.
  • Count for all API protocol fault codes is greater than 3% for 5 minutes for APIs in the prod environment for any region.

In this example, the APIs are captured in the Critical API Proxies collection (with UUID aeff4394-86b7-11e8-83d7-42010a840040). To obtain the UUID for a collection, see View all collections using the API.

A notification is sent to the specified PagerDuty code when the alert is triggered.

curl 'https://apimonitoring.enterprise.apigee.com/alerts' \
  -X POST \
  -H 'Accept: application/json, text/plain, */*' -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  -d '{
     "organization":"myorg",
     "name":"My Fault Code Alert",
     "description":"My fault code alert",
     "environment":"prod",
     "enabled":true,
     "conditions":[
     {
        "description":"",
        "dimensions": { 
            "collection":"aeff4394-86b7-11e8-83d7-42010a840040",
            "org":"myorg",
            "env":"prod",
            "proxy":"ANY",
            "region":"ANY",
            "faultCodeCategory":"Traffic Mgmt Policy",
            "faultCodeSubCategory":"Spike Arrest",
            "faultCodeName":"SpikeArrest Violation"
        },
        "metric":"count,
        "threshold":10,
        "durationSeconds":300,
        "comparator":">"
     },
     {
       "description":"",
       "dimensions": { 
           "collection":"aeff4394-86b7-11e8-83d7-42010a840040",
           "org":"myorg",
           "env":"prod",
           "proxy":"ANY",
           "region":"ANY",
           "faultCodeCategory":"API Protocol",
           "faultCodeSubCategory":"ALL"
       },
       "metric":"rate",
       "threshold":0.03,
       "durationSeconds":300,
       "comparator":">"
     }   
     ],
     "notifications":[{ "channel":"pagerduty", "destination":"abcd1234efgh56789"}],
     "playbook":"http://acme.com/myplaybook.html",
     "throttleIntervalSeconds":3600,
     "reportEnabled":true
}'

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

View alerts and notifications

The following sections provide examples of viewing alert definitions and information about triggered alerts by using the API:

View all alert definitions for an org

View all alert and notification definitions by issuing a GET request to the following API: https://apimonitoring.enterprise.apigee.com/alerts.

You must pass the name of your organization using the org query parameter.

For example:

curl 'https://apimonitoring.enterprise.apigee.com/alerts?org=myorg'
    -X GET
    -H 'Accept: application/json, text/plain, */*'
    -H "Authorization: Bearer $ACCESS_TOKEN"

The UUID of the alert is shown in the uuid field of the response. You need this UUID to make calls to determine information specific to an alert definition. The following provides an example of the response:

[ 
 {
    "uuid": "4fa49a87-3463023ea7c4",
    "name": "PublicAPI latency alert",
    "enabled": true,
    "description": "Public API Latency alerts, 90th %ile > 6secs for 5 minute window trigger this alert",
    "conditions": [
      {
        "uuid": "4fa49a87-3463023ea7c4",
        "description": "",
        "dimensions": {
          "env": "prod",
          "org": "myorg",
          "percentile": "90",
          "proxy": "PublicAPI",
          "region": "ANY"
        },
        "metric": "totalLatency",
        "threshold": 6000,
        "durationSeconds": 300,
        "comparator": ">",
        "updatedBy": "me@foo.com"
      }
    ],
    "playbook": "PublicAPI Latency alert, setup to go off when 90th %ile is > 4 secs for 5 minute window",
    "throttleIntervalSeconds": 3600,
    "self": "/alerts/4fa49a87-3463023ea7c4",
    "feed": "/o/myorg/events/4fa49a87-3463023ea7c4",
    "organization": "myorg",
    "environment": "prod",
    "notifications": [
      {
        "channel": "email",
        "destination": "me@foo.com"
      }
    ],
    "updatedAt": "2018-07-19T18:19:31.654738Z",
    "updatedBy": "me@foo.com"
  },
  {
    "uuid": "ef1a5249-345ed3023ea7c4",
    "name": "Minty API Latency alert",
    "enabled": true,
    "description": "Minty API Latency alerts, 90th %ile > 6secs for 5 minute window trigger this alert",
    "conditions": [
      {
        "uuid": "ef1a5249-345ed3023ea7c4",
        "description": "",
        "dimensions": {
          "env": "prod",
          "org": "myorg",
          "percentile": "90",
          "proxy": "minty",
          "region": "ANY"
        },
        "metric": "totalLatency",
        "threshold": 6000,
        "durationSeconds": 300,
        "comparator": ">",
        "updatedBy": "me@foo.com"
      }
    ],
    "playbook": "Minty API",
    "throttleIntervalSeconds": 3600,
    "self": "/alerts/ef1a5249-345ed3023ea7c4",
    "feed": "/o/myorg/events/ef1a5249-345ed3023ea7c4",
    "organization": "myorg",
    "environment": "prod",
    "notifications": [
      {
        "channel": "email",
        "destination": "me@foo.com"
      }
    ],
    "updatedAt": "2018-07-19T18:19:33.22479Z",
    "updatedBy": "me@foo.com"
  },
 ...
]

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

View a specifc alert definition

View a specific alert definition by issuing a GET request to the following resource: https://apimonitoring.enterprise.apigee.com/alerts/alert_uuid where alert_uuid specifies the UUID of the alert definition. Get the UUID when you create the alert or use the API call shown in the previous section to list all alerts and their associated UUID.

For example:

curl 'https://apimonitoring.enterprise.apigee.com/alerts/4fa49a87-3463023ea7c4'
    -X GET
    -H 'Accept: application/json, text/plain, */*'
    -H "Authorization: Bearer $ACCESS_TOKEN"

The following provides an example of the response:

  {
    "uuid": "4fa49a87-3463023ea7c4",
    "name": "PublicAPI latency alert",
    "enabled": true,
    "description": "Public API Latency alerts, 90th %ile > 6secs for 5 minute window trigger this alert",
    "conditions": [
      {
        "uuid": "4fa49a87-3463023ea7c4",
        "description": "",
        "dimensions": {
          "env": "prod",
          "org": "myorg",
          "percentile": "90",
          "proxy": "PublicAPI",
          "region": "ANY"
        },
        "metric": "totalLatency",
        "threshold": 6000,
        "durationSeconds": 300,
        "comparator": ">",
        "updatedBy": "me@foo.com"
      }
    ],
    "playbook": "PublicAPI Latency alert, setup to go off when 90th %ile is > 4 secs for 5 minute window",
    "throttleIntervalSeconds": 3600,
    "self": "/alerts/4fa49a87-3463023ea7c4",
    "feed": "/o/myorg/events/4fa49a87-3463023ea7c4",
    "organization": "myorg",
    "environment": "prod",
    "notifications": [
      {
        "channel": "email",
        "destination": "me@foo.com"
      }
    ],
    "updatedAt": "2018-07-19T18:19:31.654738Z",
    "updatedBy": "me@foo.com"
  }

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

View history of all triggered alerts for an org

View the history of all alerts triggered for an org by issuing a GET request to the following resource: https://apimonitoring.enterprise.apigee.com/metrics/alerthistory.

You must pass the name of your organization using the org query parameter. You can optionally specify a time period used to search for triggered alerts. The default is to return all alerts triggered in the past hour.

For example:

curl 'https://apimonitoring.enterprise.apigee.com/metrics/alerthistory?org=myorg'
    -X GET
    -H 'Accept: application/json, text/plain, */*'
    -H "Authorization: Bearer $ACCESS_TOKEN"

The response contains an array of all alerts triggered during the requested time period. In the response body, the id field specifes the UUID of the triggered alert, and the shared_id field specifies the UUID of the alert definition associated with the triggered alert.

The following provides an example of the response. 

[
  {
    "id": "80cbe560-f6e0-475c6f7ed2d",
    "shared_id": "4fa49a87-3463023ea7c4",
    "organization": "myorg",
    "environment": "prod",
    "name": "PublicAPI latency alert",
    "type": "Alert",
    "source": "null/current",
    "raw_payload": "{\"reportUUID\":\"\",\"reportEnabled\":false,\"organization\":\"myorg\",\"name\":\"emgmt-api 404\",\"self\":\"/alerts/4fa49a87-3463023ea7c4\",\"description\":\"go/apigee-extensions-playbook\",\"conditions\":[
  {\"comparator\":\">\",\"metric\":\"rate\",\"durationSeconds\":300,\"name\":\"PublicAPI latency alert\",\"description\":\"\",\"threshold\":0.05,\"dimensions\":{\"proxy\":\"emgmt-api\",\"org\":\"myorg\",\"env\":\"prod\",\"region\":\"any\",\"statusCode\":\"404\"}}],\"uuid\":\"4fa49a87-3463023ea7c4\",\"playbook\":\"go/apigee-extensions-playbook\"}",
    "time": "2019-03-25T15:30:18Z"
  },
  {
    "id": "8131d740-6680-45b9c72c3",
    "shared_id": "1a64885b-f9-42010a850039",
    "organization": "apigee-pinpoint",
    "environment": "prod",
    "name": "Demo 5xx alert",
    "type": "Alert",
    "source": "null/current",
    "raw_payload": "{\"reportUUID\":\"\",\"reportEnabled\":false,\"organization\":\"myorg\",\"name\":\"Demo 5xx alert\",\"self\":\"/alerts/1a64885b-f9-42010a850039\",\"description\":\"Demo 5xx alert\",\"conditions\":[
  {\"comparator\":\">\",\"metric\":\"rate\",\"durationSeconds\":300,\"name\":\"Demo 5xx alert\",\"description\":\"\",\"threshold\":0.4,\"dimensions\":{\"proxy\":\"ALL\",\"org\":\"myorg\",\"env\":\"prod\",\"region\":\"any\",\"statusCode\":\"5xx\"}}],\"uuid\":\"1a64885b-f9-42010a850039\",\"playbook\":\"Recommended Playbook\"}",
    "time": "2019-03-25T15:57:30Z"
  },
 ...
]

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.

View history of specific alert

View the history of triggered alerts for a specific alert definition by issuing a GET request to the following resource: https://apimonitoring.enterprise.apigee.com/metrics/alerthistory.

You must pass the name of your organization using the org query parameter and the UUID of the alert definition. You can optionally specify a time period used to search for alerts. The default is to return all alerts triggered in the past hour.

You can obtain the alert definition UUID from the alert history as shown in the previous section, when you created the alert definition, or by using the API call shown in View all alert definitions.

For example:

curl 'https://apimonitoring.enterprise.apigee.com/metrics/alerthistory?org=myorg&alertId=4fa49a87-3463023ea7c4'
    -X GET
    -H 'Accept: application/json, text/plain, */*'
    -H "Authorization: Bearer $ACCESS_TOKEN"

The response contains an array of all alerts triggered during the requested time period for the specified alert definition UUID. In the response body, the id field specifes the UUID of the triggered alert, and the shared_id field specifies the UUID of the alert definition associated with the triggered alert.

The following provides an example of the response. 

[
  {
    "id": "80cbe560-f6e0-475c6f7ed2d",
    "shared_id": "4fa49a87-3463023ea7c4",
    "organization": "myorg",
    "environment": "prod",
    "name": "PublicAPI latency alert",
    "type": "Alert",
    "source": "null/current",
    "raw_payload": "{\"reportUUID\":\"\",\"reportEnabled\":false,\"organization\":\"myorg\",\"name\":\"emgmt-api 404\",\"self\":\"/alerts/4fa49a87-3463023ea7c4\",\"description\":\"go/apigee-extensions-playbook\",\"conditions\":[
  {\"comparator\":\">\",\"metric\":\"rate\",\"durationSeconds\":300,\"name\":\"PublicAPI latency alert\",\"description\":\"\",\"threshold\":0.05,\"dimensions\":{\"proxy\":\"emgmt-api\",\"org\":\"myorg\",\"env\":\"prod\",\"region\":\"any\",\"statusCode\":\"404\"}}],\"uuid\":\"4fa49a87-3463023ea7c4\",\"playbook\":\"go/apigee-extensions-playbook\"}",
    "time": "2019-03-25T15:30:18Z"
  },
  {
    "id": "9fc442d5-d607-40ef118c4e7",
    "shared_id": "4fa49a87-3463023ea7c4",
    "organization": "myorg",
    "environment": "prod",
    "name": "PublicAPI latency alert",
    "type": "Alert",
    "source": "null/current",
    "raw_payload": "{\"reportUUID\":\"\",\"reportEnabled\":false,\"organization\":\"myorg\",\"name\":\"emgmt-api 404\",\"self\":\"/alerts/4fa49a87-3463023ea7c4\",\"description\":\"go/apigee-extensions-playbook\",\"conditions\":[{\"comparator\":\">\",\"metric\":\"rate\",\"durationSeconds\":300,\"name\":\"PublicAPI latency alert\",\"description\":\"\",\"threshold\":0.05,\"dimensions\":{\"proxy\":\"emgmt-api\",\"org\":\"myorg\",\"env\":\"prod\",\"region\":\"any\",\"statusCode\":\"404\"}}],\"uuid\":\"4fa49a87-3463023ea7c4\",\"playbook\":\"go/apigee-extensions-playbook\"}",
    "time": "2019-03-25T15:17:55Z"
  },
 ...
]

Set $ACCESS_TOKEN to your OAuth 2.0 access token, as described in Obtain an OAuth 2.0 access token.

For information about the cURL options used in this example, see Use cURL.