Apigee hybrid collects operations metrics that you can use to monitor the health of hybrid services, to set up alerts, and so on. Apigee uses the industry-standard Prometheus add-on for metrics collection. Once collected, Hybrid sends the metrics data to Stackdriver, at which point you can use the Stackdriver console for viewing, searching, and analyzing metrics and managing alerts.
The following diagram shows the metrics collection process:
As you can see in this diagram, there is one Prometheus server running per cluster, and it can run on any pod in the cluster. Prometheus scrapes application metrics data from all Hybrid services and sends the metrics data to Stackdriver. You can access metrics data through the Stackdriver console.
Metrics collection is enabled by default. To disable it, see Disable metrics collection.
Application metrics data is made available on a port as an internal Kubernetes service. Data collected by this service is scraped by the hybrid metrics collector service. You can use the Stackdriver Metrics Explorer to select metrics you want to view, such as:
- read request count
- read request latency
- write request count
- write request latency
For example, you can create a dashboard in Stackdriver to show your metrics:
Configure metrics collection
To send metrics to the
application for your GCP account, you must set configuration properties in the
Do this configuration whether you are on a GKE or a non-GKE cluster.
To configure metrics collection:
- Create a GCP service account with the Monitoring Metrics Writer role. You can create
the service account using the hybrid CLI command create-service-account:.
./tools/create-service-account my-metrics-svc-account apigee-metricsFor more information about GCP service accounts, see Creating and managing service accounts.
create-service-accountcommand saves a key on your system as a
.jsonfile. Note the path to the file. You will need the path in the following steps.
- Add the following configuration to your
overrides.yaml metrics: enableMetrics: true serviceAccountPath: sa_json_file_path project_id: GCP_project_ID k8s_cluster_name: cluster_name k8s_cluster_location: cluster_region (Must be a GCP region name. See the note below.)
- sa_json_file_path is the path
on your filesystem to the service account JSON file the
- GCP_project_ID is your GCP project ID for the project where you want the metrics to be sent to.
- cluster_name is the name of the cluster running hybrid.
- cluster_region is a GCP region name. See the note below for more information:
metrics: enableMetrics: true serviceAccountPath: $HOME/hybrid/user-9c0dcb0c6c45.json project_id: myproject k8s_cluster_name: user-cluster k8s_cluster_location: us-west2
- sa_json_file_path is the path on your filesystem to the service account JSON file the
- GKE only: If you are on GKE, in addition to
the mandatory overrides, you need to grant your own user the GKE
cluster admin role:
ACCOUNT=$(gcloud info --format='value(config.account)')
kubectl create clusterrolebinding owner-cluster-admin-binding \ --clusterrole=cluster-admin --user=$ACCOUNT
- Apply the configuration to your cluster.
View metrics on Stackdriver
You can use Stackdriver to view your metrics information. You do this by selecting a resource type and a metric that has data from that resource type.
To view metrics on Stackdriver:
- Open the Stackdriver Metrics Explorer in a browser. Alternatively, if you're already in the Stackdriver console, select Resources > Metrics Explorer.
- Select the metric you want to examine. For example, to see metrics for proxy 4xx errors:
- In Find resource type and metric, begin typing "proxy" in the search box.
- Filter on
GETfrom the menu.
Stackdriver displays data for the selected metric.
- To save the selected metrics settings, click the Save Chart button.
For more information about metrics, see the following:
Disable metrics collection
Metrics collection is enabled by default for hybrid. To disable metrics collection, add the following attribute to overrides.yaml and apply the change to your cluster:
overrides.yaml metrics: enableMetrics: false
Apply the change:
apigeectl apply -c metrics -v beta
Checking service account permissions
To check the permissions for a GCP service account:
- In the GCP console, be sure your project is selected.
- Go to the GKE Kubernetes clusters screen.
- Select your cluster.
- Select the Nodes tab. This tab shows you a list of all of the nodes in your cluster.
- Select one of the nodes from the list.
- Select the Details tab.
- Click the VM instance link.
- Locate the Service account and make a note of the ID (it looks like an email address).
- Go to IAM & Permissions in the GCP console.
- Select IAM.
- Locate the service account ID in the list.
- The service account roles are listed there.
For more information, see Granting roles to service accounts.