Prometheus Metrics Setup for GCP | Aviator Documentation

This how-to guide explains how to setup Prometheus Metrics collection for GCP. We are going to use Open Telemetry collector running on GKE to scrape and feed the Aviator Prometheus metrics. Pre-requisite GKE cluster Step 1: Create a service account and setup permissions We are going to create a GCP service account that has necessary permission to write metrics. This service account is used via GKE's workload identify federation. Create a GCP SA

metric-collector@YOUR_PROJECT.iam.gserviceaccount.com

and give it

roles/monitoring.metricWriter

and

roles/monitoring.viewer

permission at the project level. On the service account permission, give Kubernetes Service Account to use that GCP SA. Give

roles/iam.workloadIdentityUser

to

serviceAccount:YOUR_PROJECT.svc.id.goog[YOUR_K8S_NAMESPACE/metric-collector

. Step 2: Create a Kubernetes secret for otel-collector Open Telemetry collector requires a config where to scrape the metrics and where to send the metrics. Create the following Kubernetes Secret for the configuration. Copy

apiVersion: v1
kind: Secret
metadata:
  name: otel-config
stringData:
  config.yaml: |
    receivers:
      prometheus:
        config:
          scrape_configs:
            - job_name: "aviator-mergequeue"
              metrics_path: "/api/metrics"
              scheme: "https"
              params:
                repos:
                  - "YOUR_ORG/YOUR_REPO"
              authorization:
                type: "Bearer"
                credentials: "mq_live_ ... YOUR API KEY HERE"
              static_configs:
                - targets:
                  - "app.aviator.co"

    processors:
      resourcedetection:
        detectors: [gcp]
        timeout: 10s

      batch:
        send_batch_max_size: 200
        send_batch_size: 200
        timeout: 5s

      memory_limiter:
        check_interval: 1s
        limit_percentage: 65
        spike_limit_percentage: 20

    exporters:
      googlemanagedprometheus:

    service:
      pipelines:
        metrics:
          receivers: [prometheus]
          processors: [batch, memory_limiter, resourcedetection]
          exporters: [googlemanagedprometheus]

Update

YOUR_ORG/YOUR_REPO

and

YOUR API KEY HERE

parts in the config. The API key can be obtained from https://app.aviator.co/integrations/api . Apply this with

kubectl apply -n YOUR_K8S_NAMESPACE -f FILE

. Step 3: Deploy otel-collector Deploy the open-telemetry collector to your GKE cluster. Apply this config to your namespace. Step 4: Setup a dashboard The metrics should appear as a Prometheus Metric. You can setup a dashboard like this to monitor the queue length and GitHub API usage on your account. Previous GraphQL API Quickstart Next Reference Last updated 1 year ago Was this helpful?

The technical content and code provided contain several issues and inaccuracies. Here are the identified problems and suggestions for corrections:

1. Spelling and Grammar:

   • "setup" should be "set up" when used as a verb. The correct phrase is "how to set up."
   • "Pre-requisite" should be "Prerequisite."
   • "identify" should be "identity" in "workload identify federation."

2. Service Account Configuration:

   • The service account email should be formatted correctly. Ensure that YOUR_PROJECT is replaced with the actual project ID.
   • The IAM role roles/iam.workloadIdentityUser should be granted to the Kubernetes service account, but the syntax for specifying the service account is incorrect. It should be: serviceAccount:YOUR_PROJECT.svc.id.goog[YOUR_K8S_NAMESPACE/metric-collector] Note the closing bracket ] at the end.

3. Kubernetes Secret Configuration:

   • The stringData field in the Kubernetes Secret is correct, but ensure that the YAML indentation is maintained when applying it.
   • The exporters section under googlemanagedprometheus is incomplete. It should specify the configuration for the Google Managed Prometheus exporter. For example: yaml exporters: googlemanagedprometheus: project: YOUR_PROJECT_ID Replace YOUR_PROJECT_ID with your actual GCP project ID.

4. Kubernetes Command:

   • The command kubectl apply -n YOUR_K8S_NAMESPACE -f FILE should specify the actual file name instead of FILE. Replace FILE with the path to your configuration file.

5. General Instructions:

   • Ensure that placeholders like YOUR_ORG/YOUR_REPO and YOUR API KEY HERE are replaced with actual values before applying the configuration.
   • The guide should include more detailed steps for deploying the OpenTelemetry Collector, such as providing a sample deployment YAML file or Helm chart instructions.

6. Additional Clarifications:

   • The guide should clarify how to obtain the API key from Aviator and any specific permissions required for it.
   • Consider adding a step to verify the deployment and ensure that metrics are being collected and exported correctly.

By addressing these issues, the guide will be more accurate and helpful for users setting up Prometheus metrics collection on GCP using OpenTelemetry.

No product messaging analysis available for this page.

No competitive analysis available for this page.