Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Grafana Enterprise Metrics Documentation

Admin API

Enterprise

Admin API documentation

NOTE: Admin API version v1 endpoints /admin/api/v1/ are deprecated and will be removed in a future release of GEM. Use the new Admin API version v2 endpoints /admin/api/v2/ instead.

In addition to the standard Cortex API endpoints, Grafana Enterprise Metrics supports an admin HTTP API for managing cluster resources such as tenants and tokens.

API operations

The API supports standard CRUD (create, read, update, and delete) operations for most resources. Creates are issued via POST requests, reads via GET requests, updates via PUT requests, and deletes via DELETE requests. When running multiple instances of the Admin API, we recommend that you enable leader election so that all mutations occur on a single, leader-elected instance.

Write consistency (If-Match and ETag headers)

To ensure concurrent changes are propagated in a way that avoids unintentional overwrites, an If-Match header is required for all requests that mutate data. The current version of a resource to be sent in the If-Match header can be found in the ETag header returned on all GET and PUT requests that return a single resource in the response body.

Here are a few examples of how to get the value of the ETag header using curl. The first example uses the argument -D - to write all headers to standard out, and then parses for the ETag header.

curl -u :$API_TOKEN -sD - -o /dev/null http://localhost:8080/admin/api/v2/accesspolicies/{name} | grep -i ETag | cut -d " " -f 2

This second example uses the -i option to print the headers along with the response payload.

curl -u :$API_TOKEN -i http://localhost:8080/admin/api/v2/accesspolicies/{name}

When making a request to mutate data, make sure to include the double quotes (") around the value in the If-Match header. So, for example, the following is a request to delete the access policy named the-access-policy, currently at version 1:

curl -u :$API_TOKEN -X DELETE -H 'If-Match: "1"' http://localhost:8080/admin/api/v2/accesspolicies/the-access-policy

Optionally, a wildcard "*" may be passed as the If-Match version which effectively disables the write consistency protections and allows any current version to be updated.

Cluster API

List clusters

GET /admin/api/v1/clusters

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/clusters instead.

GET /admin/api/v2/clusters

Response:

{
  "items": [
    {
      "name":  "<string>",              // cluster name
      "display_name": "<string>",       // cluster display name
      "created_at": "<rfc3339 string>", // date created 
      "kind": "<string>",               // cluster type, currently only cortex is supported
      "base_url": "<string>"            // Base URL of the cluster, optional
    }
    ...
  ],
  "type": "cluster"
}

Example:

$ curl -u :$API_TOKEN http://localhost:8080/admin/api/v2/clusters | jq
{
  "items": [
    {
      "name": "metrics-enterprise-dev",
      "display_name": "metrics-enterprise-dev",
      "created_at": "2020-07-13T16:50:41.953793Z",
      "kind": "cortex",
      "base_url": ""
    }
  ],
  "type": "cluster"
}

Get cluster

GET /admin/api/v1/clusters/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/clusters/{name} instead.

GET /admin/api/v2/clusters/{name}

NOTES:

There is no ETag header on a cluster response because clusters are immutable.

Response:

{
  "name":  "<string>",              // cluster name
  "display_name": "<string>",       // cluster display name
  "created_at": "<rfc3339 string>", // date created 
  "kind": "<string>",               // cluster type, currently only cortex is supported
  "base_url": "<string>"            // Base URL of the cluster, optional
}

Example:

$ curl -u :$API_TOKEN http://localhost:8080/admin/api/v2/clusters/metrics-enterprise-dev | jq
{
  "name": "metrics-enterprise-dev",
  "display_name": "metrics-enterprise-dev",
  "created_at": "2020-07-13T16:50:41.953793Z",
  "kind": "cortex",
  "base_url": ""
}

Tenant API

Model

The tenant model is made up of the following fields:

name: The machine-readable name of a tenant. It must be between 3 and 64 characters and only include the following characters, [a-z0-9-_]. Once set, this field is immutable.
display_name: The human-readable name of a tenant. It can contain any set of characters and can be changed.
status: The current status of this tenant. It can have the following values:
- active
- inactive
- unknown
cluster: The name of the Grafana Enterprise Metrics cluster that this tenant is scoped to. You will only be able to write/query this tenant from this cluster.
limits: Tenant specific resource usage limits. Limits are key-value pairs (JSON object) that correspond to Cortex limits configuration. This field is optional and will be omitted from responses when unset.

List tenants

GET /admin/api/v1/instances

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tenants instead.

GET /admin/api/v2/tenants

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "status": "<string>",
      "cluster": "<string>",
      "limits": {
        ...
      }
    },
    ...
  ],
  "type": "tenant"
}

Example:

$ curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants | jq
{
  "items": [
    {
      "name": "metrics-enterprise-dev",
      "display_name": "Grafana Enterprise Metrics Dev Tenant",
      "created_at": "2020-07-13T17:37:59.341728283Z",
      "status": "active",
      "cluster": "metrics-enterprise-dev",
      "limits": {
        "ingestion_rate": 350000,
        "ingestion_burst_size": 350000,
        "max_series_per_query": 100000,
        "max_global_series_per_user": 300000,
        "max_global_series_per_metric": 300000,
        "ruler_max_rules_per_rule_group": 0,
        "ruler_max_rule_groups_per_tenant": 0
      }
    }
  ],
  "type": "tenant"
}

Create tenant

POST /admin/api/v1/instances

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v2/tenants instead.

POST /admin/api/v2/tenants

Example:

curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants \
--data '{"name":"metrics-enterprise-dev", "display_name":"Grafana Enterprise Metrics Dev Tenant", "cluster": "metrics-enterprise-dev"}'

Update tenant

PUT /admin/api/v1/instances/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint PUT /admin/api/v2/tenants/{name} instead.

PUT /admin/api/v2/tenants/{name}

NOTES:

The name field of the object in the request body is ignored during updates.
The created_at field may not be modified during updates.
If-Match header matching the current version is required.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "cluster": "<string>",
  "limits": {
    ...
  }
}

Example:

curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants/metrics-enterprise-dev \
-X PUT -H 'If-Match: "123"` \
--data '{"display_name":"Grafana Enterprise Metrics Dev Tenant", "cluster": "metrics-enterprise-dev"}' | jq
{
  "name": "metrics-enterprise-dev",
  "display_name": "Grafana Enterprise Metrics Dev Tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "metrics-enterprise-dev",
  "limits": {
    "ingestion_rate": 350000,
    "ingestion_burst_size": 350000,
    "max_series_per_query": 100000,
    "max_global_series_per_user": 300000,
    "max_global_series_per_metric": 300000,
    "ruler_max_rules_per_rule_group": 0,
    "ruler_max_rule_groups_per_tenant": 0
  }
}

Get tenant

GET /admin/api/v1/instances/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tenants/{name} instead.

GET /admin/api/v2/tenants/{name}

NOTES:

The current version of the tenant will be returned in an ETag header on the response.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "cluster": "<string>",
  "limits": {
    ...
  }
}

Example:

$ curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants/metrics-enterprise-dev | jq
{
  "name": "metrics-enterprise-dev",
  "display_name": "Grafana Enterprise Metrics Dev Tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "metrics-enterprise-dev",
  "limits": {
    "ingestion_rate": 350000,
    "ingestion_burst_size": 350000,
    "max_series_per_query": 100000,
    "max_global_series_per_user": 300000,
    "max_global_series_per_metric": 300000,
    "ruler_max_rules_per_rule_group": 0,
    "ruler_max_rule_groups_per_tenant": 0
  }
}

Delete tenant

DELETE /admin/api/v1/instances/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint DELETE /admin/api/v2/tenants/{name} instead.

DELETE /admin/api/v2/tenants/{name}

NOTES:

If-Match header matching the current version is required.

Example:

curl -X "DELETE" -u :$API_TOKEN -H 'If-Match: "123"' localhost:8080/admin/api/v2/tenants/metrics-enterprise-dev

Access policy API

List access policies

GET /admin/api/v1/accesspolicies

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/accesspolicies instead.

GET /admin/api/v2/accesspolicies

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "realms": [
        {
          "tenant": "<string>",
          "cluster": "<string>"
        }
      ],
      "scopes": [
        "<string>",
        ...
      ]
    },
    ...
  ],
  "type": "access_policy"
}

Create access policy

POST /admin/api/v1/accesspolicies

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v2/accesspolicies instead.

POST /admin/api/v2/accesspolicies

Payload:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Realms

Realms designate the tenant/cluster pairs that the access policy allows requests for:

tenant: This field must be set to an existing tenant or *. * denotes access to all tenants.
cluster: This field must be set to an existing cluster.
label_policies: This is an optional field that can be set to a list of label policies. An access policy gets access to metrics that match at least one of the label policies. If label policies are not set, all metrics will match.

Label policy, selector, and label matchers

A label policy contains a selector, that consists of a list of label matchers.

The following label matcher types are supported:

EQ: Select labels that are exactly equal to the provided string.
NEQ: Select labels that are not equal to the provided string.
RE: Select labels that regex-match the provided string.
NRE: Select labels that do not regex-match the provided string.

{
  "selector": [
    {
      "type": "<enum: EQ | NEQ | RE | NRE>",
      "name": "<string>",
      "value": "<string>"
    }
  ]
}

The PromQL selector {job="grafana-labs", role!="software-engineer"}, would translate into this label policy:

{
  "selector": [
    {
      "type": "EQ",
      "name": "job",
      "value": "grafana-labs"
    },
    {
      "type": "NE",
      "name": "role",
      "value": "software-engineer"
    }
  ]
}

Scopes

Scopes designate what operations tokens assigned to this access policy will be able to do when calling the GEM API.

metrics:read: Permission to view data from a tenant
metrics:write: Permission to write data to a tenant
metrics:delete: Permission to delete data from a tenant
rules:read: Permission to read ruler rules
rules:write: Permission to write ruler rules
admin: Permission to perform admin operations

Update access policy

PUT /admin/api/v1/accesspolicies/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint PUT /admin/api/v2/accesspolicies/{name} instead.

PUT /admin/api/v2/accesspolicies/{name}

NOTES:

If-Match header matching the current version is required.
Update differs from create in that only the Display Name, Realms, and Scopes are updatable.

Payload:

{
  "display_name": "<string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Get access policy

GET /admin/api/v1/accesspolicies/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/accesspolicies/{name} instead.

GET /admin/api/v2/accesspolicies/{name}

NOTES:

The current version of the access policy will be returned in an ETag header on the response.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Delete access policy

DELETE /admin/api/v1/accesspolicies/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint DELETE /admin/api/v2/accesspolicies/{name} instead.

DELETE /admin/api/v2/accesspolicies/{name}

NOTES:

If-Match header matching the current version is required.

Example:

$ curl -X "DELETE" -u :$API_TOKEN -H 'If-Match: "123"' localhost:8080/admin/api/v2/accesspolicies/dev-read-write

Token API

List tokens

GET /admin/api/v1/tokens

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tokens instead.

GET /admin/api/v2/tokens

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "expiration": "<rfc3339 string>",
      "access_policy": "<string>"
    },
    ...
  ],
  "type": "key"
}

Create token

POST /admin/api/v1/tokens

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v2/tokens instead.

POST /admin/api/v2/tokens

Payload:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "expiration": "<rfc3339 string>",
  "access_policy": "<string>"
}

Get token

GET /admin/api/v1/tokens/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tokens/{name} instead.

GET /admin/api/v2/tokens/{name}

NOTES:

The current version of the token will be returned in an ETag header on the response.

Response:

{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "expiration": "<rfc3339 string>",
  "access_policy": "<string>"
}

Delete token

DELETE /admin/api/v1/tokens/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint DELETE /admin/api/v2/tokens/{name} instead.

DELETE /admin/api/v2/tokens/{name}

NOTES:

If-Match header matching the current version is required.

Example:

curl -X "DELETE" -u :$API_TOKEN -H 'If-Match: "123"' localhost:8080/admin/api/v2/token/dev-read-write

License API

List licenses

GET /admin/api/v1/licenses

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/licenses instead.

GET /admin/api/v2/licenses

Response:

{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "token": {
        "jti": "<string>",
        "iss": "<string>",
        "sub": "<string>",
        "iat": "<int64>",
        "exp": "<int64>",
        "nbf": "<int64>",
        "lexp": "<int64>",
        "lid": "<string>",
        "max_users": "<int64>",
        "included_admins": "<int64>",
        "included_viewers": "<int64>",
        "lic_exp_warn_days": "<int64>",
        "prod": [
          "<string>",
          ...
        ],
        "company": "<string>",
        "slug": "<string>",
      }
    },
    ...
  ],
  "type": "license"
}

Feature detection API

Get product and feature information

GET /admin/api/v1/features

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/features instead.

GET /admin/api/v2/features

Response:

{
  "product": "<string>",
  "product_version": "<string>",
  "features": {
      "<string>": "<string>",
      ...
  }
}

Example response:

{
  "product": "GEM",
  "product_version": "1.1",
  "features": {
      "lbac": "v2",
      "self_monitoring": "v1"
  }
}

Was this page helpful?

Email docs@grafana.com

Help and support

Community

Feedback

Relevant sources:

Feedback

Admin API documentation

API operations

Write consistency (If-Match and ETag headers)

Cluster API

List clusters

Get cluster

Tenant API

Model

List tenants

Create tenant

Update tenant

Get tenant

Delete tenant

Access policy API

List access policies

Create access policy

Realms

Label policy, selector, and label matchers

Scopes

Update access policy

Get access policy

Delete access policy

Token API

List tokens

Create token

Get token

Delete token

License API

List licenses

Feature detection API

Get product and feature information

Was this page helpful?

Related documentation

Admin API documentation

API operations

Write consistency (If-Match and ETag headers)

Cluster API

List clusters

Get cluster

Tenant API

Model

List tenants

Create tenant

Update tenant

Get tenant

Delete tenant

Access policy API

List access policies

Create access policy

Realms

Label policy, selector, and label matchers

Scopes

Update access policy

Get access policy

Delete access policy

Token API

List tokens

Create token

Get token

Delete token

License API

List licenses

Feature detection API

Get product and feature information

Was this page helpful?

Related documentation

Related resources from Grafana Labs