Amazon Aurora Datasource

Query and visualize your data from AWS Aurora with Grafana. This data source plugin is currently in public preview.

Introduction

Using the Amazon Aurora Datasource you can easily query and visualize your data in Amazon Aurora. This plugin currently has support for both Mysql-compatible and Postgres-compatible Aurora Engines.

Requirements

This plugin was designed for Enterprise versions of Grafana 9.4.7 and above, although it may work for older versions as well.

This plugin connects to AWS Aurora clusters using IAM with the AWS SDK for go. Please read through the Prerequisites section of the "Connecting to your DB cluster using IAM authentication and the AWS SDK for Go" guide from AWS on how to connect to your db cluster using IAM, to ensure Grafana will be able to query your cluster.

Example permissions

As part of the above, please ensure whatever user/role that you are using to query Aurora has the appropriate permissions. For example:

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
             "rds-db:connect"
         ],
         "Resource": [
             "arn:aws:rds-db:us-east-2:1234567890:dbuser:cluster-ABCDEFGHIJKL01234/db_user"
         ]
      }
   ]
}

For more information please read through the documentation from AWS

Getting started

1. Install the plugin
2. Add a new data source with the UI or provision one
3. Configure the data source
4. Start making queries

Configuration options

The Amazon Aurora data source uses the AWS SDK for Go to connect to your DB cluster with AWS IAM Credentials. There are a few possible ways to configure your data source instance with Grafana to specify how you'd like to connect with AWS:

Advanced Settings

In order to connect to your cluster, Grafana makes 2 calls:

1. Generating an auth token with RDS with an endpoint
2. Opening a connection to your DSN with that token and your endpoint.

Often the endpoint for these two steps is the same. However if you have your DB cluster behind a load balancer you may need 2 separate endpoints. For the first step you will need to specify the endpoint behind the load balancer. For the second step you will need to specify the load balancer endpoint for open SQL connections. To have Grafana do this on your behalf, specify the endpoint behind the load balancer in the DB Host for Auth and DB Port for Auth and use the load balancer endpoint/port configuration fields above.

Example of provisioned Aurora data source

If you're provisioning your data source you can set all of the above configuration options with a YAML file. Here's an example of what that file might look like:

  - name: AWS Aurora with mysql behind a load balancer
    type: grafana-aurora-datasource
    editable: true
    jsonData:
      engine: 'aurora-mysql'
      authType: keys
      dbName: 'testDatabase'
      dbUser: 'dbuser'
      dbHost: 'aurora-mysql.cluster-123.us-east-1.rds.amazonaws.com'
      dbPort: 3306
      defaultRegion: 'us-east-1'
    secureJsonData:
      accessKey: someAccessKey
      secretKey: someSecretKey
    version: 1

Here is another example with a load balancer:

  - name: AWS Aurora with mysql behind a load balancer
    type: grafana-aurora-datasource
    editable: true
    jsonData:
      engine: 'aurora-mysql'
      authType: keys
      dbName: 'testDatabase'
      dbUser: 'dbuser'
      dbHost: 'protectedByALoadBalancer.example.com'
      dbPort: 3307
      defaultRegion: 'us-east-1'
      dbPortAuth: 3306
      dbHostAuth 'aurora-mysql.cluster-123.us-east-1.rds.amazonaws.com'
    secureJsonData:
      accessKey: someAccessKey
      secretKey: someSecretKey
    version: 1

Querying

Table vs time series

As with other SQL data source plugins in Grafana, you can specify whether to return data as a "wide" or "long" time series when querying. This is controlled by the Format data frames as dropdown.

To better explain this concept, consider a table like this:

When we render this same query with the time series visualization, we'll notice that the "neighborhood" and "city" fields collapse into 1 time series:

This is often the desired and expected behavior from a sql datasource, particularly when using the Table Visualization. However, there are times when you may want to render a separate time series for each city/neighborhood combo. To do so, in the "Format data frames as" dropdown select "Time series" which will render that same timeseries visualization like so:

When using Explore, Grafana renders your data using the appropriate time series or table visualization automatically when you change the format. With the query editor in dashboards, users control how to visualize the data.

To learn more, refer to the Grafana documentation time series formats.

Variables and macros

This plugin supports both custom and global variables to make writing queries easier. For example, if you set a custom variable to be $tableName you can use the same query in a dashboard across multiple tables:

select * from $tableName limit 3;

{{< admonition type="note" }} Variables don't work with Grafana Alerting. {{ /admonition >}}

This plugin also supports macros from the sql util library such as $__timeFrom and $__timeTo. For example:

select * from test_table where $__timeFrom(recorded_at) and $__timeTo(recorded_at)

If you have a suggestion for a macro unique to the aurora datasource, feel free to reach out to customer support.