Build a data source backend plugin
Introductionβ
Grafana supports a wide range of data sources, including Prometheus, MySQL, and Datadog. In some cases, though, you already have an in-house metrics solution that youβd like to add to your Grafana dashboards. This tutorial teaches you to build a new data source plugin to query data.
A backend component provides a number of additional capabilities to your plugin, such as custom authentication methods. To learn more, refer to the documentation on Backend plugins.
In this tutorial, you'll:
- Build a backend for your data source
- Implement a health check for your data source
- Enable Grafana Alerting for your data source
Prerequisitesβ
Create a new pluginβ
The Grafana create-plugin tool is a CLI application that simplifies Grafana plugin development, so that you can focus on code. The tool scaffolds a starter plugin, all the required configuration, and a development environment using Docker Compose for you.
-
In a new directory, create a plugin from a template using the create-plugin tool. When prompted for the kind of plugin, select and answer yes to "Do you want a backend part of your plugin?":
npx @grafana/create-plugin@latest
-
Go to the directory of your newly created plugin:
cd <your-plugin>
-
Install the dependencies:
npm install
-
Build the plugin frontend:
npm run dev
-
In a new terminal window, build the plugin backend:
mage -v build:linux
-
Start Grafana:
docker compose up
- Open Grafana, by default http://localhost:3000/, and then go to Administration > Plugins. Make sure that your plugin is there.
You can also verify that Grafana has discovered the plugin by checking the logs:
INFO[01-01|12:00:00] Plugin registered logger=plugin.loader pluginID=<your-plugin>
Now, let's verify that the plugin you've built so far can be used in Grafana when creating a new data source:
- On the side menu, go to Connections > Data Sources.
- Click Add data source.
- Search for the name of your newly created plugin and select it.
- Enter a name and then click Save & Test. If a "randomized error" occurs, you may ignore it - this is a result of the health check explained further below.
You now have a new data source instance of your plugin that is ready to use in a dashboard.
To add the data source to the dashboard:
- Create a new dashboard and add a new panel.
- On the query tab, select the data source you just created. A line graph is rendered with one series consisting of two data points.
- Save the dashboard.
Troubleshootingβ
Grafana doesn't load my pluginβ
Ensure that Grafana has been started in development mode. If you are running Grafana from source, add the following line to your conf/custom.ini
file:
app_mode = development
If you don't have a conf/custom.ini
file already, create it before proceeding.
You can then start Grafana in development mode by running make run & make run-frontend
in the Grafana repository root.
If you are running Grafana from a binary or inside a Docker container, you can start it in development mode by setting the environment variable GF_DEFAULT_APP_MODE
to development
.
By default, Grafana requires backend plugins to be signed. To load unsigned backend plugins, you need to configure Grafana to allow unsigned plugins. For more information, refer to Plugin signature verification.
Anatomy of a backend pluginβ
The folders and files used to build the backend for the data source are:
file/folder | description |
---|---|
Magefile.go | Itβs not a requirement to use mage build files, but we strongly recommend using them so that you can use the build targets provided by the plugin SDK. |
/go.mod | Go modules dependencies. |
/src/plugin.json | A JSON file describing the plugin. |
/pkg/main.go | Starting point of the plugin binary. |
The plugin.json fileβ
The plugin.json
file is required for all plugins. When building a backend plugin, pay attention especially to these properties:
property | description |
---|---|
backend | Set to true for backend plugins. This tells Grafana that it should start a binary when loading the plugin. |
executable | This is the name of the executable that Grafana expects to start. Refer to plugin.json reference for details. |
alerting | If your backend data source supports alerting, set to true . Requires backend to be set to true . |
In the next step we will look at the query endpoint!
Implement data queriesβ
We begin by opening the file /pkg/plugin/datasource.go
. In this file you will see the Datasource
struct which implements the backend.QueryDataHandler interface. The QueryData
method on this struct is where the data fetching happens for a data source plugin.
Each request contains multiple queries to reduce traffic between Grafana and plugins. So you need to loop over the slice of queries, process each query, and then return the results of all queries.
In the tutorial we have extracted a method named query
to take care of each query model. Since each plugin has their own unique query model, Grafana sends it to the backend plugin as JSON. Therefore the plugin needs to Unmarshal
the query model into something easier to work with.
As you can see the sample only returns static numbers. Try to extend the plugin to return other types of data.
For example to generate three floats equally spaced in time, you can replace the two static numbers generated, using the following code:
duration := query.TimeRange.To.Sub(query.TimeRange.From)
mid := query.TimeRange.From.Add(duration / 2)
s := rand.NewSource(time.Now().UnixNano())
r := rand.New(s)
lowVal := 10.0
highVal := 20.0
midVal := lowVal + (r.Float64() * (highVal - lowVal))
// add fields.
frame.Fields = append(frame.Fields,
data.NewField("time", nil, []time.Time{query.TimeRange.From, mid, query.TimeRange.To}),
data.NewField("values", nil, []float64{lowVal, midVal, highVal}),
)
You can read more about how to build data frames in our docs.
Add support for health checksβ
Implementing the health check handler allows Grafana to verify that a data source has been configured correctly.
When editing a data source in Grafana's UI, you can Save & Test to verify that it works as expected.
In this sample data source, there is a 50% chance that the health check will be successful. Make sure to return appropriate error messages to the users, informing them about what is misconfigured in the data source.
Open /pkg/plugin/datasource.go
. In this file, you'll see that the Datasource
struct also implements the backend.CheckHealthHandler interface. Go to the CheckHealth
method to see how the health check for this sample plugin is implemented.
To learn more, refer to other Health Check implementations in our examples repository.
Add authenticationβ
Implementing authentication allows your plugin to access protected resources like databases or APIs. To learn more about how to authenticate using a backend plugin, refer to our documentation.
Enable Grafana Alertingβ
-
Add the top level
alerting
property with a value oftrue
to specify that your plugin supports Grafana Alerting, e.g.src/plugin.json{
...
"backend": true,
"executable": "gpx_simple_datasource_backend",
"alerting": true,
"info": {
...
} -
Restart your Grafana instance.
-
Open Grafana in your web browser.
-
Verify that alerting is now supported by navigating to your created data source. You should see an "Alerting supported" message in the Settings view.
Create an alertβ
The following instructions are based on Grafana v10.1.1, consult the documentation for alerting for version appropriate guidance.
- Open the dashboard you created earlier in the Create a new plugin step.
- Edit the existing panel.
- Click on the Alert tab underneath the panel.
- Click on Create alert rule from this panel button.
- In Expressions section, in the Threshold expression
C
, set the IS ABOVE to15
. - Click on Set as alert condition on Threshold expression
C
. Your alert should now look as follows. - In Set alert evaluation behavior section, click on New folder button and create a new folder to store an evaluation rule.
- Then, click on New evaluation group button and create a new evaluation group; choose a name and set the Evaluation interval to
10s
. - Click Save rule and exit button.
- Save the dashboard. After some time, the alert rule evaluates and transitions into the Alerting state.
Run multiple queries concurrentlyβ
This feature is only available for the Grafana backend plugin SDK version 0.232.0 and later.
By default, multiple queries within a single request (that is, within one panel) are executed sequentially. To run multiple queries concurrently, you can use the concurrent.QueryData
function that the SDK exposes.
To use concurrent.QueryData
, specify how to execute a single query and a limit on the number of concurrent queries to run. Note that the maximum is 10 concurrent queries.
import (
...
"github.com/grafana/grafana-plugin-sdk-go/experimental/concurrent"
...
)
func (d *Datasource) handleSingleQueryData(ctx context.Context, q concurrent.Query) (res backend.DataResponse) {
// Implement the query logic here
}
func (d *Datasource) QueryData(ctx context.Context, req *backend.QueryDataRequest) (*backend.QueryDataResponse, error) {
return concurrent.QueryData(ctx, req, d.handleSingleQueryData, 10)
}
Summaryβ
In this tutorial you created a backend for your data source plugin.