Push metrics from Influx Telegraf to Prometheus
We have support to ingest Influx Line protocol into Grafana Cloud. You can point your Telegraf and other services pushing Line protocol metrics to GrafanaCloud via HTTP.
If you are using Telegraf to push metrics, the following configuration is required:
[[outputs.influxdb]]
urls = ["<Modified metrics instance remote_write endpoint>/api/v1/push/influx"]
skip_database_creation = true
## HTTP Basic Auth
username = "Your Metrics instance ID"
password = "Your Cloud Access Policy token"
You can find the URL, username, and password for your metrics endpoint by clicking on Details in the Prometheus card of the Cloud Portal.
The URL is based on the Remote Write Endpoint URL but is changed slightly.
You need to replace prometheus
with influx
and change the path from /api/prom/push
to api/v1/push/influx
.
For example if the remote_write endpoint is https://prometheus-prod-03-prod-us-central-0.grafana.net/api/prom/push
, your influx endpoint will be https://influx-prod-03-prod-us-central-0.grafana.net/api/v1/push/influx
.
NOTE: There is one special case exception to the new endpoint form. If your endpoint url was prometheus-us-central1.grafana.net
, your new influx endpoint will be https://influx-prod-06-prod-us-central-0.grafana.net/api/v1/push/influx
.
Influx line protocol has the following structure:
metric_name (set of k=v tags) (set of k=v fields) timestamp
We convert the above into the following series in Prometheus:
for each (field, value) in fields:
metric_name_field{tags...} value @timestamp
For example:
diskio,host=work,name=dm-1 write_bytes=651264i,read_time=29i 1661369405000000000
Will be converted to:
diskio_write_bytes{host="work", name="dm-1"} 651264 1661369405000000000
diskio_read_time{host="work", name="dm-1"} 29 1661369405000000000
The timestamp is optional. If specified, we expect the timestamp to be encoded in Nanoseconds. If no timestamp is specified, we will use the timestamp millisecond that our server receives the metric.
Pushing from applications directly
NOTE: The actual destination path for this API is /api/v1/push/influx/write
.
The telegraf plugin automatically appends “/write” to the URL provided, so it shouldn’t be specified in the configuration.
If you’re using a different plugin or pushing directly, you may need to specify the full path of /api/v1/push/influx/write
in your endpoint URL.
For example: https://influx-prod-03-prod-us-central-0.grafana.net/api/v1/push/influx/write
Range queries and single data points
If you only push a single point to test, you may see multiple points if you perform a range query for that metric in Grafana. This is due to the way that Prometheus executes range queries and specifically the concept of staleness.
A range query evaluates the metric at every step in the time range, each evaluation looks for the latest value of the metric in the five minutes before the evaluation time (due to staleness handling). If you ingest a single sample and have a step time of 1m
in your range query, you’ll get one point per minute appear on the graph during the staleness period despite only ingesting a single point.
Examples
Additional examples using cURL, Node.js, Python, and Ruby:
cURL
TOKEN="<user_id>:<token>"
URL="https://influx[...].grafana.net/api/v1/push/influx/write"
curl -X \
POST -H \
"Authorization: Bearer $TOKEN" -H \
"Content-Type: application/json" "$URL" -d "test,bar_label=abc,source=grafana_cloud_docs metric=35.2"
Node.js
import fetch from 'node-fetch';
const USER_ID = '123456';
const TOKEN = 'your-token-123-!!!';
const body = 'test,bar_label=abc,source=grafana_cloud_docs metric=35.2';
// Endpoint is on the same page as the userid
const response = await fetch('https://influx[...].grafana.net/api/v1/push/influx/write', {
method: 'post',
body,
headers: {
Authorization: `Bearer ${USER_ID}:${TOKEN}`,
'Content-Type': 'application/json',
},
method: 'post',
body,
headers: {
Authorization: `Bearer ${USER_ID}:${TOKEN}`,
'Content-Type': 'text/plain',
},
});
const data = await response.json();
console.log(data);
Python
import requests
import base64
USER_ID = '123456'
TOKEN = 'your-token-123-!!!'
body = 'test,bar_label=abc,source=grafana_cloud_docs metric=35.2'
response = requests.post('https://influx[...].grafana.net/api/v1/push/influx/write',
headers = {
'Content-Type': 'text/plain',
},
data = str(body),
auth = (USER_ID, TOKEN)
)
status_code = response.status_code
print(status_code)
Ruby
require 'net/http'
require 'uri'
require 'base64'
USER_ID = '123456'
TOKEN = 'your-token-123-!!!'
BASE64_ENCODED_AUTH = Base64.strict_encode64(USER_ID + ':' + TOKEN)
uri = "https://influx[...].grafana.net/api/v1/push/influx/write"
uri = URI.parse(uri)
body = 'test,bar_label=abc,source=grafana_cloud_docs metric=35.2'
response = Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |client|
request = Net::HTTP::Post.new(uri.path)
request.body = body
request["Authorization"] = "Basic #{BASE64_ENCODED_AUTH}"
request["Content-Type"] = "text/plain"
client.request(request)
end
data = response.code
puts data
Pushing using the remote-write output
Telegraf also supports using a native Prometheus remote-write endpoint.
You can use the same Prometheus URL given in your cloud portal for this. No need to change the URL.
Current limitations:
- No matter what the precision you send the data in, we will store it in millisecond precision. We currently don’t have support in our platform to do higher than millisecond precision.
- We support only float64 on our platform. This means all the integer and boolean values will be cast into floating point before storage. True becomes 1 and false becomes 0. We currently don’t ingest string values.
- We don’t support queries via Flux, you will need to use PromQL, but we have it part of our roadmap to support Flux.