AI Observability integration for Grafana Cloud

AI Observability Integration for Grafana Cloud provides a comprehensive solution for monitoring and analyzing the performance of AI models. This integration allows users to collect and visualize key metrics related to AI, including LLM operations and GPU usage.

This integration includes 2 pre-built dashboards to help monitor and visualize AI Observability metrics.

Dashboards

The AI Observability integration installs the following dashboards in your Grafana Cloud instance to help monitor your system.

GPU Monitoring
GenAI Observability

Metrics

The most important metrics provided by the AI Observability integration, which are used on the pre-built dashboards, are as follows:

db_requests_total
gen_ai_requests_total
gen_ai_usage_completion_tokens_total
gen_ai_usage_cost_USD_bucket
gen_ai_usage_cost_USD_sum
gen_ai_usage_input_tokens_total
gen_ai_usage_tokens_total
gpu_dec_utilization
gpu_enc_utilization
gpu_fan_speed
gpu_memory_available
gpu_memory_free
gpu_memory_total
gpu_memory_used
gpu_power_draw
gpu_power_limit
gpu_temperature
gpu_utilization
up

Changelog

# 0.0.1 - September 2024

* Initial release

Cost

By connecting your AI Observability instance to Grafana Cloud, you might incur charges. To view information on the number of active series that your Grafana Cloud account uses for metrics included in each Cloud tier, see Active series and dpm usage and Cloud tier pricing.