Incident insights
Incident insights allows you to monitor your incident data to discover key metrics and trends about your organization’s incident response performance. These insights enable you to measure response effectiveness, identify patterns, and improve your incident management processes.
About incident insights
With Incident Insights, you can:
- Evaluate key response metrics with a pre-built dashboard
- Build custom dashboards using the Grafana Incident data source
- Customize the dashboard template to fit specific needs
- Filter and explore metrics based on labels, severity, and other dimensions
- Embed incident response metrics into your existing dashboards
Incident Insights is included with Grafana IRM at no additional cost, regardless of your Grafana Cloud tier.
Set up incident insights
The Incident Insights dashboard is created automatically when you set up the feature. To enable and access Incident Insights:
- In your Grafana Cloud stack, navigate to IRM
- Select Insights, and click the Incident tab
- Click Set up Insights dashboard
- Once complete, you’ll be redirected to the Insights dashboard
During setup, the following actions are performed automatically:
- Configuration of the Grafana Incident data source
- Generation of a service account and token with viewer permissions
- Creation of the pre-built Insights dashboard
To revisit the dashboard later, click Insights in the left-side menu.
Working with incident data
Creating custom dashboards
While the pre-built dashboard provides common incident response metrics, you can create custom dashboards tailored to your organization’s needs:
- Navigate to Dashboards > New Dashboard
- Add a new panel
- Select the Grafana Incident data source
- Create queries using the expressions described below
- Configure visualizations to highlight important metrics
Query syntax
The Incident data source uses filter expressions similar to those in the Grafana Incident interface. You can reuse any bookmarked or saved expressions from Grafana Incident.
Basic query examples
Query for critical or security-related incidents:
or(severity:critical label:security)
Query for active incidents declared within a specific time frame:
status:active started:${__from:date}, ${__to:date}
Query for resolved incidents without a specific label:
status:resolved -label:value
Query for resolved incidents with certain labels:
status:resolved or($labels)
Query for incidents with multiple specific labels:
status:resolved or(label:'squad:incident' label:PIR)
Using variables
The Query Editor supports Grafana Variables to create interactive dashboards. You can create variables for:
- Status: Filter by incident status (active, resolved, etc.)
- Severity: Filter by incident severity levels
- Labels: Filter by incident labels
Variables allow you to create dynamic dashboards where users can select different values to update the visualizations.
Example variable query
To create a variable that fetches all available incident labels:
- Create a new dashboard variable
- Select the Grafana Incident data source
- Use a query that returns label values
- Configure the variable to allow multi-selection
Analyzing incident data
Effective incident analysis typically focuses on metrics such as:
- Mean Time To Resolution (MTTR): Average time from incident declaration to resolution
- Incident frequency: Number of incidents over time, by severity or service
- Team impact: Distribution of incidents across teams and services
- Root cause patterns: Common themes in incident causes identified through labels
- Time of day patterns: When incidents are most likely to occur
Best practices
- Start with the pre-built dashboard to understand available metrics
- Create custom dashboards for specific teams or services
- Use consistent labeling to enable meaningful filtering and grouping
- Track trends over time rather than focusing on point-in-time metrics
- Compare related metrics such as MTTR alongside incident count
- Share insights with stakeholders to drive improvement initiatives
Next steps
- Learn about alert insights to complement your incident metrics
- Configure incident management to improve your processes
- Set up dashboards for custom visualizations