Menu
Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Alerts and IRMbreadcrumb arrow IRMbreadcrumb arrow Usebreadcrumb arrow Incident managementbreadcrumb arrow Declare incident
Grafana Cloud

Declare an incident in Grafana IRM

Declaring an incident is the first step in the formal incident response process. This action notifies your team about an issue, initiates your response workflow, and begins tracking the incident lifecycle.

This topic explains how to declare incidents in Grafana IRM from various entry points, including the Grafana Cloud interface, Slack, and existing alert groups.

Before you begin

To declare an incident in Grafana IRM, you need:

  • Access to Grafana Cloud
  • Permission to create incidents
  • Optionally, configured integrations to enable additional incident response actions

Incident declaration methods

Grafana IRM provides multiple ways to declare an incident, allowing you to start your response process from wherever you are.

Declare an incident from the IRM interface

The most direct method is to declare an incident from the IRM application interface:

  1. In the Grafana Cloud main menu, select IRM Incidents.
  2. Click the Declare incident button in the top right corner.
  3. In the declaration form, enter the required information:
    • Title: A clear, descriptive name for the incident
    • Severity: The impact level of the incident
    • Labels: Optional tags to categorize the incident
  4. Click More options to configure additional settings:
    • Channel prefix: For Slack channel creation
    • Status: The initial state of the incident (defaults to Active)
    • Start time: When the incident began (defaults to current time)
  5. Review the information and click Declare incident.

Tip

Create a descriptive title that clearly communicates the issue. For example, “API Gateway Latency Spike” is more helpful than “System Outage.”

Declare an incident from a dashboard panel

You can quickly declare an incident while viewing metrics in a dashboard:

  1. Navigate to the dashboard panel showing the concerning metrics.
  2. Click the panel’s menu icon (three dots) in the top right corner.
  3. Select More… > Declare incident.
  4. Complete the incident declaration form with relevant details.
  5. Click Declare incident.

Declare incident from a panel

Declare an incident from OnCall alert groups

When an alert group in OnCall requires a formal incident response:

  1. Navigate to Alerts & IRM > IRM > Alert groups.
  2. Select the relevant alert group from the list.
  3. On the alert group details page, click Actions and select Declare incident.
  4. In the incident form, complete the necessary fields.
  5. Click Declare incident.

Declare an incident from firing alerts in Grafana Alerting

You can create an incident directly from a firing alert:

  1. Navigate to Alerts & IRM > Alerting > Alert rules.
  2. In the list view, locate and select the firing alert that requires incident response.
  3. Click the Declare incident button.
  4. Complete the incident form and click Declare incident.

To declare an incident from a firing alert, complete the following steps.

Declare an incident from Slack

If you’ve configured the Grafana IRM Slack integration, you can declare incidents directly from Slack:

  1. In any Slack channel where the IRM app is installed, type /incident declare.
  2. Fill in the incident details in the Slack modal that appears.
  3. Click Submit to create the incident.

A dedicated Slack channel will be created for the incident, and team members will be notified according to your configured notification settings.

To learn how to manage incident from Slack, refer to Slack integration for Grafana IRM.

Declare a drill incident

Drill incidents let you practice your incident response procedures without affecting production systems or incident metrics. Use them for training, testing integrations, or validating your response process.

To declare a drill incident:

  1. Navigate to Alerts & IRM > Incidents.
  2. Click + Declare incident.
  3. Fill in the incident details as you would for a real incident.
  4. Instead of clicking “Declare incident,” click + Start a drill at the bottom of the form.
  5. When the drill incident is created, you can use it to practice your response procedures.

Tip

Regularly conduct drill incidents to ensure your team is familiar with the response process and to validate that your integrations are working correctly.

What happens after declaring an incident

When you declare an incident in Grafana IRM, several automated actions may occur depending on your configured integrations:

  • Creation of a dedicated Slack channel for incident communication
  • Notification of relevant team members
  • Generation of incident documentation
  • Initiation of conference bridges via Zoom or other tools
  • Creation of tickets in external systems like Jira

You can update incident details such as title, severity, labels, and status at any time during the incident lifecycle.