Schedules as code

This topic explains how to manage on-call schedules in Grafana Incident Response Management (IRM) using Infrastructure as Code (IaC).

Overview

Managing on-call schedules using Infrastructure as Code enables you to:

Automate schedule creation and updates
Maintain consistency across environments
Track changes with version control
Integrate schedule management into your CI/CD pipeline
Scale schedule management across multiple teams and services

Grafana IRM supports two primary methods for managing schedules as code:

Using Terraform with the Grafana provider
Using the Grafana IRM API directly

Manage on-call schedules with Terraform

Terraform allows you to define on-call schedules as code, ensuring reproducibility and version control. Using the Grafana Terraform provider, you can create and manage on-call schedules within Grafana IRM.

Before you begin

You need:

A Grafana Cloud account using IRM
A Grafana IRM API token with appropriate permissions
Terraform installed (Install Terraform)
Grafana provider configured (Terraform Provider Documentation)

For initial setup, refer to Infrastructure as Code.

Example: Creating an on-call schedule with Terraform

The following example shows a basic on-call schedule configuration:

provider "grafana" {
  url  = "https://grafana.com/"
  auth = var.grafana_api_key
}

resource "grafana_oncall_schedule" "example_schedule" {
  name    = "Primary On-Call Schedule"
  team_id = "your-team-id"

  rotation {
    start_date           = "2025-03-01T00:00:00Z"
    time_zone            = "UTC"
    shift_length_seconds = 86400  # 24 hours in seconds
    participants         = ["user1", "user2"]
  }
}

Set up an on-call rotation via Terraform

The following example demonstrates how to set up a weekly rotation where two users alternate on-call duties.

Define users

First, reference the users who will participate in the rotation:

data "grafana_oncall_user" "user_one" {
  provider = grafana.oncall
  username = "user_one"
}

data "grafana_oncall_user" "user_two" {
  provider = grafana.oncall
  username = "user_two"
}

Create an on-call shift

Next, define the shift pattern:

resource "grafana_oncall_on_call_shift" "week_shift" {
  provider   = grafana.oncall
  name       = "Week shift"
  type       = "rolling_users"
  start      = "2025-03-01T00:00:00"
  duration   = 60 * 60 * 24  # 24 hours in seconds
  frequency  = "weekly"
  by_day     = ["MO", "TU", "WE", "TH", "FR", "SA", "SU"]
  week_start = "MO"
  rolling_users = [
    [data.grafana_oncall_user.user_one.id],
    [data.grafana_oncall_user.user_two.id]
  ]
  time_zone = "UTC"
}

For all available parameters, refer to the Grafana Terraform provider reference.

Connect the shift to a schedule

Create a schedule and associate it with the shift:

resource "grafana_oncall_schedule" "primary" {
  provider  = grafana.oncall
  name      = "Primary"
  type      = "calendar"
  time_zone = "UTC"
  shifts    = [
    grafana_oncall_on_call_shift.week_shift.id
  ]
}

Add the schedule to an escalation chain

Finally, add the schedule to an escalation chain to use it for incident notifications:

resource "grafana_oncall_escalation" "notify_schedule" {
  provider                     = grafana.oncall
  escalation_chain_id          = grafana_oncall_escalation_chain.default.id
  type                         = "notify_on_call_from_schedule"
  notify_on_call_from_schedule = grafana_oncall_schedule.primary.id
  position                     = 0
}

After applying your configuration with terraform apply, you can view your schedule in Grafana IRM under the Schedules tab or export it as an iCal link for external calendar validation.

Manage on-call schedules with the API

The Grafana IRM API provides granular control over on-call schedules, enabling dynamic updates and integrations with other systems.

API endpoints

The On-Call Schedules API supports the following operations:

List schedules: GET /api/oncall/schedules
Create a schedule: POST /api/oncall/schedules
Update a schedule: PUT /api/oncall/schedules/{schedule_id}
Delete a schedule: DELETE /api/oncall/schedules/{schedule_id}

For complete API documentation, refer to the Grafana IRM API Reference: Schedules.

Example: Creating an on-call schedule via API

The following example shows how to create a schedule using a curl command:

curl -X POST "https://grafana.com/api/oncall/schedules" \
-H "Authorization: Bearer $GRAFANA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "name": "Primary On-Call Schedule",
  "team_id": "your-team-id",
  "rotations": [
    {
      "start_date": "2025-03-01T00:00:00Z",
      "time_zone": "UTC",
      "shift_length_seconds": 86400,
      "participants": ["user1", "user2"]
    }
  ]
}'

Time zone considerations

When working with schedules as code, time zones are a critical consideration:

Always specify the time zone for each rotation to avoid scheduling confusion
Use consistent time zone formats across your configuration
Consider the global distribution of your team when setting up rotations
Test schedules across time zone boundaries to ensure proper handoffs
Be aware of daylight saving time transitions when scheduling long-term rotations

Best practices for schedules as code

Follow these best practices when managing your on-call schedules as code:

Use source control: Store Terraform configurations and API scripts in a Git repository to track changes
Automate deployments: Use CI/CD tools like GitHub Actions or Terraform Cloud for managing schedules across environments
Apply role-based access control (RBAC): Restrict who can modify schedules via API keys and IAM policies to prevent unauthorized changes
Test before applying: Use terraform plan or API dry-runs to validate configurations before deploying to production
Document your approach: Maintain documentation about your schedule structure and rotation patterns
Implement monitoring: Set up alerts for schedule configuration changes or failures
Use variables and modules: Leverage Terraform variables and modules to create reusable schedule patterns