Performance testing with Grafana k6 and GitHub Actions

Note: This content was originally published on k6.io. It was updated in July 2024 by Ayush Goyal, a senior software engineer at Grafana Labs.

By running performance tests continuously and automatically, you can identify and correct performance regressions as they occur. One way to do this is by integrating performance testing into your development process.

In this step-by-step post, we explore how to do just that, using Grafana k6 and GitHub Actions.

k6 is an open source load testing tool to test the performance of APIs, microservices, and websites. Developers use k6 to test a system’s performance under a particular load to catch performance regressions or errors.

GitHub Actions is a tool that enables developers to create custom workflows for their software development lifecycle directly inside their GitHub repositories. As of 2019, GitHub Actions now supports full CI/CD pipelines.

Integrating k6 performance tests into a new or existing GitHub Actions pipeline is quick and easy, especially using the official marketplace app. Grafana k6 has two official GitHub actions available in the GitHub marketplace to make it easy to run performance tests in a GitHub action workflow. The first is setup-k6-action, which is used to configure k6 in the workflow pipeline, and the second is run-k6-action, which is used to execute the k6 tests.

If you haven’t used GitHub Actions before, we recommend looking at the following links to get a sense of how it works:

• Workflow syntax for GitHub Actions
• Events that trigger workflows

Writing your performance test

We’ll start small by writing a simple test that measures the performance of a single endpoint. As with most, if not all, development efforts, performance testing yields the best results if we work in small increments, iterating and expanding as our knowledge increases.

Our test will consist of three parts:

1. An HTTP request against our system under test (SUT).
2. A load configuration controlling the test duration and amount of virtual users.
3. A performance goal, or service level objective (SLO), defined as a threshold.

Creating the test script

When we execute our test script, each virtual user will execute the default function as many times as possible until the duration is up. To make sure we don’t flood our system under test, we’ll make the virtual user sleep for a second before it continues.

JavaScript Copy

import http from 'k6/http';
import { sleep } from 'k6';

export default function () {
  const res = http.get('https://quickpizza.grafana.com/');
  sleep(1);
}

Configuring the load

We’ll configure our test to run 50 virtual users continuously for one minute. Because of the sleep we added earlier, this will result in just below 50 iterations per second, giving us a total of about 2900 iterations.

JavaScript Copy

import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  duration: '1m',
  vus: 50,
};

export default function () {
  const res = http.get('https://quickpizza.grafana.com/');
  sleep(1);
}

If you have installed k6 on your local machine, you can run your test locally in your terminal using the command k6 run test.js.

Configuring our thresholds

The next step in this load testing example is to define your service level objectives, or SLOs, around your application performance. SLOs are vital to ensure the reliability of your systems and applications. If you do not currently have any defined SLAs or SLOs, now is an excellent time to consider your requirements.

You can define SLOs as Pass/Fail criteria with thresholds in your k6 script. k6 evaluates them during the test execution and informs about the threshold results. If any of the thresholds in our test fails, k6 will return with a non-zero exit code, communicating to the CI tool that the step has failed.

Now, we will add one threshold to our previous script to validate that the 95th percentile response time must be below 500ms and also that our error rate is less than 1%. After this change, the script will be as in the snippet below.

JavaScript Copy

import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  duration: '1m',
  vus: 50,
  thresholds: {
    http_req_failed: ['rate<0.01'], // http errors should be less than 1%
    http_req_duration: ['p(95)<500'], // 95 percent of response times must be below 500ms
  },
};

export default function () {
  const res = http.get('https://quickpizza.grafana.com/');
  sleep(1);
}

Thresholds are a powerful feature providing a flexible API to define various types of Pass/Fail criteria in the same test run. For example:

• The 99th percentile response time must be below 700 ms.
• The 95th percentile response time must be below 400 ms.
• No more than 1% failed requests.
• The content of a response must be correct more than 95% of the time.

Check out the thresholds documentation for additional details on the API and its usage.

Setting up the GitHub Actions workflow

To have GitHub Actions pick up and execute our load test, we need to create a workflow configuration and place it in .github/workflows. Once this file has been pushed to our repository, each commit to our repository will result in the workflow being run.

Copy

name: k6 Load Test

on:
  push:
    branches:
      - '**'

jobs:
  run-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup K6
        uses: grafana/setup-k6-action@v1
      - name: Run local k6 test
        uses: grafana/run-k6-action@v1
        with:
          path: test.js

k6 has official GitHub actions to make it easy to execute your load tests. They are divided into two parts:

1. Setup k6 action: This action allows you to install and configure k6 in the pipeline easily. You can use it to customize what version of k6 you want or optionally configure a browser to easily run k6 browser tests.

You can use the following options with this action to modify its behavior:

• k6-version: Specify the k6 version to use – for example, ‘0.49.0’. If not set, the latest k6 version will be used.
• browser: If set to true, a chromium browser is set up along with k6, allowing you to run k6 browser tests. By default, it is set to false.

2. Run k6 action: This action allows you to execute k6 tests with ease. You can use it to execute multiple test scripts in parallel, run cloud test runs, automatically add a comment on a PR with URLs to test run results, and more.

The following action inputs can be used along with the action:

• path: Glob pattern to select one or multiple test scripts to run.
• cloud-run-locally: If true, the tests are executed locally and the results are uploaded to Grafana Cloud k6, our fully managed performance testing platform powered by Grafana k6. By default, it is true.
• parallel: If true, and multiple tests are executed, all of them run in parallel.
• fail-fast: If true, the whole pipeline fails as soon as the first test fails.
• flags: Additional flags to be passed on to the k6 run command, e.g., --vus 10 --duration 20s
• cloud-comment-on-pr: If true, the workflow comments a link to the cloud test run on the pull request (if present). By default, it is true.
• only-verify-scripts: If true, only check if the test scripts are valid and skip the test execution.

Running cloud tests

There are two common execution modes to run k6 tests as part of the CI process.

• Locally on the CI server.
• In Grafana Cloud k6, from one or multiple geographic locations.

You can also run k6 tests locally on the CI server, and then push the results to Grafana Cloud k6.

⚠️ Try it locally first

Before we start with the configuration, it is good to familiarize ourselves with how cloud execution works. We recommend you test how to trigger a cloud test from your machine.

Check out our cloud execution guide to learn how to distribute the test load across multiple geographic locations and for more information about the cloud execution.

Now, we will show how to run cloud tests using GitHub Actions. If you do not have an account with Grafana Cloud already, you can sign up for a free one today.

After that, get your account token and add it to your GitHub project’s Secrets page by going to Settings > Security > Secrets and Variables > Actions > Secrets.

We also need to specify the project ID where the test run should be stored. To do this, grab your project ID and store it as a GitHub action secret or define it directly in the workflow.

We will pass the token and project ID to the k6 action via environment variables.

Copy

...
      - name: Run local k6 test
        uses: grafana/run-k6-action@v1
        env:
          K6_CLOUD_TOKEN: ${{ secrets.K6_CLOUD_TOKEN }}
          K6_CLOUD_PROJECT_ID: ${{ secrets.K6_CLOUD_PROJECT_ID }}
        with:
          path: test.js

...

Running a test locally and pushing results to Grafana Cloud k6

Once you add the token and project ID environment variables to the GitHub action, this is the default execution mode. Here, the test is executed locally in your pipeline, allowing you to test your system without deploying it anywhere, and the results are pushed to Grafana Cloud k6.

The following is a sample workflow executing this kind of test.

Copy

name: k6 Cloud Load Test

on:
  push:
    branches:
      - '**'

jobs:
  run-cloud-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup K6
        uses: grafana/setup-k6-action@v1
      - name: Run local k6 test
        uses: grafana/run-k6-action@v1
        env:
          K6_CLOUD_TOKEN: ${{ secrets.K6_CLOUD_TOKEN }}
          K6_CLOUD_PROJECT_ID: ${{ secrets.K6_CLOUD_PROJECT_ID }}
        with:
          path: test.js

Once we commit and push these changes, k6 will run the cloud test, and output the URL to our test results as part of the workflow logs.

And if we copy the highlighted URL and navigate to it in a new tab:

Running a test on Grafana Cloud k6

You might want to use cloud tests in these common cases:

• You’re going to run a test from multiple geographic locations (load zones).
• You’re going to run a high-load test that will need more compute resources than available in the runner.

The only change in the workflow compared to running test locally and pushing results to the cloud is to set cloud-run-locally for the action to false. This will now execute the test on Grafana Cloud k6.

Copy

name: k6 Cloud Load Test

on:
  push:
    branches:
      - '**'

jobs:
  run-cloud-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup K6
        uses: grafana/setup-k6-action@v1
      - name: Run local k6 test
        uses: grafana/run-k6-action@v1
        env:
          K6_CLOUD_TOKEN: ${{ secrets.K6_CLOUD_TOKEN }}
          K6_CLOUD_PROJECT_ID: ${{ secrets.K6_CLOUD_PROJECT_ID }}
        with:
          path: test.js
          cloud-run-locally: false

Comment cloud test run URLs on the PR

In both the cloud execution modes discussed above, if the commit is associated with an open pull request, the action will automatically try to add a comment on the pull request with the cloud test run URLs.

To disable this behavior, set the action input cloud-comment-on-pr to false.

If the comment is not being created for you, make sure the GitHub action has write permissions on the repository. Follow the instructions from the official GitHub docs to update the permissions for the GITHUB_TOKEN to have read and write permissions.

Running k6 extensions

k6 extensions allow users to extend the usage of k6 to cover use cases that are not natively supported. With extensions, users can test new protocols, build clients that communicate with other systems during tests, and improve test performance by writing it in Go and consuming it from tests written in JavaScript. k6 extensions can be imported as JavaScript modules and used in the testing script.

As an example, we’ll use xk6-counter to execute the following test.

JavaScript Copy

import counter from 'k6/x/counter';

export const options = {
  vus: 10,
  duration: '5s',
};

export default function () {
  console.log(counter.up(), __VU, __ITER);
}

The standard k6 executable won’t be able to import the k6/x/counter module. On your local machine, you can run this test by using a custom k6 executable built with the k6/x/counter extension.

Copy

# Install xk6
go install go.k6.io/xk6/cmd/xk6@latest
# Build binary
xk6 build --with github.com/mstoykov/xk6-counter@latest
# Run test using the compiled k6 binary
./k6 run test.js

To achieve the same result on GitHub, simply set up the following workflow.

Copy

name: k6 Cloud Load Test with extension

on:
  push:
    branches:
      - '**'

jobs:
  run-extension-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup Go
        uses: actions/setup-go@v5

      - name: Install xk6
        run: go install go.k6.io/xk6/cmd/xk6@latest

      - name: Build xk6-counter binary
        run: xk6 build --with github.com/mstoykov/xk6-counter@latest

      - name: Run k6 extension test
        runs: ./k6 run extension/script.js

Since the Go development environment is required, we are using the setup-go action to provide a suitable environment for compiling our extension. The install, build, and run commands are exactly the same as those used in the local machine.

Variations

Using a different runner

GitHub provides Windows and macOS environments to run your workflows. You can also set up custom runners that operate on your premises or your cloud infrastructure.

The workflow we used above is based on the official k6 action, provided through the GitHub Marketplace. This action, however, currently only runs on Linux. To be able to run it on a Windows or macOS runner, we’ll have to install k6 as part of our pipeline.

To use a Windows runner:

Copy

name: k6 Load Test - windows runner

on:
  push:
    branches:
      - '**'

jobs:
  k6_local_test:
    name: k6 local test run on windows
    runs-on: windows-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Install k6
        run: choco install k6
        shell: bash

      - name: Run k6 test
        run: k6.exe run ./test.js
        shell: bash

Here are the most up-to-date instructions for k6 Windows installation.

And to use a macOS runner:

Copy

name: k6 Load Test - macos runner

on:
  push:
    branches:
      - '**'
jobs:
  k6_local_test:
    name: k6 local test on macos
    runs-on: macos-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Install k6 by homebrew
        run: brew install k6

      - name: Run k6 test
        run: k6 run ./test.js

The brew package manager is the best tool for grabbing and installing the latest version of k6 whenever the workflow is run.

Nightly builds

Triggering a subset of performance tests at a specific time is a best practice to automate performance testing.

It’s common to run some performance tests during the night when users do not access the system under test. This can be helpful to isolate more extensive tests from other types of testing, or to generate a performance report periodically.

To configure a scheduled nightly build that runs at a given time, head over to your GitHub action workflow and update the on section. Here is an example that triggers the workflow every 15 minutes.

Copy

on:
  schedule:
    # * is a special character in YAML, so you have to quote this string
    - cron: '*/15 * * * *'

You’ll have to use POSIX cron syntax to schedule a workflow to run at specific UTC times. Here is an interactive tool for creating crontab scheduling expressions.

Simply save, commit, and push the file. GitHub will take care of running the workflow at the time intervals you specified.

Using the Docker image

Using the k6 Docker image directly is almost as easy as the marketplace app. The example below uses the cloud service, but you could just as easily use it for local execution as well.

Copy

name: k6 Load Test - docker image

on:
  push:
    branches:
      - '**'

jobs:
  run-test:
    runs-on: ubuntu-latest
    container:
      image: grafana/k6:latest
      options: --user root

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Run k6 test
        uses: grafana/run-k6-action@v1
        env:
          K6_CLOUD_TOKEN: ${{ secrets.K6_CLOUD_TOKEN }}
          K6_CLOUD_PROJECT_ID: ${{ secrets.K6_CLOUD_PROJECT_ID }}
        with:
          path: test.js

Learn more

To learn more about Grafana k6, you can check our technical documentation, as well as our GitHub repository. And if you need any help, or want additional info, please feel free to reach out on our community forum. Happy testing!

Grafana Cloud is the easiest way to get started with Grafana k6 and performance testing. We have a generous forever-free tier and plans for every use case. Sign up for free now!