Smoke testing: A beginner's guide
Smoke tests are a load testing type that have a minimal load, and they are used to verify that the system works well under minimal load and to gather baseline performance values.
This type of load testing consists of running tests with a few VUs—more than 5 VUs could be considered a mini-load test.
Similarly, the test should execute for a short period, either a low number of iterations or a duration from seconds to a few minutes maximum.
In some testing conversation, smoke tests are also called shakeout tests.
When to run a smoke test
Teams should run smoke tests whenever a test script is created or updated. Smoke testing should also be done whenever the relevant application code is updated.
It’s a good practice to run a smoke test as a first step, with the following goals:
- Verify that your test script doesn’t have errors.
- Verify that your system doesn’t throw any errors (performance- or system-related) when under minimal load.
- Gather baseline performance metrics of your system’s response under minimal load.
- With simple logic, to serve as a synthetic test to monitor the performance and availability of production environments.
Considerations for smoke tests
When you prepare a smoke test, consider the following:
- Each time you create or update a script, run a smoke test.
Because smoke tests verify test scripts, try to run one every time you create or update a script. Avoid running other test types with untested scripts. - Keep throughput small and duration short.
Configure your test script to be executed by a small number of VUs (from 2 to 20) with few iterations or brief durations (30 seconds to 3 minutes). Here is a test script using the open source load testing tool Grafana k6:
import http from 'k6/http';
import { check, sleep } from 'k6';
export const options = {
vus: 3, // Key for Smoke test. Keep it at 2, 3, max 5 VUs
duration: '1m', // This can be shorter or just a few iterations
};
export default () => {
const urlRes = http.get('https://test-api.k6.io');
sleep(1);
// MORE STEPS
// Here you can have more steps or complex script
// Step1
// Step2
// etc.
};The following script is an example smoke test. You can copy it, change the endpoints, and start testing. For more comprehensive test logic, you can find more load testing examples in our k6 documentation. The VU chart of a smoke test should look similar to this:
Smoke test results: analysis
The smoke test initially validates that your script runs without errors. If any script-related errors appear, correct the script before trying any more extensive tests.
On the other hand, if you notice poor performance with these low VU numbers, report it, fix your environment, and try again with a smoke test before any further tests.
Once your smoke test shows zero errors and the performance results seem acceptable, you can proceed to other test types.
Grafana Cloud is the easiest way to get started with Grafana k6 and performance testing. We have a generous forever-free tier and plans for every use case. Sign up for free now!
