Enable multi-cluster support (optional)
The Kube-Prometheus Helm chart offers support for some multi-cluster use cases. You may need to adjust some recording and alerting rules for your specific architecture. These adjustments go beyond the scope of these instructions.
Add a cluster
label to rule aggregations
If you want to modify recording and alerting rules to aggregate metrics over a cluster
label, you can use cortex-tools’ rules prepare
command to add a cluster
label to a set of recording and alerting rules. You can modify the aggregation label using the -l
flag. The default label is cluster
. If you use this command, ensure that you have set the corresponding aggregation label using external_labels
in your Prometheus configuration. (If you completed the steps in Migrate a Kube-Prometheus Helm stack, you already set this to cluster: test
).
Extract the rules from the Prometheus container by following the instructions in Import recording and alerting rules.
Run the following command in the directory containing the rules files:
mimirtool rules prepare -i *.yaml
The
-i
flag modifies the rules files in place. You can omit this flag to test the command. This command modifies many rules to keep or add acluster
label to aggregations, which enables multi-cluster support.
You must then load the rules into Grafana Cloud Alerting by following the instructions in Import recording and alerting rules.