Grafana Mimir version 2.7 release notes

Grafana Labs is excited to announce version 2.7 of Grafana Mimir.

The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.

Features and enhancements

Store-gateway streaming enabled by default The new default value of 5000 for -blocks-storage.bucket-store.batch-series-size enables store-gateway streaming in the default configuration. This means that series are loaded from object storage in batches rather than buffering them all in memory before returning to the querier. Enabling streaming can reduce memory utilization peaks in the store-gateway.
Store-gateway index header reader no longer uses mmap by default Along with streaming enabled in the store-gateway, this change contributes to more efficient memory usage. See the Important changes section for more details.
Support for keep_firing_for option to ruler configuration This new option determines the amount of time an alert should keep firing while the ruler expression doesn’t return results.
More efficient chunks fetching and caching Enable with the new experimental feature flag -blocks-storage.bucket-store.chunks-cache.fine-grained-chunks-caching-enabled=true. This should reduce CPU, memory utilization, and receive bandwidth of a store-gateway.
Experimental query sharding improvements: A new configuration parameter, -query-frontend.query-sharding-target-series-per-shard, allows query sharding to take into account cardinality of similar requests executed previously when computing the maximum number of shards to use. If you want to try it out, we recommend starting with a value of 2500.
Experimental support for native histogram ingestion: Native histograms can now be ingested. The new per-tenant limit -ingester.native-histograms-ingestion-enabled controls whether native histograms are stored or ignored. The support for querying native histograms is not complete yet and it’s expected to be available in the next release.

Alertmanager improvements

New metrics The following upstream metrics are now exposed:
- cortex_alertmanager_dispatcher_aggregation_groups
- cortex_alertmanager_dispatcher_alert_processing_duration_seconds

Helm chart improvements

The Grafana Mimir and Grafana Enterprise Metrics Helm chart is now released independently. See the Grafana Mimir Helm chart documentation.

Important changes

In Grafana Mimir 2.7, the default vaules of the following configuration options have changed:

-blocks-storage.bucket-store.batch-series-size is now enabled by default with a value of 5000.
-ruler.evaluation-delay-duration has changed from 0 to 1m.

In Grafana Mimir 2.7, the following configuration options are now deprecated:

-blocks-storage.bucket-store.chunks-cache.subrange-size since there’s no benefit to changing the default of 16000
-blocks-storage.bucket-store.consistency-delay has been deprecated and will be removed in Mimir 2.9.
-compactor.consistency-delay has been deprecated and will be removed in Mimir 2.9.
-ingester.ring.readiness-check-ring-health has been deprecated and will be removed in Mimir 2.9.

In Grafana Mimir 2.7, the following options, metrics, and labels have been removed:

Experimental support for ephemeral storage introduced in Mimir 2.6.0 has been removed.
- Following options are no longer available:
  - -blocks-storage.ephemeral-tsdb.*
  - -distributor.ephemeral-series-enabled
  - -distributor.ephemeral-series-matchers
  - -ingester.max-ephemeral-series-per-user
  - -ingester.instance-limits.max-ephemeral-series
- The following metrics have been removed:
  - cortex_ingester_ephemeral_series
  - cortex_ingester_ephemeral_series_created_total
  - cortex_ingester_ephemeral_series_removed_total
  - cortex_ingester_ingested_ephemeral_samples_total
  - cortex_ingester_ingested_ephemeral_samples_failures_total
  - cortex_ingester_memory_ephemeral_users
  - cortex_ingester_queries_ephemeral_total
  - cortex_ingester_queried_ephemeral_samples
  - cortex_ingester_queried_ephemeral_series
- Additionally, querying using the {__mimir_storage__="ephemeral"} selector no longer works. All label values with the ephemeral- prefix within the reason label of the cortex_discarded_samples_total metric are no longer available.
The store-gateway default index header reader no longer uses mmap and the mmap-based index header reader has been removed. The following flags have been changed:
- -blocks-storage.bucket-store.index-header.map-populate-enabled has been removed
- -blocks-storage.bucket-store.index-header.stream-reader-enabled has been removed
- -blocks-storage.bucket-store.index-header.stream-reader-max-idle-file-handles has been renamed to -blocks-storage.bucket-store.index-header.max-idle-file-handles, and the corresponding configuration file option has been renamed from stream_reader_max_idle_file_handles to max_idle_file_handles

Bug fixes

Store-gateway: return Canceled rather than Aborted or Internal error when the calling querier cancels a label names or values request, and return Internal if processing the request fails for another reason. PR 4061
Querier: track canceled requests with status code 499 in the metrics instead of 503 or 422. PR 4099
Ingester: compact out-of-order data during /ingester/flush or when TSDB is idle. PR 4180
Ingester: conversion of global limits max-series-per-user, max-series-per-metric, max-metadata-per-user and max-metadata-per-metric into corresponding local limits now takes into account the number of ingesters in each zone. PR 4238
Ingester: track cortex_ingester_memory_series metric consistently with cortex_ingester_memory_series_created_total and cortex_ingester_memory_series_removed_total. PR 4312
Querier: fixed a bug which was incorrectly matching series with regular expression label matchers with begin/end anchors in the middle of the regular expression. PR 4340
Security: updated the Go version to 1.20.3 to fix CVE-2023-24538. PR 4795
Security: updated the Go version to 1.20.4 to fix CVE-2023-24539, CVE-2023-24540, CVE-2023-29400. PR 4905