This is documentation for the next version of Loki. For the latest stable release, go to the latest version.
Migrate to TSDB
TSDB is the recommended index type for Loki and is where the current development lies. If you are running Loki with boltb-shipper or any of the legacy index types that have been deprecated, we strongly recommend migrating to TSDB.
Configure TSDB index for an upcoming period
To begin the migration, add a new period_config entry in your schema_config. You can read more about schema config here.
Note
You must roll out the newperiod_config
change to all Loki components in order for it to take effect.
This example adds a new period_config
which configures Loki to start using the TSDB index for the data ingested starting from 2023-10-20
.
schema_config:
configs:
- from: 2023-01-01
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
- from: 2023-10-20 ①
store: tsdb ②
object_store: filesystem ③
schema: v13 ④
index:
prefix: index_
period: 24h
You must set the new period
from
to a date in the future.Update the new period to use TSDB as the index type by setting
store: tsdb
.This sample configuration uses filesystem as the storage in both the periods. If you want to use a different storage for the TSDB index and chunks, you can specify a different
object_store
in the new period.Update the schema to v13 which is the recommended version at the time of writing. Please refer to the configure page for the current recommended version.
Configure TSDB shipper
It’s also important that you configure the tsdb_shipper
block in storage_config. Specifically the following options:
active_index_directory
: directory where ingesters would write index files which will then be uploaded by shipper to configured storage.cache_location
: cache location for downloading index files from the storage for use in query path.
storage_config:
tsdb_shipper:
active_index_directory: /data/tsdb-index
cache_location: /data/tsdb-cache
Run compactor
We strongly recommended running the compactor when using TSDB index. It is responsible for running compaction and retention on TSDB index. Not running index compaction will result in sub-optimal query performance.
Please refer to the compactor section for more information and configuration examples.