Guide to using the new Grafana CLI user identity conflict tool in Grafana 9.3

Note: (Oct. 22, 2024) After two years, we are removing the Grafana CLI. Admins are encouraged to continue to use the CLI for supported versions, up to Grafana 11.4. See the public GitHub issue for more information.

Here at Grafana Labs, one of the things we’re always working on is making Grafana more consistent. Given the increased adoption of Grafana around the world and the number of users and authentication providers we support, we wanted to create better defaults for login and email fields. 

In this post, I’ll be introducing one improvement that you’ll find in Grafana 9.3: a new command in Grafana CLI to manage logins or emails that have case insensitive matches for users, and explaining how to use it.

The backstory

When someone signs up for a Grafana account, they can do it using an email or a login. The fields are case sensitive, which can lead to two or more accounts being created for the same user. Additionally, Grafana allows users to set up an authentication provider, and that provider might return an individual’s sign-up email with an uppercased domain name or some combination of uppercase lowercase letters.

Having several accounts leads to split user permissions, confusion among sign-up flows, and unused “zombie” accounts in your database. Plus, multiple accounts can introduce issues when switching between authentication providers. We refer to these inconsistencies in user uniqueness as a user identity conflict. 

Note: You will not encounter user identity conflicts when using Grafana Cloud or if you are running a Grafana instance with MySQL.

A user identity conflict occurs when there is the same lowercased login or email for two or more users. 

For example, UserA signs up for Grafana with their login “grafana_login” and the same person signs up in Grafana with the same login using one or more capitalized letters “grafana_LOGIN”. 

For example:

id: 1, Login: “grafana_login”, Email: “user@grafana.com”
id: 2, Login: “grafana_LOGIN”, Email: “user@grafana.com”

In the above examples, we would have two user account entries for the same user. This would create a conflict block “conflict: grafana_login@grafana.com”. A conflict block consists of the users of the user-identified conflict. The block is a lowercased email or login that all users within the block have. Each row in the block displays properties to help identify the user to keep.

We decided to develop a tool that would help resolve these inconsistencies.

The Grafana CLI user-manager conflict tool

With the new command in Grafana CLI, you can manage logins or emails that have case insensitive matches for users. The command allows you to check user identity conflicts and resolve them.

This tool will present you with four options: listing conflicts, generating the conflict file, validating the conflict file, and ingesting the resolved conflict file. All commands except ingest-file are safe to run multiple times as they do not change any state within the database.

bash Copy

# lists all the conflicting users
grafana-cli user-manager conflicts list

# creates a conflict patch file to edit
grafana-cli user-manager conflicts generate-file 

# reads edited conflict patch file for validation
grafana-cli user-manager conflicts validate-file <filepath>

# ingests the conflict users file. Can be executed once per file and will change the state of the database. 
grafana-cli user-manager conflicts ingest-file <filepath> 

> Note: This command is irreversible.

A basic workflow for using the CLI

Here is a short description of each of the commands:

1. grafana-cli user-manager conflicts list 

2. grafana-cli user-manager conflicts generate-file 

   • Edit the generated file to set a “+” on the user in each conflict to keep and a “-” for the user to be removed.
   • Note: We do not give any recommended users within conflicts. We sort the users by the id, so you need to look through the generated-file to choose which users to keep and which users should be removed.

3. grafana-cli user-manager conflicts validate-file <file-path>

   • Do this before the ingestion step if you want to be extra safe. 
   • Make a backup of your database before running the ingest-file command.

4. grafana-cli user-manager conflicts ingest-file <file-path>

   • Note: This command is irreversible. This command will delete users’ permissions, authentication tokens, teams, preferences, dashboard access, etc.

5. grafana-cli user-manager conflicts list

   • This allows you to see that conflicts were resolved.

Only two of the commands are required to deal with conflicts. The first one is generate-file, which creates a file that needs to be reviewed for the users to keep and which ones to delete. Once that happens, the second command you would run is ingest-file <filepath> pointing to the reviewed file.

You might need to run this process more than one time, as some conflicts can have multiple ways of interacting with each other. When completed, this should resolve the conflicts. You can check again that you do not have a conflict for the users with the ls command. 

To provide more information on how to handle conflicts, I’ll break down each command line below.

Commands

List conflicts in your database

Here is an example of the conflict command output that shows user conflicts with a login conflict type. The listing presents conflicts with their users shown in order of id number.

Copy

$ ./grafana-cli admin user-manager conflicts list

Showing conflicts

conflict: grafana_login
+ id: 11, email: grafana_login_2@grafana.com, login: GRAFANA_LOGIN, last_seen_at: 2012-11-02T14:10:15Z, auth_module: , conflict_email: , conflict_login: true
- id: 13, email: grafana_login@grafana.com, login: grafana_login, last_seen_at: 2012-11-02T14:10:15Z, auth_module: , conflict_email: , conflict_login: true

Generate-file

This command generates a file that you use to select a user to keep in each conflict block. The file provides a view in the form of a .diff file, much like git commit patch files where one entry for each conflict should be considered. The file can be generated repeatedly as it creates a new filename each time.

Remember to change the “+” or “-” signs in the generated file to tell which users to keep and which should be to be removed.

Note: The generated file sorts users by the id in ascending order, picking the top entry as the user to keep.

Here is an example of a generated file for conflicts that shows user conflicts with a login conflict type.

diff Copy

$ ./grafana-cli user-manager conflicts generate-file

generated file
/var/folders/tf/md89rf0x6l10l39lmd7r3rn00000gn/T/conflicting_user_3116317965.diff

once the file is edited and resolved conflicts, you can either validate or ingest the file

$ cat
/var/folders/tf/md89rf0x6l10l39lmd7r3rn00000gn/T/conflicting_user_3116317965.diff

# Conflicts File
# This file is generated by the grafana-cli command ./grafana-cli admin user-manager conflicts generate-file
#
# Commands:
# +, keep <user> = keep user
# -, delete <user> = delete user
#
# The fields conflict_email and conflict_login
# indicate that we see a conflict in email and/or login with another user.
# Both these fields can be true.
#
# There needs to be exactly one picked user per conflict block.
#
# The lines can be re-ordered.
#
# If you feel like you want to wait with a specific block,
# delete all lines regarding that conflict block.
#
conflict: grafana_login
+ id: 11, email: grafana_login_2@grafana.com, login: GRAFANA_LOGIN, last_seen_at: 2012-11-02T14:10:15Z, auth_module: , conflict_email: , conflict_login: true
- id: 13, email: grafana_login@grafana.com, login: grafana_login, last_seen_at: 2012-11-02T14:10:15Z, auth_module: , conflict_email: , conflict_login: true

Validate-file

This command can be run to validate the file given any changes made to the file. Here is an example of a validation that was successful for the resolved file:

Copy

$ ./grafana-cli admin user-manager conflicts validate-file conflicting_user_3116317965.diff

File validation complete.
File can be used with the `ingest-file`.

Ingest-file

This command is used with an argument pointing to your edited conflict file. The command will:

• Validate the file (it is the same as the validation-file command)
• Keep the users in each conflict block marked with “+”; only one user can be kept per conflict
• Delete the users in each conflict block marked with “-”; there can be more than one
• Show the changes that will be made

Once run, it will also update the entries of the users that you keep to have lowercase email and login.

Reminder: This command is irreversible.

Here is an example of the ingest-file command showing the changes made to the users and which users that will be deleted. Once you have run the command, you should see “conflicts resolved.”

diff Copy

$ ./grafana-cli admin user-manager conflicts ingest-file conflicting_user_3116317965.diff

Changes that will take place

Keep the following user.
conflict: grafana_login@grafana.com
+ id: 13, email: grafana_login_2@grafana.com, login: grafana_login

The following user(s) will be deleted.
- id: 11, email: grafana_login@grafana.com, login: GRAFANA_LOGIN

We encourage users to create a db backup before running this command.
Proceed with operation? [y|n]: y

conflicts resolved.

Future-proofing

Once you have dealt with conflicts in your instance, you can future-proof your Grafana instance by setting the following setting in your configuration file:

Copy

[users]
case_insensitive_login = true

Further reading

We hope you find this new tool helpful and it helps remove any unused accounts and headaches handling case-insensitive logins for your users. If you’re interested in learning more about the technical reasoning behind how and why we built this, check out the original issue in GitHub.

Grafana Cloud is the easiest way to get started with metrics, logs, traces, and dashboards. We have a generous free forever tier and plans for every use case. Sign up for free now!