Documentation Grafana Cloud Instrument and send data Fleet Management Onboard existing collectors

Onboard your collectors to Fleet Management

Grafana Fleet Management is a Grafana Cloud feature that centralizes monitoring, configuration, and management of all your collectors. Learn how to onboard your existing collectors to Fleet Management so you can proactively manage your observability setup.

Before you begin

To onboard collectors to Fleet Management, you must:

• Have a Grafana Cloud account.
• Have administrator privileges and the ability to create and edit access policies.
• Update Grafana Alloy to the most current version. Fleet Management must be used with Alloy collectors.
• Understand the key terms and concepts of Fleet Management and how the service works.

Create an access policy

To begin the onboarding process, you must authorize Fleet Management to access your collectors.

1. In your Grafana Cloud portal, select the stack where you want to configure Fleet Management. You can only query collectors within the same stack.
2. In the left-side navigation, select Access Policies under the Security heading.
3. Create a new access policy or edit an existing access policy. Make sure to add the scope fleet-management:read to the policy. We recommend that you also enable metrics:write and logs:write on the same policy so that a single token can be used. For more information on creating access policies, refer to Grafana Cloud access policies.
4. After setting up the access policy, click Add token and follow the prompts to add a new token to the policy. Make sure to copy the token to your clipboard and save it in a secure password file. You’ll need it for the next step, but you won’t be able to view it again once you close the confirmation window.

Add remotecfg to local configurations

Enroll your collectors in Fleet Management by adding the remotecfg code block to their local configuration files. Fleet Management does not support use of the remotecfg block in configuration modules. The block must be added to the root configuration. For help, refer to the Alloy configuration documentation.

Here is an example of a remotecfg code block:

alloy Copy

remotecfg {
    url = "<SERVICE_URL>"
    basic_auth {
        username      = "<USERNAME>"
        password      = "<PASSWORD>"
    }

    id             = constants.hostname
    attributes     = {"cluster" = "dev", "namespace" = "otlp-dev"}
    poll_frequency = "5m"
}

None of the arguments in this remotecfg are required.

• url: Set this value to the base URL of the Fleet Management service. If url is unset, the service is non-operational.

  You can find the base URL in the Grafana Cloud Fleet Management interface. In your Grafana Cloud stack, click Connections > Collector > Fleet Management in the left-side menu and switch to the API tab. Find the remotecfg URL in the Base URL section. It looks like the following URL, where <CLUSTER_NAME> is the production cluster of your stack:

Copy

https://fleet-management-<CLUSTER_NAME>.grafana.net/

• basic_auth: Use this block to authenticate to the service.
  • The username is the same as your instance ID.
  • The password is the token you added to your access policy in the previous section. If you prefer to save your token in a secure file, use password_file = "<PASSWORD_FILE_PATH>" instead.
• id: Set this value to a unique identifier for the collector. If the collector is deployed with persistent storage, id is reused across restarts.
  • If unset, id defaults to a universally unique identifier (UUID) that is stored on the collector’s data path. By default this path is data-alloy/alloy_seed.json.
  • If overridden, id must be set to some uniquely identifiable value, such as a unique hostname or a ${cluster}_${namespace}_${podname} value. If the value of id is not unique, collectors might be assigned different configurations than expected and be indistinguishable in the Fleet Management interface.
• attributes: Use this argument to assign a set of key-value pairs that meaningfully characterize the collector.
  • attributes defined in remotecfg are system attributes. Fleet Management automatically includes collector.os and collector.version as system attributes, which cannot be overwritten.
  • Use the Fleet Management interface to add custom attributes. You can assign up to 100 custom attributes per collector, but only 15 attributes can be used in a GetConfig API request. Custom attributes take precedence over system attributes.
  • Both attribute sets are merged together and used for configuration assignment and filtering.
• poll_frequency: Use this argument to define how often to poll the API for new configurations.

For the full list of supported arguments, refer to the remotecfg reference documentation.

Kubernetes

To register a collector running in Kubernetes, modify your Helm chart according to the following guidelines.

Grafana Alloy Helm chart

Onboard and configure your collector using the Alloy Helm chart.

1. Create a secret you can use as an access token for Fleet Management.

   shell Copy

   kubectl create secret --namespace NAMESPACE generic gc-token --from-literal=token=VALUE

2. Create a ConfigMap from a file and add the following remotecfg block to the configuration.

   alloy Copy

   remotecfg {
           url            = "<URL>"
           id             = sys.env("GCLOUD_FM_COLLECTOR_ID")
   
           attributes = { "platform" = "kubernetes" }
   
           basic_auth {
                   username = "<USERNAME>"
                   password = sys.env("GCLOUD_RW_API_KEY")
           }
   }

   Replace <URL> with the base URL of your production cluster and <USERNAME> with your instance ID, both of which can be found on the API tab in the Fleet Management interface.

3. Update your Helm chart as follows.

   yaml Copy

   gc:
     secret:
       name: gc-token
   
   alloy:
     configMap:
       create: false
       name: alloy-config
       key: config.alloy
     stabilityLevel: "public-preview"
     extraEnv:
       - name: NAMESPACE
         valueFrom:
           fieldRef:
             fieldPath: metadata.namespace
       - name: POD_NAME
         valueFrom:
           fieldRef:
             fieldPath: metadata.name
       - name: GCLOUD_FM_COLLECTOR_ID
         value: "clusterName-$(NAMESPACE)-$(POD_NAME)"
       - name: GCLOUD_RW_API_KEY
         valueFrom:
           secretKeyRef:
             name: gc-token
             key: token

   In addition to specifying the Alloy configuration file, this chart section also sets:

   • The stabilityLevel to public-preview. Alloy requires by default that its components be generally available. Since the remotecfg block is in public preview, the stabilityLevel must be set to public-preview.
   • The environment variables GCLOUD_FM_COLLECTOR_ID and GCLOUD_RW_API_KEY. These variables must be set so that the self-monitoring configuration pipelines are properly assigned.
     • GCLOUD_FM_COLLECTOR_ID is set to the pod name. The variable is reset each time the pod gets a new name, which means a new collector appears in the Inventory tab. You can delete old, unused collectors from your inventory.
     • GCLOUD_RW_API_KEY is set to the secret you created in step 1.

Grafana Kubernetes Monitoring Helm chart

From v2.0.0 of the Kubernetes Monitoring Helm chart, support for Fleet Management is built in. You can enable Fleet Management by setting remoteConfig to true:

yaml Copy

cluster:
  name: remote-config-example-cluster

alloy-metrics:
  enabled: true
  alloy:
    stabilityLevel: public-preview
  remoteConfig:
    enabled: true
    url: "<URL>"
    auth:
      type: "basic"
      username: "<USERNAME>"
      password: "<PASSWORD>"

Replace <URL> with the base URL of your production cluster and <USERNAME> with your instance ID, both of which can be found on the API tab in the Fleet Management interface. The password value is the token assigned to your Fleet Management access policy. Refer to Create an access policy for more details.

Fleet Management is not recommended for use with earlier versions of the Kubernetes Monitoring Helm chart.

Self-monitoring configuration pipelines

Fleet Management automatically creates and assigns self-monitoring configuration pipelines when a collector is registered with the service. The internal telemetry collected by the self-monitoring pipelines powers the health dashboards and logs in the collector’s details view in the Fleet Management interface. These pipelines, which begin with self_monitoring_*, rely on environment variables to authenticate requests and set collector_id labels that match telemetry to collectors. In order to see a collector’s health details in Fleet Management, make sure the environment variables GCLOUD_RW_API_KEY and GCLOUD_FM_COLLECTOR_ID are set wherever the collector is running.

Although development of self-monitoring configuration pipelines is ongoing, the logs pipeline does not work out of the box with Kubernetes deployments.

Run your collectors

Follow the instructions to start running your Alloy collectors. If your collectors are already running, restart them so the new configuration takes effect. Check that you’re running the latest version of Alloy.

Test that the remotecfg block is working by pinging the collector’s root /metrics endpoint and looking for remotecfg_* metrics, such as remotecfg_load_attempts_total.

Add custom attributes

Collectors are configured remotely in Fleet Management when their attributes match the matching attributes of a configuration pipeline. If you haven’t specified system attributes in the remotecfg blocks of your collectors, or if additional attributes are needed, you can add custom attributes directly through the Fleet Management interface. You have greater control over configurations when your collectors are well categorized with meaningful attributes. Here’s how to add custom attributes to your collectors:

1. In your Grafana Cloud instance, click Connections > Collector > Fleet Management.
2. From the Inventory tab, select a group of collectors.
3. Click the Edit icon. The bulk edit tool appears on screen.
4. Click Add custom attributes.
5. Follow the instructions to add custom attributes to the selected group of collectors.

Check the Custom Attributes column on the Inventory tab to see that your new attribute is assigned to the collectors you chose. Continue assigning custom attributes based on your configuration needs.

Next steps

You have registered your collectors with Fleet Management and assigned attributes to them. Learn how to configure them remotely.