Documentation Grafana Enterprise Traces Configure Gateway

Gateway

The Grafana Enterprise Traces (GET) gateway is a service target. It can proxy requests to other GET microservices. You can also use it for client-side load balancing of requests proxied to the distributors.

GET also offers a federation-frontend service. This service can also perform cross-tenant and cross-cluster query federation by aggregating the data from multiple GET cluster and from multiple tenants in a single trace lookup or search query. It works by reverse proxying the read path of GET clusters. To learn more, refer to Federation-frontend.

You can configure the gateway using a code block in the GET YAML configuration file or by using command-line flags.

Configure using the GET configuration file

The gateway has its own configuration block in the GET configuration files. For more information about the configuration file, refer to the configuration reference.

Here is an example configuration block:

yaml Copy

gateway:
  proxy:
    default: <backend_proxy_config>
    [ admin_api: <backend_proxy_config> ]
    [ compactor: <backend_proxy_config> ]
    [ distributor: <backend_proxy_config> ]
    [ ingester: <backend_proxy_config> ]
    [ querier: <backend_proxy_config> ]
    [ query_frontend: <backend_proxy_config> ]

Configure using command-line flags

To configure the gateway via command-line flags, make sure that each flag contains the path to the equivalent configuration field.

To configure the distributor backend proxy URL, use the flag -gateway.proxy.distributor.url=<distributor URL>.

Likewise, given a configuration file field path of gateway.proxy.query_frontend.url, the flag is gateway.proxy.query-frontend.url.

The backend_proxy_config section

A backend_proxy section specifies the URL and configuration of the backend that you want proxied. A URL is only required for the default proxy configuration. All other proxy configurations use the default URL if it’s not explicitly configured. Other proxy configurations don’t inherit any other configuration from the default proxy configuration.

yaml Copy

url: <url> | default = <gateway.proxy.default.url>
[ enable_keepalive: <boolean> | default = true ]
[ tls_enabled: <boolean> | default = false ]
[ tls_ca_path: <string> | default = "" ]
[ tls_server_name: <string> | default = "" ]
[ tls_cert_path: <string> | default = "" ]
[ tls_key_path: <string> | default = "" ]
[ tls_insecure_skip_verify: <boolean> | default = false ]
[ read_timeout: <duration> | default = "120s" ]
[ write_timeout: <duration> | default = "30s" ]

Client-side load balancing

If you use a backend proxy URL that beings with dns:///, the gateway creates a gRPC proxy with client-side round-robin load balancing instead of the default HTTP reverse proxy.

To configure client-side load balancing for requests to the distributors only, set the gateway.proxy.distributor.url to dns:///<distributor service>.

Client-side load balancing is useful in ensuring that distributors are evenly loaded with requests.

Kubernetes Services aren’t load balancers; initial TCP connections are made using a random endpoint. After establishing the connection, the same remote-write client talks to the same distributor server for its lifetime. This can mean an uneven load for your distributors and worse cluster performance overall.

The GET gateway solves this problem by exposing an HTTP server for receiving the client requests while using gRPC to talk to the distributors. The gRPC proxy maintains a list of endpoints that are returned from the DNS lookup, and keeps persistent connections to each endpoint. The proxies are also configured to perform per-request, client-side load balancing across the endpoints.

Ring endpoints

The Gateway exposes the ring endpoints for distributors, ingesters, and compactors. These endpoints require admin scope permissions.

• /distributor/ring for the distributors
• /ingester/ring for the ingesters
• /compactor/ring for the compactors