Emit contextualized JSON logs with Java

Most popular logging frameworks, such as Log4j or SLF4J/Logback in Java, support emitting JSON formatted logs.

To integrate OpenTelemetry with logging libraries you need to specify the resource attributes available in the log line.

To keep your architecture simple only add attributes you require to filter and correlate logs, for example service.name, service.namespace, deployment.environment, and service.instance.id.

Example application

The following example uses Spring Boot to enrich logs with OpenTelemetry attributes and emit them formatted in JSON through stdout.

Note
You can find the full code of the example in the Docker LGTM repository.

Create a Spring Boot application 3.3.0+ using the default logging instrumentation with the Logback library and the stdout output.

Instrument the application

Instrument the Spring Boot application with the OpenTelemetry Java Agent following the setup flow defined by the Grafana Cloud integration for Java:

Open the Grafana Cloud home page in a web browser.
Navigate to Connections > Add new Connection.
Select Java OpenTelemetry and follow the setup instructions.

Enrich Logback logs with OpenTelemetry resource attributes and log attributes using:

export OTEL_INSTRUMENTATION_COMMON_MDC_RESOURCE_ATTRIBUTES=service.namespace,service.name,service.instance.id,service.version,deployment.environment

Change the Logback SpringBoot configuration to emit JSON logs with OpenTelemetry contextualization and add the following logback-spring.xml under src/main/resources:

<!-- tested with Logback 1.5.0 and Spring Boot 3.3.0 -->
<configuration>
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <include resource="org/springframework/boot/logging/logback/console-appender.xml"/>

    <root level="INFO">
        <appender-ref ref="CONSOLE_JSON"/>
    </root>

    <appender name="CONSOLE_JSON" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="ch.qos.logback.classic.encoder.JsonEncoder">
            <withFormattedMessage>true</withFormattedMessage>
            <withMessage>false</withMessage>
            <withArguments>false</withArguments>
            <withSequenceNumber>false</withSequenceNumber>
            <withNanoseconds>false</withNanoseconds>
        </encoder>
    </appender>
</configuration>

Deploy the application

Deploy the Spring Boot application on Kubernetes and verify that the application outputs logs to the container stdout stream with JSON formatting and OpenTelemetry contextualization, for example kubectl logs my_pod_name | jq outputs:

{
  "timestamp": 1727346005788,
  "level": "INFO",
  "threadName": "http-nio-8080-exec-5",
  "loggerName": "com.grafana.example.RollController",
  "context": {
    "name": "default",
    "birthdate": 1727345887787,
    "properties": {
    }
  },
  "mdc": {
    "trace_id": "97a39974ba2dfc9275e4d31dc2730ee4",
    "trace_flags": "01",
    "service.name": "dice",
    "service.instance.id": "0fb18318-06c0-4893-9a8b-353c00b45227",
    "service.version": "1.1",
    "span_id": "08d6d83d645e3ad2",
    "service.namespace": "shop",
    "deployment.environment": "staging"
  },
  "formattedMessage": "Anonymous player is rolling the dice: 6",
  "throwable": null
}

Configure OpenTelemetry Collector

Configure the OpenTelemetry Collector instance with the file log receiver:

Add a file log receiver.
Add the container parser operator.
Add the json parser operator.
- Map body to attributes.formattedMessage
- Map timestamp.parse_from to attributes.timestamp
- Map severity.parse_from to attributes.level
- Map trace.trace_id.parse_from to attributes.mdc.trace_id
- Map trace.span_id.parse_from to attributes.mdc.span_id
- Map trace.trace_flags.parse_from to attributes.mdc.trace_flags
Add the move operator.
- Move top level fields, such as threadName, to their corresponding OTel attributes, such as thread.name
- Move entries from the mdc field to the resource entry of the log entry
Remove all unnecessary fields using the attributes operator.