Introducing Tanka, Our Way of Deploying to Kubernetes

YAML sucks! This blog post explains why existing tools hardly ease this pain, and what we at Grafana Labs did about it with our new project, Tanka.

TL;DR

1. YAML and Helm both have their shortcomings, which made them a bad fit for our needs.
2. Jsonnet is an interesting language that solves most of these problems.
3. Grafana Labs created Tanka to use Jsonnet with Kubernetes (and as a replacement for the discontinued ksonnet project).
4. Follow the tutorial to see how it works.
5. Go to https://github.com/grafana/tanka and report back how it worked!

Yet Another Misplaced Language

Actually – YAML is not a bad thing at all. It is one of the best languages for expressing software configuration – such as database connection details, TLS parameters, and the like.

But complex tools like Kubernetes require powerful solutions. YAML however is not one of them, because it is designed for representing data, not generating it:

1. Repetition: If information is required in multiple places (dev and prod environments, etc.), all YAML has to offer is copying and pasting those lines. This can lead to nasty drift between your meant-to-be equal resources.

2. Boilerplate: As you need to express entire API objects by hand, you soon find yourself confronted by a wall of YAML, making it impossible to quickly spot what’s actually important.

3. Static nature: YAML is not a dynamic language; it does not know about abstraction, variables, loops, conditionals, and all of the fancy things we know and love from programming languages that keep our code clean and maintainable.

Templates to the Rescue … Wait What?

The most prominent project aiming to improve the YAML situation is Helm. Its approach is incredibly simple in theory: Instead of copying YAML and changing parameters, your computer itself copies the YAML and fills in the parameters.

Unfortunately, Helm introduces new pain points:

1. String templating: Helm uses YAML documents with special syntax embedded instructing the computer how to insert the parameters. But as YAML has a very fragile syntax (relying on indentation, etc.), this can become a nightmare because the templating engine does not even have a rudimentary knowledge of the YAML syntax, so it could create completely invalid output.

2. Not extensible: Customization of the Chart relies on existing substitution sites. If I wanted to change the storage request of a PersistentVolumeClaim, but the Chart’s author forgot to expose this as a parameter, I must fork the entire Chart.

3. Weak abstraction: Helm does not allow you to build higher level abstraction than substituting strings. Helper utilities such as generating an entire Service based on a Deployment are not possible.

4. Complex: All the user ever sees is a values.yml. The actual resources are very far away (not even on the local file system) and written in YAML messed up by templating statements. Wondering where a Deployment comes from? You must go to GitHub, find the source of the Chart, and open some files to find it.

A New Tool?

A tool capable of the following could be the solution to this issue:

1. Reusability:

• Re-using code from other parts of the project, e.g. references and imports, should be possible.

• It would be cool if common code could be shared (on GitHub) and used from other projects, but only to the degree required.

2. Concise syntax: Humans should not be required to remember API objects meant for computers. Instead, the tool could help out by providing utility functions to create the boilerplate for us.

The Jsonnet Data Templating Language

Luckily, a language called Jsonnet exists and provides most of the functionality we are looking for. Jsonnet is a superset of JSON, so the syntax is close to what we are used to from YAML. Still, the language supports abstraction, arithmetic, conditionals, and much more.

In Jsonnet, creating a Deployment can be as concise as the following:

jsonnet Copy

import "k.libsonnet" +
{
  local deploy = $.apps.v1.deployment,
  local container = $.core.v1.container,
  grafana: deploy.new(
    name="grafana",
    replicas=1,
    containers=[
      container.new("grafana", "grafana/grafana")
    ]
  )
}

For more details, check out the language overview.

The now discontinued ksonnet project pioneered the idea of using Jsonnet for Kubernetes, inspiring us to take this even further.

Introducing Tanka

We are happy to introduce Tanka, our open source project for managing Kubernetes.

Tanka is very similar to ksonnet; however, it is not a fork, but a rewrite from scratch. This allowed us to focus on providing only the important parts: The main workflow (show, diff, apply) is still there, and environments are still first class citizens.

The conceptual overhead of Components, Prototypes, Parts, Packages, and Parameters are gone in favor of plain Jsonnet import and deep-merging. Furthermore, installing libraries is handed over to jsonnet-bundler, while good old kubectl is used for cluster communication.

Highlights

1. Rich abstractions: Jsonnet (and thus Tanka) allows abstraction on multiple levels.

• By using variables and references to them.

• Imports allow spreading code over multiple files.

• Functions can be used for doing additional computation, for example creating a Service for a given Deployment.

2. Environment aware: Configuration in Tanka is tied to a namespace in a specific cluster, to prevent applying your config to the wrong place.

3. Evaluated: The Jsonnet compiler converts Jsonnet to JSON by evaluating every single line until it reaches the end of the file. This guarantees that the output is at least valid JSON, and usually logical errors are caught as well.

4. Human friendly: Tanka provides a special library for creating Kubernetes objects (Deployment, Container, Service, etc.) so you can stop remembering the API and let the computer do it for you. Furthermore we can take care of labels for selectors and much more!

5. Non intrusive: Tanka does not impose any concepts (such as Charts) on you. You could even import your .yaml files from Jsonnet and would be good to go. Remember, in the end Tanka just converts Jsonnet to YAML and pipes it to kubectl.

A Bright Future?

This depends on you! While we have created Tanka to solve our own problems, we believe these are similar to yours, and thus everyone could benefit from the project.

To get your hands dirty, go ahead and follow the tutorial to see how it works in detail.

Tanka is open source on GitHub, and we welcome all contributors. Even if you do not write code, you can try out the tool and report back if you had any issues, so we can improve it.

We are actively working on new features, fixes and other enhancements, including better support for packaging, garbage collection of removed objects, Vault secrets, and many more!

Watch this blog for more posts sharing Tanka tips, tricks, and new features!