Monitoring Setup Made Simple with Tanka and the Prometheus-Ksonnet Library

As mentioned in a previous post, at Grafana Labs we make heavy use of Tanka and the Jsonnet programming language to manage our Kubernetes infrastructure.

One of the benefits of the use of Jsonnet is the depth of collaboration that it allows with others outside of your company.

For example, the open source prometheus-ksonnet library can be used to install both Prometheus and Grafana. While it’s at it, it also installs the kubernetes-mixin library, which adds a set of best practice dashboards and alerts for monitoring Kubernetes resources. It also makes it easy to add Prometheus configuration and Grafana dashboards specific to your application.

To see how this can be used to programmatically create dashboards, see this example.

In this post, we will explain the steps needed to get prometheus-ksonnet working.

Notes Before You Get Started

We start with the assumption that you have a running Kubernetes cluster and have kubectl installed, and that it is correctly connected and authenticated with your cluster. Below you will need to provide the name of a KubeConfig context into which you can install resources.

If you do not have a cluster available, you could try k3d, which runs a complete (but cut down) Kubernetes cluster within Docker. Once k3d is installed, use this to start a cluster:

Copy

k3d create --name tanka-demo
export KUBECONFIG="$(k3d get-kubeconfig --name tanka-demo)"

After this, use tanka-demo as your KubeConfig context in the following steps.

Step-by-Step Instructions

First, download and install Tanka. Tanka allows us to interact with Kubernetes.

Then download and install Jsonnet Bundler. This allows us to download “bundles” of jsonnet that have been shared via Git/GitHub.

To install prometheus-ksonnet, use these commands, replacing <my-kubernetes-context> with the one you selected above:

Copy

mkdir tanka
cd tanka
tk init
tk env set environments/default --server-from-context <my-kubernetes-context>
jb install https://github.com/grafana/jsonnet-libs/prometheus-ksonnet

Next, edit the file environments/default/main.jsonnet and replace what’s there with this content:

Copy

local prometheus = import 'prometheus-ksonnet/prometheus-ksonnet.libsonnet';

prometheus {
  _config+:: {
    namespace: "default",
    cluster_name: "grafana",
  },
}

Then, run:

Copy

tk apply environments/default

And watch your Grafana/Prometheus/NodeExporter/Dashboards arrive in your cluster (e.g. by using kubectl get pods).

Once the pods are running, you can access these services with this command:

Copy

kubectl port-forward deploy/nginx 8080:80

Then visit http://localhost:8080. You will see a simple page that gives links to both your local Grafana, and your local Prometheus instances.

For More Information

For a demonstration of Grafana and the dashboards that this creates, watch this webinar cohosted by Grafana Labs and Oracle.

Try it out and let us know what you think. And stay tuned for a followup blog post about how you can use this installation to push metrics to Grafana Cloud.