How to deploy promtail on Kubernetes |・∀・ (2024)

authentik¹ is an open-source Identity Provider, focused on flexibility and versatility. With authentik, site administrators, application developers, and security engineers have a dependable and secure solution for authentication in almost any type of environment.

There are robust recovery actions available for the users and applications, including user profile and password management. You can quickly edit, deactivate, or even impersonate a user profile, and set a new password for new users or reset an existing password.

You can use authentik in an existing environment to add support for new protocols, so introducing authentik to your current tech stack doesn't present re-architecting challenges. We already support all of the major providers, such as OAuth2, SAML, LDAP , and SCIM, so you can pick the protocol that you need for each application.

See a comparison with other IDPs here.

Promtail requirements

Preparation

Promtail Namespace

We need a namespace to deploy our HelmRelease and associated YAMLs into. Per the flux design, I create this example yaml in my flux repo at /bootstrap/namespaces/namespace-promtail.yaml:

apiVersion: v1kind: Namespacemetadata: name: promtail

Promtail HelmRepository

We're going to install the Promtail helm chart from the grafana repository, so I create the following in my flux repo (assuming it doesn't already exist):

apiVersion: source.toolkit.fluxcd.io/v1beta1kind: HelmRepositorymetadata: name: grafana namespace: flux-systemspec: interval: 15m url: https://grafana.github.io/helm-charts

Promtail Kustomization

Now that the "global" elements of this deployment (just the HelmRepository in this case) have been defined, we do some "flux-ception", and go one layer deeper, adding another Kustomization, telling flux to deploy any YAMLs found in the repo at /promtail/. I create this example Kustomization in my flux repo:

apiVersion: kustomize.toolkit.fluxcd.io/v1beta2kind: Kustomizationmetadata: name: promtail namespace: flux-systemspec: interval: 30m path: ./promtail prune: true # remove any elements later removed from the above path timeout: 10m # if not set, this defaults to interval duration, which is 1h sourceRef: kind: GitRepository name: flux-system healthChecks: - apiVersion: helm.toolkit.fluxcd.io/v2beta1 kind: HelmRelease name: promtail namespace: promtail

Promtail DNSEndpoint

If, like me, you prefer to create your DNS records the "GitOps way" using ExternalDNS, create something like the following example to create a DNS entry for your Authentik ingress:

apiVersion: externaldns.k8s.io/v1alpha1kind: DNSEndpointmetadata: name: "promtail.example.com" namespace: promtailspec: endpoints: - dnsName: "promtail.example.com" recordTTL: 180 recordType: CNAME targets: - "traefik-ingress.example.com" 

Promtail HelmRelease

Lastly, having set the scene above, we define the HelmRelease which will actually deploy promtail into the cluster. We start with a basic HelmRelease YAML, like this example:

apiVersion: helm.toolkit.fluxcd.io/v2beta1kind: HelmReleasemetadata: name: promtail namespace: promtailspec: chart: spec: chart: grafana version: 5.36.x # auto-update to semver bugfixes only (1) sourceRef: kind: HelmRepository name: grafana namespace: flux-system interval: 15m timeout: 5m releaseName: grafana values: # paste contents of upstream values.yaml below, indented 4 spaces (2)

1. I like to set this to the semver minor version of the Promtail current helm chart, so that I'll inherit bug fixes but not any new features (since I'll need to manually update my values to accommodate new releases anyway)
2. Paste the full contents of the upstream values.yaml here, indented 4 spaces under the values: key

If we deploy this helmrelease as-is, we'll inherit every default from the upstream Promtail helm chart. That's probably hardly ever what we want to do, so my preference is to take the entire contents of the Promtail helm chart's values.yaml, and to paste these (indented), under the values key. This means that I can then make my own changes in the context of the entire values.yaml, rather than cherry-picking just the items I want to change, to make future chart upgrades simpler.

Then work your way through the values you pasted, and change any which are specific to your configuration.

Configure authentik Helm Chart

The following sections detail suggested changes to the values pasted into /promtail/helmrelease-promtail.yaml from the Promtail helm chart's values.yaml. The values are already indented correctly to be copied, pasted into the HelmRelease, and adjusted as necessary.

Set authentik secret key

Authentik needs a secret key for signing cookies (not singing for cookies! ), so set it below, and don't change it later (or feed it after midnight!):

 authentik: # -- Log level for server and worker log_level: info # -- Secret key used for cookie singing and unique user IDs, # don't change this after the first install secret_key: "ilovesingingcookies"

Set bootstrap credentials

By default, when you install the authentik helm chart, you'll get to set your admin user's (akadmin) when you first login. You can pre-configure this password by setting the AUTHENTIK_BOOTSTRAP_PASSWORD env var as illustrated below.

If you're after a more hands-off implementation, you can also pre-set a "bootstrap token", which can be used to interact with the authentik API programatically (see example below):

 env: AUTHENTIK_BOOTSTRAP_PASSWORD: "iamusedbyhumanz" AUTHENTIK_BOOTSTRAP_TOKEN: "iamusedbymachinez"

Configure Redis for authentik

authentik uses Redis as the broker for Celery background tasks. The authentik helm chart defaults to provisioning an 8Gi PVC for redis, which seems like overkill for a simple broker. You can tweak the size of the Redis PVC by setting:

 redis: master: persistence: size: 1Gi

Configure PostgreSQL for authentik

Although technically you can leave the PostgreSQL password blank, authentik-server will just error with an error like fe_sendauth: no password supplied, so ensure you set the password, both in authentik.postgresql.password and in postgresql.postgresqlPassword:

At the very least, you'll want to set the following

 authentik: postgresql: password: "Iamaverysecretpassword" postgresql: postgresqlPassword: "Iamaverysecretpassword"

As with Redis above, you may feel (like I do) that provisioning an 8Gi PVC for a database containing 1 user and a handful of app configs is overkill. You can adjust the size of the PostgreSQL PVC by setting:

 postgresql: persistence: size: 1Gi 

Ingress

Setup your ingress for the authentik UI. If you plan to add outposts to proxy other un-authenticated endpoints later, this is where you'll add them:

 ingress: enabled: true ingressClassName: "nginx" # (1)! annotations: {} labels: {} hosts: - host: authentik.example.com paths: - path: "/" pathType: Prefix tls: []

1. Either leave blank to accept the default ingressClassName, or set to whichever ingress controller you want to use.

Install Promtail!

Commit the changes to your flux repository, and either wait for the reconciliation interval, or force a reconcilliation using flux reconcile source git flux-system. You should see the kustomization appear...

~ ❯ flux get kustomizations promtailNAME READY MESSAGE REVISION SUSPENDEDpromtail True Applied revision: main/70da637 main/70da637 False~ ❯

The helmrelease should be reconciled...

~ ❯ flux get helmreleases -n promtail promtailNAME READY MESSAGE REVISION SUSPENDEDpromtail True Release reconciliation succeeded v5.36.x False~ ❯

And you should have happy pods in the promtail namespace:

~ ❯ k get pods -n authentikNAME READY STATUS RESTARTS AGEauthentik-redis-master-0 1/1 Running 1 (3d17h ago) 26dauthentik-server-548c6d4d5f-ljqft 1/1 Running 1 (3d17h ago) 20dauthentik-postgresql-0 1/1 Running 1 (3d17h ago) 26dauthentik-worker-7bb8f55bcb-5jwrr 1/1 Running 0 23h~ ❯

Browse to the URL you configured in your ingress above, and confirm that the authentik UI is displayed.

Create your admin user

You may be a little confused re how to login for the first time. If you didn't use a bootstrap password as above, you'll want to go to https://<ingress-host-name>/if/flow/initial-setup/, and set an initial password for your akadmin user.

Now store the akadmin password somewhere safely, and proceed to create your own user account (you'll presumably want to use your own username and email address).

Navigate to Admin Interface → Directory → Users, and create your new user. Edit your user and manually set your password.

Next, navigate to Directory → Groups, and edit the authentik Admins group. Within the group, click the Users tab to add your new user to the authentik Admins group.

Eureka!

Your user is now an authentik superuser. Confirm this by logging out as akadmin, and logging back in with your own credentials.

Summary

What have we achieved? We've got authentik running and accessible, we've created a superuser account, and we're ready to flex the power of authentik to deploy an OIDC provider for Kubernetes, or simply secure unprotected UIs with proxy outposts!

Chef's notes 📓

Tip your waiter (sponsor) 👏

Did you receive excellent service? Want to compliment the chef? (..and support development of current and future recipes!) Sponsor me on Github / Ko-Fi / Patreon, or see the contribute page for more (free or paid) ways to say thank you! 👏

Employ your chef (engage) 🤝

Is this too much of a geeky PITA? Do you just want results, stat? I do this for a living - I'm a full-time Kubernetes contractor, providing consulting and engineering expertise to businesses needing short-term, short-notice support in the cloud-native space, including AWS/Azure/GKE, Kubernetes, CI/CD and automation.

Learn more about working with me here.

Flirt with waiter (subscribe) 💌

Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the RSS feed, or leave your email address below, and we'll keep you updated.

Promtail resources 📝

• Promtail (official site)

• Promtail GitHub Repo