Kubernetes on nixOS using k3s (Part 1)

June 4th, 2020

This is outdated! k3s is now packaged in nixpkgs!

k3s is a lightweight kubernetes distribution and works incredibly well. It packages all its dependencies, runs using sqlite in place of etcd (although it can use a number of backends!), and has a much lower memory footprint.

Building the k3s binary from scratch is quite confusing, but since its a static binary, we don’t have to worry about this. We can simply fetch the built binary from github and run it. This is not the most nix way of doing things, but it is the easiest. It allows me to get all the benefits of nixos without the complexity of building everything myself or using a more heavy kubernetes distribution.

Create a folder named k3s, and inside it, create a file named default.nix with the following contents.

{ pkgs ? import <nixpkgs> {} }:
pkgs.stdenv.mkDerivation {
  name = "k3s";
  src = pkgs.fetchurl {
    url = "https://github.com/rancher/k3s/releases/download/v1.18.2-rc3%2Bk3s1/k3s";
    sha256 = "812205e670eaf20cc81b0b6123370edcf84914e16a2452580285367d23525d0f";
  };
  phases = [ "installPhase" ];
  installPhase = ''
    mkdir -p $out/bin
    cp $src $out/bin/k3s
    chmod +x $out/bin/k3s
  '';
}

This file declares a custom nix package. This package can then be imported as an overlay to nixpkgs so that you can use it the way that you would use any other nix package. Add the following stanza to the top of your configuration.nix, but replace the path to your k3s folder.

  nixpkgs.overlays = [
      (self: super: {
          k3s = super.callPackage ../pkgs/k3s {};
        })
  ];

Overlays and packages can be managed in a multitude of ways and it doesn’t really matter what method you use, as long as the package is the same.

Finally, you can add k3s to your system packages in your configuration.nix and you’ll have access to the k3s cli tools.

  environment.systemPackages = with pkgs; [
     k3s
  ];

Part 2 talks about running k3s as a service.