Using a custom Docker or containerd version (LEGACY)

    Some system tooling can’t be run on Container Linux via containers and this is especially true for the container runtime itself. As with other special binaries you want to bring to the system you can use an Ignition config that downloads the binaries. Starting from Flatcar version ≥ 3185.0.0 a systemd-sysext images should be used instead of the below.

    For custom Docker/containerd binaries sysext images are the recommended way. However, the Flatcar versions below 3185.0.0 don’t support it yet, and even in case support is there you may find it too complicated to build a sysext image and host it elsewhere. In this case you can directly place the custom binaries to /opt/bin/ as done by the following Butane Config which you can transpile to an Ignition config with butane .

    This replicates the Docker setup as of Flatcar Container Linux 3033.2.3 but under /etc and /opt/bin/, and with additional support for the upstream Containerd socket location. You can modify it to use different socket paths or plugins, or even only ship containerd if you don’t need Docker.

    variant: flatcar
    version: 1.0.0
    systemd:
      units:
        - name: prepare-docker.service
          enabled: true
          contents: |
            [Unit]
            Description=Unpack docker binaries to /opt/bin
            ConditionPathExists=!/opt/bin/docker
            [Service]
            Type=oneshot
            RemainAfterExit=true
            Restart=on-failure
            ExecStartPre=/usr/bin/mkdir -p /opt/bin
            ExecStartPre=/usr/bin/tar -v --extract --file /opt/docker.tgz --directory /opt/ --no-same-owner
            ExecStartPre=/usr/bin/rm /opt/docker.tgz
            ExecStartPre=/usr/bin/sh -c "mv /opt/docker/* /opt/bin/"
            ExecStart=/usr/bin/rmdir /opt/docker
            [Install]
            WantedBy=multi-user.target
        - name: docker.socket
          enabled: true
          contents: |
            [Unit]
            PartOf=docker.service
            Description=Docker Socket for the API
            [Socket]
            ListenStream=/var/run/docker.sock
            SocketMode=0660
            SocketUser=root
            SocketGroup=docker
            [Install]
            WantedBy=sockets.target
        - name: docker.service
          enabled: false
          contents: |
            [Unit]
            Description=Docker Application Container Engine
            After=containerd.service docker.socket network-online.target prepare-docker.service
            Wants=network-online.target
            Requires=containerd.service docker.socket prepare-docker.service
            [Service]
            Type=notify
            EnvironmentFile=-/run/flannel/flannel_docker_opts.env
            Environment=DOCKER_SELINUX=--selinux-enabled=true
            # the default is not to use systemd for cgroups because the delegate issues still
            # exists and systemd currently does not support the cgroup feature set required
            # for containers run by docker
            Environment=PATH=/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
            ExecStart=/opt/bin/dockerd --host=fd:// --containerd=/run/docker/libcontainerd/docker-containerd.sock $DOCKER_SELINUX $DOCKER_OPTS $DOCKER_CGROUPS $DOCKER_OPT_BIP $DOCKER_OPT_MTU $DOCKER_OPT_IPMASQ
            ExecReload=/bin/kill -s HUP $MAINPID
            LimitNOFILE=1048576
            # Having non-zero Limit*s causes performance problems due to accounting overhead
            # in the kernel. We recommend using cgroups to do container-local accounting.
            LimitNPROC=infinity
            LimitCORE=infinity
            # Uncomment TasksMax if your systemd version supports it.
            # Only systemd 226 and above support this version.
            TasksMax=infinity
            TimeoutStartSec=0
            # set delegate yes so that systemd does not reset the cgroups of docker containers
            Delegate=yes
            # kill only the docker process, not all processes in the cgroup
            KillMode=process
            # restart the docker process if it exits prematurely
            Restart=on-failure
            StartLimitBurst=3
            StartLimitInterval=60s
            [Install]
            WantedBy=multi-user.target
        - name: containerd.service
          enabled: false
          contents: |
            [Unit]
            Description=containerd container runtime
            After=network.target prepare-docker.service
            Requires=prepare-docker.service
            [Service]
            Delegate=yes
            Environment=PATH=/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
            ExecStartPre=mkdir -p /run/docker/libcontainerd
            ExecStartPre=ln -fs /run/containerd/containerd.sock /run/docker/libcontainerd/docker-containerd.sock
            ExecStart=/opt/bin/containerd --config /etc/containerd/config.toml
            KillMode=process
            Restart=always
            # (lack of) limits from the upstream docker service unit
            LimitNOFILE=1048576
            LimitNPROC=infinity
            LimitCORE=infinity
            TasksMax=infinity
            [Install]
            WantedBy=multi-user.target
    storage:
      files:
        - path: /etc/systemd/system-generators/torcx-generator
        - path: /opt/docker.tgz
          mode: 0644
          contents:
            source: https://download.docker.com/linux/static/stable/x86_64/docker-20.10.12.tgz
            verification:
              hash: sha512-90c3ab8c465bfa6fa51e9e77cf5257ff4bf139723eeb4878afbf294e71a2f2f13558840708e392ff24f8b8853c519938013d4dff8d50b17d66ca0eeb6a1b3c1a
        - path: /etc/containerd/config.toml
          mode: 0644
          contents:
            inline: |
              version = 2
              # set containerd's OOM score
              oom_score = -999
              [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
              # setting runc.options unsets parent settings
              runtime_type = "io.containerd.runc.v2"
              [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
              SystemdCgroup = true
      links:
        - path: /etc/extensions/docker-flatcar.raw
          target: /dev/null
          overwrite: true
        - path: /etc/extensions/containerd-flatcar.raw
          target: /dev/null
          overwrite: true
    

    While the system services have a PATH variable that prefers /opt/bin/ by placing it first, you have to run the following command on every interactive login shell (also after sudo or su) to make sure you use the correct binaries.

    export PATH="/opt/bin:$PATH"
    

    The empty file /etc/systemd/system-generators/torcx-generator serves the purpose of disabling Torcx to make sure it is not used accidentally in case /opt/bin was missing from the PATH variable. Flatcar releases newer than major release 3760 do not ship torcx so that line can as well be removed from the above config. However, leaving it in does not have any side effects.

    The /etc/extensions/ symlinks make sure that the future built-in Docker/containerd sysext images won’t be enabled.