Using a custom Docker or containerd version

    Some system tooling can’t be run on Container Linux via containers and this is especially true for the container runtime itself. As with other special binaries you want to bring to the system you can use an Ignition config that downloads the binaries.

    For custom Docker binaries you can consider to use the Torcx addon manager when you build your own Torcx image and profile. However, since the Torcx makes certain assumtions about the files being shipped you may find it too limiting (e.g., the wrapper scripts under /usr/bin/ exist for only a fixed set of binaries). In this case you can directly place the custom binaries to /opt/bin/ as done by the following Container Linux Config which you can transpile to an Ignition config with ct .

    This replicates the Docker setup as of Flatcar Container Linux 2605.10.0 but under /etc and /opt/bin/, and with additional support for the upstream Containerd socket location. You can modify it to use different socket paths or plugins, or even only ship containerd if you don’t need Docker.

    systemd:
      units:
        - name: prepare-docker.service
          enabled: true
          contents: |
            [Unit]
            Description=Unpack docker binaries to /opt/bin
            ConditionPathExists=!/opt/bin/docker
            [Service]
            Type=oneshot
            RemainAfterExit=true
            Restart=on-failure
            ExecStartPre=/usr/bin/mkdir -p /opt/bin
            ExecStartPre=/usr/bin/tar -v --extract --file /opt/docker.tgz --directory /opt/ --no-same-owner
            ExecStartPre=/usr/bin/rm /opt/docker.tgz
            ExecStartPre=/usr/bin/sh -c "mv /opt/docker/* /opt/bin/"
            ExecStart=/usr/bin/rmdir /opt/docker
            [Install]
            WantedBy=multi-user.target
        - name: docker.socket
          enabled: true
          contents: |
            [Unit]
            PartOf=docker.service
            Description=Docker Socket for the API
            [Socket]
            ListenStream=/var/run/docker.sock
            SocketMode=0660
            SocketUser=root
            SocketGroup=docker
            [Install]
            WantedBy=sockets.target
        - name: docker.service
          enabled: false
          contents: |
            [Unit]
            Description=Docker Application Container Engine
            After=containerd.service docker.socket network-online.target prepare-docker.service
            Wants=network-online.target
            Requires=containerd.service docker.socket prepare-docker.service
            [Service]
            Type=notify
            EnvironmentFile=-/run/flannel/flannel_docker_opts.env
            Environment=DOCKER_SELINUX=--selinux-enabled=true
            # the default is not to use systemd for cgroups because the delegate issues still
            # exists and systemd currently does not support the cgroup feature set required
            # for containers run by docker
            Environment=PATH=/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
            ExecStart=/opt/bin/dockerd --host=fd:// --containerd=/run/docker/libcontainerd/docker-containerd.sock $DOCKER_SELINUX $DOCKER_OPTS $DOCKER_CGROUPS $DOCKER_OPT_BIP $DOCKER_OPT_MTU $DOCKER_OPT_IPMASQ
            ExecReload=/bin/kill -s HUP $MAINPID
            LimitNOFILE=1048576
            # Having non-zero Limit*s causes performance problems due to accounting overhead
            # in the kernel. We recommend using cgroups to do container-local accounting.
            LimitNPROC=infinity
            LimitCORE=infinity
            # Uncomment TasksMax if your systemd version supports it.
            # Only systemd 226 and above support this version.
            TasksMax=infinity
            TimeoutStartSec=0
            # set delegate yes so that systemd does not reset the cgroups of docker containers
            Delegate=yes
            # kill only the docker process, not all processes in the cgroup
            KillMode=process
            # restart the docker process if it exits prematurely
            Restart=on-failure
            StartLimitBurst=3
            StartLimitInterval=60s
            [Install]
            WantedBy=multi-user.target
        - name: containerd.service
          enabled: false
          contents: |
            [Unit]
            Description=containerd container runtime
            After=network.target prepare-docker.service
            Requires=prepare-docker.service
            [Service]
            Delegate=yes
            Environment=PATH=/opt/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
            ExecStartPre=mkdir -p /run/docker/libcontainerd
            ExecStartPre=ln -fs /run/containerd/containerd.sock /run/docker/libcontainerd/docker-containerd.sock
            ExecStart=/opt/bin/containerd --config /etc/containerd/config.toml
            KillMode=process
            Restart=always
            # (lack of) limits from the upstream docker service unit
            LimitNOFILE=1048576
            LimitNPROC=infinity
            LimitCORE=infinity
            TasksMax=infinity
            [Install]
            WantedBy=multi-user.target
    storage:
      files:
        - path: /opt/docker.tgz
          filesystem: root
          mode: 0644
          contents:
            remote:
              url: https://download.docker.com/linux/static/stable/x86_64/docker-19.03.14.tgz
        - path: /etc/containerd/config.toml
          filesystem: root
          mode: 0644
          contents:
            inline: |
              # persistent data location
              root = "/var/lib/containerd"
              # runtime state information
              state = "/run/containerd"
              # set containerd as a subreaper on linux when it is not running as PID 1
              subreaper = true
              # set containerd's OOM score
              oom_score = -999
              disabled_plugins = []
              # grpc configuration
              [grpc]
              address = "/run/containerd/containerd.sock"
              # socket uid
              uid = 0
              # socket gid
              gid = 0
              [plugins.linux]
              # shim binary name/path
              shim = "containerd-shim"
              # runtime binary name/path
              runtime = "runc"
              # do not use a shim when starting containers, saves on memory but
              # live restore is not supported
              no_shim = false
    

    While the system services have a PATH variable that prefers /opt/bin/ by placing it first, you have to run the following command on every interactive login shell (also after sudo or su) to make sure you use the correct binaries.

    export PATH="/opt/bin:$PATH"