Flatcar Container Linux developer SDK guide

    These are the instructions for building Flatcar Container Linux itself. By the end of the guide you will build a developer image that you can run under KVM and have tools for making changes to the code.

    Flatcar Container Linux is an open source project. All of the source for Flatcar Container Linux is available on github . If you find issues with these docs or the code please send a pull request.

    Direct questions and suggestions to the IRC channel or mailing list .

    Getting started

    Let’s get set up with an SDK chroot and build a bootable image of Flatcar Container Linux. The SDK chroot has a full toolchain and isolates the build process from quirks and differences between host OSes. The SDK must be run on an x86-64 Linux machine, the distro should not matter (Ubuntu, Fedora, etc).

    Prerequisites

    System requirements to get started:

    • curl
    • git
    • bzip2
    • gpg
    • sudo

    You also need a proper git setup:

    git config --global user.email "[email protected]"
git config --global user.name "Your Name"

    NOTE: Do the git configuration as a normal user and not with sudo.

    Using Cork

    The cork utility, included in the Flatcar Container Linux mantle project, is used to create and work with an SDK chroot.

    First, download the cork utility and verify it with the signature:

    curl -L -o cork https://github.com/kinvolk/mantle/releases/download/v0.15.2/cork-0.15.2-amd64
curl -L -o cork.sig https://github.com/kinvolk/mantle/releases/download/v0.15.2/cork-0.15.2-amd64.sig
curl -LO https://www.flatcar-linux.org/security/image-signing-key/Flatcar_Image_Signing_Key.asc
gpg --import Flatcar_Image_Signing_Key.asc
rm Flatcar_Image_Signing_Key.asc
gpg --verify cork.sig cork

    The gpg --verify command should output something like this:

    gpg: Signature made Mon 07 Jan 2019 14:51:50 CET
gpg:                using RSA key 84C8E771C0DF83DFBFCAAAF03ADA89DEC2507883
gpg: Good signature from "Flatcar Application Signing Key <[email protected]>" [unknown]
Primary key fingerprint: C1C0 B82A 2F75 90B2 E369  822B E52F 0DB3 9145 3C45
     Subkey fingerprint: 84C8 E771 C0DF 83DF BFCA  AAF0 3ADA 89DE C250 7883

    Then proceed with the installation of the cork binary to a location on your path:

    chmod +x cork
mkdir -p ~/.local/bin
mv cork ~/.local/bin
export PATH=$PATH:$HOME/.local/bin

    You may want to add the PATH export to your shell profile (e.g. .bashrc).

    Next, use the cork utility to create a project directory. This will hold all of your git repos and the SDK chroot. A few gigabytes of space will be necessary.

    mkdir flatcar-sdk
cd flatcar-sdk
cork create # This will request root permisions via sudo
cork enter # This will request root permisions via sudo

    By default cork create will download the latest SDK used for the Alpha release which can be changed with --sdk-version x.y.z. You can also change the SDK release later.

    A manifest defines which source code branches are checked out. The default manifest uses the main development branches for the next Alpha major version. If you want to check out a specific release, you can pass --manifest-branch flatcar-CHANNEL-x.y.z. If you want to add changes to a major version that is propagated through the channels and gets maintenance releases increasing the minor version, you need to use the maintenance branch flatcar-MAJORVERSION (the major version is the x in x.y.z) which you can check out by passing --manifest-branch flatcar-CHANNEL-x.y.z --manifest-name maintenance.xml for any release of that major version family (not for releases <= 2513.2.0). You can also change the branches later with git.

    Verify you are in the SDK chroot:

    $ grep NAME /etc/os-release
NAME="Flatcar Container Linux by Kinvolk"

    To leave the SDK chroot, simply run exit.

    To use the SDK chroot in the future, run cork enter from the above directory.

    Building an image

    Set up the chroot

    When entering the SDK you are in the ~/trunk/src/scripts repository which can be seen as the build system. It is one of the three repositories that define a Flatcar build. The other two are the ~/trunk/src/third_party/portage-stable and the ~/trunk/src/third_party/coreos-overlay repository. These contain Gentoo package build files. The portage-stable repository hosts build files for upstream Gentoo packages and the coreos-overlay repository contains additional and patched packages. You have to check out the right combination of branches if you don’t use the default. For example, checking out a feature branch based on the main branch of coreos-overlay means that portage-stable and scripts also need to point to main (or a feature branch based on it). Any other repositories under third_party/ are managed by commit references and you should not alter them.

    The chroot you entered has the SDK packages installed. The actual image packages are built in a new chroot under /build/amd64-usr/ or /build/arm64-usr/, called the board. Both chroots use Gentoo’s portage to manage the packages (via sudo emerge for the SDK and emerge-BOARD for the boards). To reduce the build time it is important that as many binary packages as possible are downloaded from a previous SDK release, Flatcar release, or nightly build.

    To reference the latest nightly builds as source for board packages, use the following command. It has to be rerun if you later want to switch to a recent nightly version because it only dereferences the nightly of the day when you ran this command, to keep your build environment reproducible. For BRANCH use the branch you checked out or want to base your work on, e.g., the Alpha development branch main or the flatcar-MAJORVERSION maintenance branches.

    ./set_version --dev-board --board-version amd64-usr/BRANCH-nightly --dev-sdk --sdk-version sdk-main-nightly

    To switch to a Flatcar release when you checked out release branches use the following:

    ./set_version --no-dev-board --board-version x.y.z --no-dev-sdk --sdk-version x.0.0
# --no-dev-board disables any previous --dev-board setting, same for --no-dev-sdk

    Normally it is enough to switch to the first SDK release of a major version (as above) to ensure that you don’t have to rebuild many SDK packages. If you notice that no binary packages are used, try a later minor release.

    Optional: Set a password for console login

    After entering the chroot via cork for the first time, you can set user core's password:

    ./set_shared_user_password.sh

    This is the password you will use to log into the console of images built and launched with the SDK.

    You can also rely on SSH key detection which normally works well when booting the QEMU images.

    Selecting the architecture to build

    amd64-usr and arm64-usr are the only targets supported by Flatcar. When setting up a new build target you can select it as default architecture, otherwise you need to specify the architecture for each step.

    64 bit AMD: The amd64-usr target

    The --board option can be set to one of a few known target architectures, or system “boards”, to build for a given CPU.

    To create a root filesystem for the amd64-usr target beneath the directory /build/amd64-usr/:

    ./setup_board [--default] --board=amd64-usr
    64 bit ARM: The arm64-usr target

    The SDK runs on an amd64 host system, relying on cross-compilation to create arm64 binaries. Still, during the compilation phase many build systems perform compiler tests by running a compiled binary or invoke compiled helper binaries. This requires the host system to use binary translation for seamlessly running arm64 binaries.

    On a Fedora/Debian host system there is a qemu-user-static package that sets everything up. If you have troubles or use a different host system, check that a statically compiled aarch64 qemu-user binary is referenced with the :F flag for pinning it into the kernel so that it is not needed in every chroot:

    $ cat /usr/lib/binfmt.d/qemu-aarch64-static.conf
:qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-aarch64-static:F
$ sudo systemctl restart systemd-binfmt.service

    You can run docker run --rm -ti arm64v8/alpine on your host system as an easy check to verify everything is ready.

    Once in the SDK, create a root filesystem for the arm64-usr target beneath the directory /build/arm64-usr/:

    ./setup_board [--default] --board=arm64-usr

    The SDK will set up the QEMU_LD_PREFIX environment variable, allowing to run any binaries under /build/arm64-usr/, without an additional chroot command.

    Build all of the target binary packages:

    ./build_packages [--board=...]

    You can make sure that a package is rebuilt with your changes by running emerge manually. For board packages run emerge-amd64-usr PACKAGE and for SDK packages run sudo emerge PACKAGE with PACKAGE being either CATEGORY/PACKAGEFOLDER or just PACKAGEFOLDER.

    Render the Flatcar Container Linux image

    Build a production image based on the binary packages built above:

    ./build_image [--board=...]

    After build_image completes, it prints commands for converting the raw bin into a bootable virtual machine. Run the image_to_vm.sh command.

    Booting

    Once you build an image you can launch it with KVM (instructions will print out after image_to_vm.sh runs).

    If you encounter errors with KVM, verify that virtualization is supported by your CPU by running egrep '(vmx|svm)' /proc/cpuinfo. The /dev/kvm directory will be in your host OS when virtualization is enabled in the BIOS.

    The ./flatcar_production_qemu.sh file can be found in the ~/trunk/src/build/images/amd64-usr/latest directory inside the SDK chroot.

    Boot Options

    After image_to_vm.sh completes, run ./flatcar_production_qemu.sh -curses to launch a graphical interface to log in to the Flatcar Container Linux VM.

    You could instead use the -nographic option, ./flatcar_production_qemu.sh -nographic, which gives you the ability to switch from the VM to the QEMU monitor console by pressing CTRL+a and then c. To close the Flatcar Container Linux Guest OS VM, run sudo systemctl poweroff inside the VM.

    You could also log in via SSH by running ./flatcar_production_qemu.sh and then running ssh [email protected] -p 2222 to enter the guest OS. Running without the -p 2222 option will arise a ssh: connect to host 127.0.0.1 port 22: Connection refused or Permission denied (publickey,gssapi-keyex,gssapi-with-mic) warning. Additionally, you can log in via SSH keys or with a different ssh port by running this example ./flatcar_production_qemu.sh -a ~/.ssh/authorized_keys -p 2223 -- -curses. Refer to the Booting with QEMU guide for more information on this usage.

    The default login username is core and the password is the one set in the ./set_shared_user_password step of this guide. If you forget your password, you will need to rerun ./set_shared_user_password and then ./build_image again.

    Making changes

    git and repo

    Flatcar Container Linux is managed by repo, a tool built for the Android project that makes managing a large number of git repositories easier. From the repo announcement blog:

    The repo tool uses an XML-based manifest file describing where the upstream repositories are, and how to merge them into a single working checkout. repo will recurse across all the git subtrees and handle uploads, pulls, and other needed items. repo has built-in knowledge of topic branches and makes working with them an essential part of the workflow.

    (from the Google Open Source Blog )

    You can find the full manual for repo by visiting android.com - Developing .

    Updating repo manifests

    The repo manifest for Flatcar Container Linux lives in a git repository in .repo/manifests. If you need to update the manifest edit default.xml in this directory.

    repo uses a branch called ‘default’ to track the upstream branch you specify in repo init, this defaults to ‘origin/master’. Keep this in mind when making changes, the origin git repository should not have a ‘default’ branch.

    Building release images

    The production images document is unmaintained and out of date, but contains useful pointers as to how official release images are built.

    Tips and tricks

    We’ve compiled a list of tips and tricks that can make working with the SDK a bit easier.

    Testing images

    Mantle is a collection of utilities used in testing and launching SDK images.

    github Edit this page github File documentation issue