Lokomotive Packet configuration reference

Introduction

This configuration reference provides information on configuring a Lokomotive cluster on Packet with all the configuration options available to the user.

Prerequisites

Configuration

To create a Lokomotive cluster, we need to define a configuration.

Example configuration file:

# mycluster.lokocfg
variable "packet_token" {}
variable "asset_dir" {}
variable "facility" {}
variable "cluster_name" {}
variable "controllers_count" {}
variable "workers_count" {}
variable "controller_type" {}
variable "controller_clc_snippets" {}
variable "workers_type" {}
variable "dns_zone" {}
variable "route53_zone_id" {}
variable "packet_project_id" {}
variable "ssh_public_keys" {}
variable "management_cidrs" {}
variable "node_private_cidr" {}
variable "state_s3_bucket" {}
variable "lock_dynamodb_table" {}
variable "oidc_issuer_url" {}
variable "oidc_client_id" {}
variable "oidc_username_claim" {}
variable "oidc_groups_claim" {}
variable "worker_clc_snippets" {}

backend "s3" {
  bucket         = var.state_s3_bucket
  key            = var.state_s3_key
  region         = var.state_s3_region
  dynamodb_table = var.lock_dynamodb_table
}

# backend "local" {
#   path = "path/to/local/file"
#}

cluster "packet" {
  auth_token = var.packet_token

  asset_dir = var.asset_dir

  cluster_name = var.cluster_name

  controller_count = var.controllers_count

  controller_type = "c3.small.x86"

  controller_clc_snippets = var.controller_clc.snippets

  facility = var.facility

  os_channel = "stable"

  os_version = "current"

  os_arch = "amd64"

  ipxe_script_url = ""

  project_id = var.packet_project_id

  dns {
    zone     = var.dns_zone
    provider = "route53"
  }

  ssh_pubkeys = var.ssh_public_keys

  management_cidrs = var.management_cidrs

  node_private_cidr = var.node_private_cidr

  cluster_domain_suffix = "cluster.local"

  network_mtu = 1500

  tags {
    key1 = "value1"
    key2 = "value2"
  }

  enable_aggregation = true

  enable_tls_bootstrap = true

  encrypt_pod_traffic = true

  enable_reporting = false

  network_ip_autodetection_method = "first-found"

  pod_cidr = "10.2.0.0/16"

  service_cidr = "10.3.0.0/16"

  reservation_ids = { controller-0 = "55555f20-a1fb-55bd-1e11-11af11d11111" }

  reservation_ids_default = ""

  certs_validity_period_hours = 8760

  oidc {
    issuer_url     = var.oidc_issuer_url
    client_id      = var.oidc_client_id
    username_claim = var.oidc_username_claim
    groups_claim   = var.oidc_groups_claim
  }

  worker_pool "worker-pool-1" {
    count = var.workers_count

    clc_snippets = var.worker_clc_snippets

    tags = {
      pool = "storage"
    }

    ipxe_script_url = ""

    os_arch = "amd64"

    disable_bgp = false

    node_type = var.workers_type

    os_channel = "stable"

    os_version = "current"

    labels = {
      "testlabel" = ""
    }

    taints = {
      "nodeType" = "storage:NoSchedule"
    }

    setup_raid = false

    setup_raid_hdd = false

    setup_raid_ssd = false

    setup_raid_ssd_fs = false
  }
}

NOTE: Should you feel differently about the default values, you can set default values using the variable block in the cluster configuration.

Example:

The default for node_type is c3.small.x86. If you wish to change the default, then you define the variable and use it to refer in the cluster configuration.

variable "custom_default_worker_type" {
  default = "c2.medium.x86"
}

.
.
.
node_type = var.custom_default_worker_type
.
.
.

Attribute reference

| Argument | Description | Default | Type | Required | |—————————————|————————————————————————————————————————————————————————————————————————————————————————————————————————————|-:-:————-|-:-:———-|-:-:——| | auth_token | Packet Auth token. Use the PACKET_AUTH_TOKEN environment variable instead. | - | string | false | | asset_dir | Location where Lokomotive stores cluster assets. | - | string | true | | cluster_name | Name of the cluster. | - | string | true | | tags | List of tags that will be propagated to master nodes. | - | map(string) | false | | controller_count | Number of controller nodes. | 1 | number | false | | controller_type | Packet instance type for controllers. Changing this field on existing cluster will be ignored. To actually apply it, cluster needs to be destroyed and re-created. | “c3.small.x86” | string | false | | controller_clc_snippets | Controller Flatcar Container Linux Config snippets. | [] | list(string) | false | | dns | DNS configuration block. | - | object | true | | dns.zone | A DNS zone to use for the cluster. The following format is used for cluster-related DNS records: <record>.<cluster_name>.<dns_zone> | - | string | true | | dns.provider | DNS provider to use for the cluster. Valid values: cloudflare, route53, manual. | - | string | true | | oidc | OIDC configuration block. | - | object | false | | oidc.issuer_url | URL of the provider which allows the API server to discover public signing keys. Only URLs which use the https:// scheme are accepted. | - | string | false | | oidc.client_id | A client id that all tokens must be issued for. | “clusterauth” | string | false | | oidc.username_claim | JWT claim to use as the user name. | “email” | string | false | | oidc.groups_claim | JWT claim to use as the user’s group. | “groups” | string | false | | facility | Packet facility to use for deploying the cluster. | - | string | false | | project_id | Packet project ID. | - | string | true | | ssh_pubkeys | List of SSH public keys for user core. Each element must be specified in a valid OpenSSH public key format, as defined in RFC 4253 Section 6.6, e.g. “ssh-rsa AAAAB3N…”. | - | list(string) | true | | os_arch | Flatcar Container Linux architecture to install (amd64, arm64). | “amd64” | string | false | | os_channel | Flatcar Container Linux channel to install from (stable, beta, alpha, edge). | “stable” | string | false | | os_version | Flatcar Container Linux version to install. Version such as “2303.3.1” or “current”. | “current” | string | false | | ipxe_script_url | Boot via iPXE. Required for arm64. | - | string | false | | management_cidrs | List of IPv4 CIDRs authorized to access or manage the cluster. Example [“0.0.0.0/0”] to allow all. | - | list(string) | true | | node_private_cidr | Private IPv4 CIDR of the nodes used to allow inter-node traffic. Example “10.0.0.0/8” | - | string | true | | enable_aggregation | Enable the Kubernetes Aggregation Layer. | true | bool | false | | enable_tls_bootstrap | Enable TLS bootstraping for Kubelet. | true | bool | false | | encrypt_pod_traffic | Enable in-cluster pod traffic encryption. If true network_mtu is reduced by 60 to make room for the encryption header. | false | bool | false | | ignore_x509_cn_check | Ignore check of common name in x509 certificates. If any application is built pre golang 1.15 then API server rejects x509 from such application, enable this to get around apiserver. | false | bool | false | | network_mtu | Physical Network MTU. | 1500 | number | false | | pod_cidr | CIDR IPv4 range to assign Kubernetes pods. | “10.2.0.0/16” | string | false | | service_cidr | CIDR IPv4 range to assign Kubernetes services. | “10.3.0.0/16” | string | false | | cluster_domain_suffix | Cluster’s DNS domain. | “cluster.local” | string | false | | enable_reporting | Enables usage or analytics reporting to upstream. | false | bool | false | | reservation_ids | Block with Packet hardware reservation IDs for controller nodes. Each key must have the format controller-${index} and the value is the reservation UUID. Can’t be combined with reservation_ids_default. Key indexes must be sequential and start from 0. Example: reservation_ids = { controller-0 = "<reservation_id>" }. | - | map(string) | false | | reservation_ids_default | Default reservation ID for controllers. The valuenext-available will choose any reservation that matches the pool’s device type and facility. Can’t be combined with reservation_ids | - | string | false | | certs_validity_period_hours | Validity of all the certificates in hours. | 8760 | number | false | | worker_pool | Configuration block for worker pools. There can be more than one. | - | list(object) | true | | worker_pool.count | Number of workers in the worker pool. Can be changed afterwards to add or delete workers. | 1 | number | true | | worker_pool.clc_snippets | Flatcar Container Linux Config snippets for nodes in the worker pool. | [] | list(string) | false | | worker_pool.tags | List of tags that will be propagated to nodes in the worker pool. | - | map(string) | false | | worker_pool.disable_bgp | Disable BGP on nodes. Nodes won’t be able to connect to Packet BGP peers. | false | bool | false | | worker_pool.ipxe_script_url | Boot via iPXE. Required for arm64. | - | string | false | | worker_pool.os_arch | Flatcar Container Linux architecture to install (amd64, arm64). | “amd64” | string | false | | worker_pool.os_channel | Flatcar Container Linux channel to install from (stable, beta, alpha, edge). | “stable” | string | false | | worker_pool.os_version | Flatcar Container Linux version to install. Version such as “2303.3.1” or “current”. | “current” | string | false | | worker_pool.node_type | Packet instance type for worker nodes. | “c3.small.x86” | string | false | | worker_pool.labels | Map of extra Kubernetes Node labels for worker nodes. | - | map(string) | false | | worker_pool.taints | Map of Taints to assign to worker nodes. | - | map(string) | false | | worker_pool.reservation_ids | Block with Packet hardware reservation IDs for worker nodes. Each key must have the format worker-${index} and the value is the reservation UUID. Can’t be combined with reservation_ids_default. Key indexes must be sequential and start from 0. Example: reservation_ids = { worker-0 = "<reservation_id>" }. | - | map(string) | false | | worker_pool.reservation_ids_default | Default reservation ID for workers. The valuenext-available will choose any reservation that matches the pool’s device type and facility. Can’t be combined with reservation_ids. | - | string | false | | worker_pool.setup_raid | Attempt to create a RAID 0 from extra disks to be used for persistent container storage. Can’t be used with setup_raid_hdd nor setup_raid_sdd. | false | bool | false | | worker_pool.setup_raid_hdd | Attempt to create a RAID 0 from extra Hard Disk drives only, to be used for persistent container storage. Can’t be used with setup_raid nor setup_raid_sdd. | false | bool | false | | worker_pool.setup_raid_ssd | Attempt to create a RAID 0 from extra Solid State Drives only, to be used for persistent container storage. Can’t be used with setup_raid nor setup_raid_hdd. | false | bool | false | | worker_pool.setup_raid_ssd_fs | When set to true file system will be created on SSD RAID device and will be mounted on /mnt/node-local-ssd-storage. To use the raw device set it to false. | false | bool | false |

Applying

To create the cluster, execute the following command:

lokoctl cluster apply

Destroying

To destroy the Lokomotive cluster, execute the following command:

lokoctl cluster destroy --confirm

ARM support and hybrid clusters

Lokomotive and Flatcar currently include an alpha-quality preview for the Packet arm64 server type c2.large.arm. They can be used for both worker and controller nodes. Besides specifying them in controller_type/node_type you need to configure some additional variables in the respective controller/worker module:

os_arch = "arm64"
os_channel = "alpha"
ipxe_script_url = "https://alpha.release.flatcar-linux.net/arm64-usr/current/flatcar_production_packet.ipxe"

The iPXE boot variable can be removed once Flatcar is available for installation in the Packet OS menu for the ARM servers.

If you have a hybrid cluster with both x86 and ARM nodes, you need to either use Docker multiarch images such as the standard debian:latest or python:latest images, or restrict Pods to nodes of the correct architecture with an entry like this for ARM (or with amd64 for x86) in your YAML deployment:

nodeSelector:
  kubernetes.io/arch: arm64

An example on how to build multiarch images yourself is here .

Note: Lokomotive Components are not supported in this preview, so if you need them you’ll have to install them manually using Kubernetes directly.