Lokomotive AWS configuration reference
Introduction
This configuration reference provides information on configuring a Lokomotive cluster on AWS with all the configuration options available to the user.
Prerequisites
lokoctl
installed locally.kubectl
installed locally to access the Kubernetes cluster.
Configuration
To create a Lokomotive cluster, we need to define a configuration.
Example configuration file:
#myawscluster.lokocfg
variable "dns_zone" {}
variable "route53_zone_id" {}
variable "ssh_public_keys" {}
variable "state_s3_bucket" {}
variable "lock_dynamodb_table" {}
variable "asset_dir" {}
variable "cluster_name" {}
variable "controllers_count" {}
variable "workers_count" {}
variable "state_s3_key" {}
variable "state_s3_region" {}
variable "workers_type" {}
variable "controller_clc_snippets" {}
variable "worker_clc_snippets" {}
variable "region" {}
variable "disk_size" {}
variable "disk_type" {}
variable "disk_iops" {}
variable "worker_disk_size" {}
variable "worker_disk_type" {}
variable "worker_disk_iops" {}
variable "worker_spot_price" {}
variable "lb_http_port" {}
variable "lb_https_port" {}
variable "target_groups" {}
variable "oidc_issuer_url" {}
variable "oidc_client_id" {}
variable "oidc_username_claim" {}
variable "oidc_groups_claim" {}
backend "s3" {
bucket = var.state_s3_bucket
key = var.state_s3_key
region = var.state_s3_region
dynamodb_table = var.lock_dynamodb_table
}
# backend "local" {
# path = "path/to/local/file"
#}
cluster "aws" {
asset_dir = var.asset_dir
cluster_name = var.cluster_name
controller_count = var.controllers_count
controller_type = var.controller_type
os_channel = "stable"
os_version = "current"
tags {
key1 = "value1"
key2 = "value2"
}
dns_zone = var.dns_zone
dns_zone_id = route53_zone_id
enable_csi = true
expose_nodeports = false
ssh_pubkeys = var.ssh_public_keys
certs_validity_period_hours = 8760
controller_clc_snippets = var.controller_clc_snippets
region = var.region
enable_aggregation = true
enable_tls_bootstrap = true
encrypt_pod_traffic = true
disk_size = var.disk_size
disk_type = var.disk_type
disk_iops = var.disk_iops
network_mtu = 1500
host_cidr = ""
pod_cidr = "10.2.0.0/16"
service_cidr = "10.3.0.0/16"
cluster_domain_suffix = "cluster.local"
enable_reporting = false
conntrack_max_per_core = 32768
oidc {
issuer_url = var.oidc_issuer_url
client_id = var.oidc_client_id
username_claim = var.oidc_username_claim
groups_claim = var.oidc_groups_claim
}
worker_pool "my-worker-pool" {
count = 2
instance_type = var.workers_type
ssh_pubkeys = var.ssh_public_keys
os_channel = "stable"
os_version = "current"
labels = {
"testlabel" = ""
}
taints = {
"nodeType" = "storage:NoSchedule"
}
disk_size = var.worker_disk_size
disk_type = var.worker_disk_type
disk_iops = var.worker_disk_iops
spot_price = var.worker_spot_price
lb_http_port = var.lb_http_port
lb_https_port = var.lb_https_port
target_groups = var.target_groups
clc_snippets = var.worker_clc_snippets
tags = {
"key" = "value"
}
}
}
NOTE: Should you feel differently about the default values, you can set default values using the variable
block in the cluster configuration.
Example:
The default for instance_type in worker pool block is t3.small
. If you wish to change the default, then you
define the variable and use it to refer in the cluster configuration.
variable "custom_worker_type" {
default = "i3.large"
}
.
.
worker_pool "my-worker-pool" {
worker_type = var.custom_worker_type
.
.
}
.
Attribute reference
| Argument | Description | Default | Type | Required |
|———————————-|————————————————————————————————————————————————————————————————————————————————————————————————————–|-:-:————-|-:-:———-|-:-:——|
| asset_dir
| Location where Lokomotive stores cluster assets. | - | string | true |
| cluster_name
| Name of the cluster. NOTE: It must be unique per DNS Zone and region. | - | string | true |
| tags
| Optional details to tag on AWS resources. | - | map(string) | false |
| os_channel
| Flatcar Container Linux AMI channel to install from (stable, beta, alpha, edge). | “stable” | string | false |
| os_version
| Flatcar Container Linux version to install. Version such as “2303.3.1” or “current”. | “current” | string | false |
| dns_zone
| Route 53 DNS Zone. | - | string | true |
| dns_zone_id
| Route 53 DNS Zone ID. | - | string | true |
| oidc
| OIDC configuration block. | - | object | false |
| oidc.issuer_url
| URL of the provider which allows the API server to discover public signing keys. Only URLs which use the https:// scheme are accepted. | - | string | false |
| oidc.client_id
| A client id that all tokens must be issued for. | “clusterauth” | string | false |
| oidc.username_claim
| JWT claim to use as the user name. | “email” | string | false |
| oidc.groups_claim
| JWT claim to use as the user’s group. | “groups” | string | false |
| enable_csi
| Set up IAM role needed for dynamic volumes provisioning to work on AWS | false | bool | false |
| expose_nodeports
| Expose node ports 30000-32767
in the security group, if set to true
. | false | bool | false |
| ssh_pubkeys
| List of SSH public keys for user core
. Each element must be specified in a valid OpenSSH public key format, as defined in RFC 4253 Section 6.6, e.g. “ssh-rsa AAAAB3N…”. | - | list(string) | true |
| controller_count
| Number of controller nodes. | 1 | number | false |
| controller_type
| AWS instance type for controllers. | “t3.small” | string | false |
| controller_clc_snippets
| Controller Flatcar Container Linux Config snippets. | [] | list(string) | false |
| region
| AWS region to use for deploying the cluster. | “eu-central-1” | string | false |
| enable_aggregation
| Enable the Kubernetes Aggregation Layer. | true | bool | false |
| enable_tls_bootstrap
| Enable TLS bootstraping for Kubelet. | true | bool | false |
| encrypt_pod_traffic
| Enable in-cluster pod traffic encryption. If true network_mtu
is reduced by 60 to make room for the encryption header. | false | bool | false |
| ignore_x509_cn_check
| Ignore check of common name in x509 certificates. If any application is built pre golang 1.15 then API server rejects x509 from such application, enable this to get around apiserver. | false | bool | false |
| disk_size
| Size of the EBS volume in GB. | 40 | number | false |
| disk_type
| Type of the EBS volume (e.g. standard, gp2, io1). | “gp2” | string | false |
| disk_iops
| IOPS of the EBS volume (e.g 100). | 0 | number | false |
| network_mtu
| Physical Network MTU. When using instance types with Jumbo frames, use 9001. | 1500 | number | false |
| host_cidr
| CIDR IPv4 range to assign to EC2 nodes. | “10.0.0.0/16” | string | false |
| pod_cidr
| CIDR IPv4 range to assign Kubernetes pods. | “10.2.0.0/16” | string | false |
| service_cidr
| CIDR IPv4 range to assign Kubernetes services. | “10.3.0.0/16” | string | false |
| cluster_domain_suffix
| Cluster’s DNS domain. | “cluster.local” | string | false |
| enable_reporting
| Enables usage or analytics reporting to upstream. | false | bool | false |
| certs_validity_period_hours
| Validity of all the certificates in hours. | 8760 | number | false |
| conntrack_max_per_core
| Maximum number of entries in conntrack table per CPU on all nodes in the cluster. If you require more fain-grained control over this value, set it to 0 and add CLC snippet setting net.netfilter.nf_conntrack_max sysctl setting per node pool. See [Flatcar documentation about sysctl] for more details. | 32768 | number | false | |
worker_pool | Configuration block for worker pools. There can be more than one. **NOTE**: worker pool name must be unique per DNS zone and region. | - | list(object) | true | |
worker_pool.count | Number of workers in the worker pool. Can be changed afterwards to add or delete workers. | - | number | true | |
worker_pool.cpu_manager_policy| CPU Manager policy to use. Possible values:
none,
static. | "none" | string | false | |
worker_pool.instance_type | AWS instance type for worker nodes. | "t3.small" | string | false | |
worker_pool.ssh_pubkeys | List of SSH public keys for user
core. Each element must be specified in a valid OpenSSH public key format, as defined in RFC 4253 Section 6.6, e.g. "ssh-rsa AAAAB3N...". | - | list(string) | true | |
worker_pool.os_channel | Flatcar Container Linux channel to install from (stable, beta, alpha, edge). | "stable" | string | false | |
worker_pool.os_version | Flatcar Container Linux version to install. Version such as "2303.3.1" or "current". | "current" | string | false | |
worker_pool.labels | Map of extra Kubernetes Node labels for worker nodes. | - | map(string) | false | |
worker_pool.taints | Map of Taints to assign to worker nodes. | - | map(string) | false | |
worker_pool.disk_size | Size of the EBS volume in GB. | 40 | number | false | |
worker_pool.disk_type | Type of the EBS volume (e.g. standard, gp2, io1). | "gp2" | string | false | |
worker_pool.disk_iops | IOPS of the EBS volume (e.g 100). | 0 | number | false | |
worker_pool.spot_price | Spot price in USD for autoscaling group spot instances. Leave as empty string for autoscaling group to use on-demand instances. Switching in-place from spot to on-demand is not possible. | "" | string | false | |
worker_pool.target_groups | Additional target group ARNs to which worker instances should be added. | [] | list(string) | false | |
worker_pool.lb_http_port | Port the load balancer should listen on for HTTP connections. | 80 | number | false | |
worker_pool.lb_https_port | Port the load balancer should listen on for HTTPS connections. | 443 | number | false | |
worker_pool.clc_snippets | CWorker Flatcar Container Linux Config snippets. | [] | list(string) | false | |
worker_pool.tags` | Optional details to tag on AWS resources. | - | map(string) | false |
Applying
To create the cluster, execute the following command:
lokoctl cluster apply
Destroying
To destroy the Lokomotive cluster, execute the following command:
lokoctl cluster destroy --confirm