Rook Ceph configuration reference for Lokomotive

Introduction

Rook is a storage orchestrator for Kubernetes. This component installs a Ceph cluster managed by the Rook operator.

Prerequisites

  • A Lokomotive cluster accessible via kubectl.

  • The Rook component deployed on the cluster.

Configuration

Rook-Ceph component configuration example:

component "rook-ceph" {
  # Optional arguments
  namespace       = "rook-test"
  monitor_count   = 3
  enable_toolbox  = true
  metadata_device = "md127"
  node_affinity {
    key      = "node-role.kubernetes.io/storage"
    operator = "Exists"
  }
  node_affinity {
    key      = "storage.lokomotive.io"
    operator = "In"

    # If the `operator` is set to `"In"`, `values` should be specified.
    values = [
      "foo",
    ]
  }
  toleration {
    key      = "storage.lokomotive.io"
    operator = "Equal"
    value    = "rook-ceph"
    effect   = "NoSchedule"
  }

  storage_class {
    enable  = true
    default = true
  }
}

The Ceph cluster needs to be deployed in the same namespace as the Rook operator at the moment. Additional Roles and RoleBindings need to be created if deploying across separate namespaces is desired.

Attribute reference

Table of all the arguments accepted by the component.

Argument Description Default Type Required
namespace Namespace to deploy the Ceph cluster into. Must be the same as the rook operator. “rook” string false
monitor_count Number of Ceph monitors to deploy. An odd number like 3 or 5 is recommended which should also be sufficient for most cases. 1 number false
enable_toolbox Deploy the toolbox pod to debug and manage the Ceph cluster. false bool false
node_affinity Node affinity for deploying the Ceph cluster pods. - list(object({key = string, operator = string, values = list(string)})) false
toleration Tolerations that the Ceph cluster pods will tolerate. - list(object({key = string, effect = string, operator = string, value = string, toleration_seconds = string })) false
metadata_device Name of the device to store the metadata on each storage machine. Note: Provide just the name of the device and skip prefixing with /dev/. - string false
storage_class.enable Install Storage Class config. false bool false
storage_class.default Make this Storage Class as a default one. false bool false

Applying

To apply the Rook-Ceph component:

lokoctl component apply rook-ceph

Once the Ceph cluster is ready, an object store can be deployed to start writing to Ceph. More information is available at configuring Ceph object store CRD

Deleting

To destroy the component:

lokoctl component delete rook-ceph

After removing the component from the cluster, make sure to delete /var/lib/rook from the host filesystem of all worker nodes for a clean reinstallation.