Custom Trace Resource

    Inspektor Gadget uses a custom Trace resource to communicate between the endpoints and the gadget DaemonSet running on the cluster. These resources are used to control which gadgets run in a given cluster.

    This is the basic format of this resource:

    kind: Trace
      name: trace-name
      namespace: gadget
      node: node-name
      gadget: gadget-name
        namespace: namespace-name
        podname: podname-name
      runMode: Manual
      outputMode: Status

    Some gadgets work at the node level, while others support specific filters, like namespace, podname, labels, and so on.

    The possible values for outputMode also depend on the gadget. The seccomp gadget, for example, can create seccomp policies as an external resource when ExternalResource is selected. If outputMode is set to Status, the output of the trace will be stored in the status field of the trace resource.

    See the corresponding gadgets specs to find out what’s available.

    Note that all traces should be created in the gadget namespace. And, for now, the node name needs to be explicitly set in the trace.

    Setting the Trace operation

    Once the Trace resource is created, the field is used to control when the gadget should run.

    Some gadgets accept a single operation that triggers a run and stores the output. Other gadgets need to be started and stay running until they get stopped later on to obtain the output.

    To control whether a gadget should get started or stopped, annotate the corresponding trace resource with<operation-name>

    Note that the trace controller will remove the annotation after it has processed it. So, if you find yourself having to forcefully overwrite the value of this field, it means that the trace controller is having trouble processing your Trace resource.

    Using Trace resources from the command line

    It’s possible to create and interact with the Trace resources directly from the command line, using kubectl apply with a YAML file containing the specified trace.

    After creating the resource, the corresponding operation can be set with kubectl annotate, as in the following example:

    kubectl annotate -n gadget trace/trace-name

    Using Trace resources from graphical interfaces

    Graphical interfaces that interact with Kubernetes, can integrate with Inspektor Gadget by creating and modifying Trace resources in the gadget namespace, and following the corresponding gadget specifications.

    Gadgets can then be controlled by annotating the field with the corresponding operation.

    As an example, see the biolatency plugin implementation for Headlamp.

    Interacting with traces using the kubectl-gadget CLI

    The kubectl-gadget plugin may create, annotate and delete Trace resources as necessary to interact with the gadget DaemonSet running on the nodes. This is mostly transparent to the user, who will just get the results through the command-line.