The "profile" gadget

    The profile gadget takes samples of the stack traces. Pods can be selected by Kubernetes labels, pod names, namespaces, and nodes. Here we deploy a small demo pod “random”:

    $ kubectl run --restart=Never --image=busybox random -- sh -c 'cat /dev/urandom > /dev/null'
    pod/random created
    

    Using the profile gadget, we can see the list of stack traces. The following command filters only for pods named “random”, execute the command and interrupt it after ~30 seconds. The -K option is passed to show only the kernel stack traces.

    $ kubectl gadget profile --podname random -K
    Node numbers: 0 = ip-10-0-23-61 1 = ip-10-0-3-62^C
    

    After a while press with Ctrl-C to stop trace collection

    ^C
    Terminating...
    [ 0] cat;entry_SYSCALL_64_after_hwframe;do_syscall_64;ksys_read;vfs_read;urandom_read;_copy_to_user;copy_user_enhanced_fast_string;copy_user_enhanced_fast_string 4
    [ 0] cat 8
    [ 0] cat;entry_SYSCALL_64_after_hwframe;do_syscall_64;ksys_read;vfs_read;urandom_read;_raw_spin_unlock_irqrestore;_raw_spin_unlock_irqrestore 136
    

    From the traces above, you can see that the pod is spending CPU time in the Linux function urandom_read.

    Finally, we need to clean up our pod:

    $ kubectl delete pod random