Site icon Ron Ekins' – Oracle Technology, DevOps and Kubernetes Blog

Working with Kubernetes Labels

Kubernetes labels provide an important role in the management of large complex Kubernetes clusters.

Kubernetes implements labels as key value pairs which are attached to objects, such as nodes and pods to store identifying and meaningful attributes, which can then be used to manage the running Kubernetes platform.

For the purpose of the post I will be using an Oracle Cloud Infrastructure (OCI) Oracle Container Engine for Kubernetes (OKE) Kubernetes cluster, and 2 labels which I will be using to install Portworx in a future blog.

Working with Kubernetes Labels

There are a number of well-known Kubernetes labels which are common across Cloud vendors, for example kubernetes.io/os and kuberenetes.io/arch.

rekins@rekins--MacBookPro15 PX % kubectl get nodes -L kubernetes.io/os,kubernetes.io/arch 
NAME          STATUS   ROLES   AGE     VERSION   OS      ARCH
10.0.10.133   Ready    node    5h16m   v1.24.1   linux   amd64
10.0.10.167   Ready    node    5h16m   v1.24.1   linux   amd64
10.0.10.182   Ready    node    5h16m   v1.24.1   linux   amd64
10.0.10.205   Ready    node    5h16m   v1.24.1   linux   amd64
10.0.10.226   Ready    node    5h16m   v1.24.1   linux   amd64
10.0.10.39    Ready    node    5h16m   v1.24.1   linux   amd64

However, there are also many Kubernetes labels which are Cloud vendor specific and will return no data if queried on another Cloud, this can lead to operational and automation challenges.

Using the OKE console, I built a 6 node Kubernetes 1.24 Cluster in the UK-LONDON-1 Region, across the 3 Availability Domains (ADs).

Note: not every OKE Region has 3 ADs so you may need to confirm local availability.

I can determine where each worker nodes is running by using the Kubernetes labels OKE populated labels during the build.

For example using kubectl get nodes -L –label-columns and above labels.

rekins@rekins--MacBookPro15 PX % kubectl get nodes -L topology.kubernetes.io/region,topology.kubernetes.io/zone,oci.oraclecloud.com/fault-domain 
NAME          STATUS   ROLES   AGE     VERSION   REGION        ZONE               FAULT-DOMAIN
10.0.10.133   Ready    node    5h24m   v1.24.1   uk-london-1   UK-LONDON-1-AD-2   FAULT-DOMAIN-1
10.0.10.167   Ready    node    5h24m   v1.24.1   uk-london-1   UK-LONDON-1-AD-2   FAULT-DOMAIN-1
10.0.10.182   Ready    node    5h24m   v1.24.1   uk-london-1   UK-LONDON-1-AD-3   FAULT-DOMAIN-2
10.0.10.205   Ready    node    5h24m   v1.24.1   uk-london-1   UK-LONDON-1-AD-1   FAULT-DOMAIN-3
10.0.10.226   Ready    node    5h24m   v1.24.1   uk-london-1   UK-LONDON-1-AD-3   FAULT-DOMAIN-2
10.0.10.39    Ready    node    5h24m   v1.24.1   uk-london-1   UK-LONDON-1-AD-1   FAULT-DOMAIN-3 

Add Label(s)

We can add a single label to an object using kubectl label, for example to label a node.

rekins@rekins--MacBookPro15 PX % kubectl label node 10.0.10.133 px/enabled=true
node/10.0.10.133 labeled

For multiple labels provide a space delimited list, for example

rekins@rekins--MacBookPro15 PX % kubectl label node 10.0.10.167 px/enabled=true px/metadata-node=true
node/10.0.10.167 labeled

Show Label(s)

We can check the labels have been applied using –show-labels

rekins@rekins--MacBookPro15 PX % kubectl get node/10.0.10.133 --show-labels
NAME          STATUS   ROLES   AGE   VERSION   LABELS
10.0.10.133   Ready    node    59m   v1.24.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=VM.Standard.E3.Flex,beta.kubernetes.io/os=linux,displayName=oke-csqdad5zmma-nxao222j34a-sq4lna5wcma-0,failure-domain.beta.kubernetes.io/region=uk-london-1,failure-domain.beta.kubernetes.io/zone=UK-LONDON-1-AD-2,hostname=oke-csqdad5zmma-nxao222j34a-sq4lna5wcma-0,internal_addr=10.0.10.133,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.0.10.133,kubernetes.io/os=linux,last-migration-failure=get_kubesvc_failure,name=portworx,node-role.kubernetes.io/node=,node.info.ds_proxymux_client=true,node.info/compartment.id_prefix=ocid1.compartment.oc1,node.info/compartment.id_suffix=aaaaaaaa74sai3gxw4wikfatzj7t7rzniyhxuhpuvxsb4hahlbc5wwcbbcaa,node.info/compartment.name=PX-compartment,node.info/kubeletVersion=v1.24,node.kubernetes.io/instance-type=VM.Standard.E3.Flex,oci.oraclecloud.com/fault-domain=FAULT-DOMAIN-1,oke.oraclecloud.com/node.info.private_subnet=false,oke.oraclecloud.com/node.info.private_worker=true,oke.oraclecloud.com/tenant_agent.version=1.48.5-3e9306de62-942,px/enabled=true,topology.kubernetes.io/region=uk-london-1,topology.kubernetes.io/zone=UK-LONDON-1-AD-2

Or if you prefer with the -o / –output json

rekins@rekins--MacBookPro15 PX % kubectl get node 10.0.10.167 -o json                            
{
    "apiVersion": "v1",
    "kind": "Node",
    "metadata": {
       ...
        "labels": {
            "beta.kubernetes.io/arch": "amd64",
            "beta.kubernetes.io/instance-type": "VM.Standard.E3.Flex",
            "beta.kubernetes.io/os": "linux",
            "displayName": "oke-csqdad5zmma-nxao222j34a-sq4lna5wcma-3",
            "failure-domain.beta.kubernetes.io/region": "uk-london-1",
            "failure-domain.beta.kubernetes.io/zone": "UK-LONDON-1-AD-2",
            "hostname": "oke-csqdad5zmma-nxao222j34a-sq4lna5wcma-3",
            "internal_addr": "10.0.10.167",
            "kubernetes.io/arch": "amd64",
            "kubernetes.io/hostname": "10.0.10.167",
            "kubernetes.io/os": "linux",
            "last-migration-failure": "get_kubesvc_failure",
            "name": "portworx",
            "node-role.kubernetes.io/node": "",
            "node.info.ds_proxymux_client": "true",
            "node.info/compartment.id_prefix": "ocid1.compartment.oc1",
            "node.info/compartment.id_suffix": "aaaaaaaa74sai3gxw4wikfatzj7t7rzniyhxuhpuvxsb4hahlbc5wwcbbcaa",
            "node.info/compartment.name": "PX-compartment",
            "node.info/kubeletVersion": "v1.24",
            "node.kubernetes.io/instance-type": "VM.Standard.E3.Flex",
            "oci.oraclecloud.com/fault-domain": "FAULT-DOMAIN-1",
            "oke.oraclecloud.com/node.info.private_subnet": "false",
            "oke.oraclecloud.com/node.info.private_worker": "true",
            "oke.oraclecloud.com/tenant_agent.version": "1.48.5-3e9306de62-942",
            "px/enabled": "true",
            "px/metadata-node": "true",
            "topology.kubernetes.io/region": "uk-london-1",
            "topology.kubernetes.io/zone": "UK-LONDON-1-AD-2"
        },
...

Select Label(s)

Use -l or –selector= to filter objects to specific value, for example

rekins@rekins--MacBookPro15 PX % kubectl get nodes -l px/enabled=true 
NAME          STATUS   ROLES   AGE    VERSION
10.0.10.133   Ready    node    109m   v1.24.1
10.0.10.167   Ready    node    109m   v1.24.1

With multiple labels.

rekins@rekins--MacBookPro15 PX % kubectl get nodes -l px/enabled=true,px/metadata-node=true
NAME          STATUS   ROLES   AGE    VERSION
10.0.10.167   Ready    node    111m   v1.24.1

Or return objects that are not equal using != for example.

rekins@rekins--MacBookPro15 PX % kubectl get nodes --selector px/enabled!=true
NAME          STATUS   ROLES   AGE   VERSION
10.0.10.182   Ready    node    96m   v1.24.1
10.0.10.205   Ready    node    96m   v1.24.1
10.0.10.226   Ready    node    96m   v1.24.1
10.0.10.39    Ready    node    96m   v1.24.1

Update Label

If we try to re-add or update an existing label we can expect to see an error message informing us that the key value pair already exists and we need to use –overwrite to update.

rekins@rekins--MacBookPro15 PX % kubectl label node 10.0.10.167 px/enabled=flase                       
error: 'px/enabled' already has a value (true), and --overwrite is false

This time with the –overwrite option.

rekins@rekins--MacBookPro15 PX % kubectl label node 10.0.10.167 px/enabled=false --overwrite
node/10.0.10.167 labeled

Remove Label(s)

To remove a label we append a to the key, thus

rekins@rekins--MacBookPro15 PX % kubectl label node 10.0.10.133 px/enabled-       
node/10.0.10.133 unlabeled

Bulk Updates

As we have seen from above add, updating, and deleting labels is very simple, however what if we have many objects ?

The bash script below labels all nodes within my Kubernetes Cluster with px/enabled=true and px/metadata-node

for node in $(kubectl get nodes --output=jsonpath={.items..metadata.name}); 
do 
  echo Label $node for PX && kubectl label node $node px/enabled=true px/metadata-node=true --overwrite;  
done

Looking at the script, the first line returns the node name from the kubectl get nodes command.

rekins@rekins--MacBookPro15 PX % kubectl get nodes --output=jsonpath={.items..metadata.name}
10.0.10.133 10.0.10.167 10.0.10.182 10.0.10.205 10.0.10.226 10.0.10.39

Line 3 processes each node within the loop labeling each node with kubectl label node

We can confirm all my worker nodes have been labeled correctly using kubectl get nodes -l

rekins@rekins--MacBookPro15 PX % kubectl get nodes -l px/enabled=true,px/metadata-node=true           
NAME          STATUS   ROLES   AGE     VERSION
10.0.10.133   Ready    node    3h58m   v1.24.1
10.0.10.167   Ready    node    3h58m   v1.24.1
10.0.10.182   Ready    node    3h58m   v1.24.1
10.0.10.205   Ready    node    3h58m   v1.24.1
10.0.10.226   Ready    node    3h57m   v1.24.1
10.0.10.39    Ready    node    3h58m   v1.24.1

Summary

In the short post I have shared how to add, update and delete Kubernetes labels, I have demonstrated how we can use labels to filter results and also perform bulk updates.

Exit mobile version