Site icon Ron Ekins' – Oracle Technology, DevOps and Kubernetes Blog

Portworx Cloud Drive Support for Oracle Container Engine for Kubernetes (OKE)

Portworx 2.13 brings support for Cloud Drives to Oracle Container Engine for Kubernetes (OKE), Cloud Drives provide the ability to automatically provision storage, this is great news for anyone using Portworx on OKE as it means you no longer need to pre-allocate block storage.

I have previously shared how to install Portworx on the Oracle Cloud Infrastructure (OCI) managed Kubernetes service Oracle Container Engine for Kubernetes (OKE)

This involved pre-creating OCI Block Volumes and attaching them to OKE Worker nodes, my Portworx 2.8 installation on Oracle Kubernetes Engine post and Portworx Essentials installation on Oracle Kubernetes Engine provide detailed walkthroughs on the required steps.

I have also previously provided a fully automated Terraform deployment detailed in a post Automated OKE deployment with Portworx using Terraform, so it looks like I will have to revisit some of these.

So, let’s see how much simpler it is to deploy Portworx with Cloud Drives.

Oracle Kubernetes Engine (OKE) Cluster

For this blog post I will be using an 3 node v1.24.4 Kubernetes cluster which I have created using the OKE Wizard within the uk-london-1 region, as below.

% kubectl get nodes -L,
NAME          STATUS   ROLES   AGE    VERSION   REGION        ZONE   Ready    node    2m3s   v1.25.4   uk-london-1   UK-LONDON-1-AD-2    Ready    node    2m9s   v1.25.4   uk-london-1   UK-LONDON-1-AD-3    Ready    node    2m2s   v1.25.4   uk-london-1   UK-LONDON-1-AD-1

OKE Storage Classes

OCI provides two Kubernetes storage classes oci, and oci-bv, the oci block volume (oci-bv) is now the default and is used by the CSI volume plugin. The oci class is considered legacy and is only used by the FlexVolume plugin.

% kubectl get sc 
oci                          Delete          Immediate              false                  5m47s
oci-bv (default)   Delete          WaitForFirstConsumer   true                   5m47s

Portworx Namespace

Before we do anything let’s create a dedicated namespace for Portworx using kubectl create ns.

% kubectl create ns portworx
namespace/portworx created

Portworx OKE Secret

To install the Portworx Operator we need to create a Kubernetes secret to allow Portworx to manage OCI block storage using our Oracle API signing key and fingerprint details, for example.

kubectl create secret generic ociapikey \
--namespace <namespace> \
--from-file=oci_api_key.pem=/Users/rekins/.oci/oci_api_key.pem \
--from-literal=PX_ORACLE_user_ocid="<ocid>" \

Check the secret has been created in the correct namespace.

% kubectl get secret -n portworx                
NAME        TYPE     DATA   AGE
ociapikey   Opaque   3      9s


Visit and logon, select Portworx Enterprise, and then select Portworx Essentials or Portworx Enterprise.

We can see the spec generator now includes Oracle as a Cloud option, select this.

Generate Spec

Confirm Kubernetes Namespace

Review Summary and click Show More if required, take special notice of Storage Size and VPUs Count and set to match application performance requirements.

Performance Characteristics

Volume Size and Performance

Complete the wizard and grab the Portworx deployment commands.

Deploy Operator

To install the Portworx Operator using kubectl apply -f ‘<portworx version>?comp=pxoperator=<Kubernetes Version>&ns=portworx’ for example.

% kubectl apply -f ''
serviceaccount/portworx-operator created created created
deployment.apps/portworx-operator created

Deploy StorageCluster

To deploy the StorageCluster use the second command provide by PX-Central

% kubectl apply -f '' created

Storage Class

Using kubectl get sc we can see a number of new Storage Classes have been created by Portworx.

% kubectl get sc              
NAME                                 PROVISIONER                       RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
oci                                            Delete          Immediate              false                  166m
oci-bv (default)              Delete          WaitForFirstConsumer   true                   166m
px-csi-db                                    Delete          Immediate              true                   2m13s
px-csi-db-cloud-snapshot                     Delete          Immediate              true                   2m12s
px-csi-db-cloud-snapshot-encrypted                  Delete          Immediate              true                   2m12s
px-csi-db-encrypted                          Delete          Immediate              true                   2m13s
px-csi-db-local-snapshot                     Delete          Immediate              true                   2m12s
px-csi-db-local-snapshot-encrypted                  Delete          Immediate              true                   2m12s
px-csi-replicated                            Delete          Immediate              true                   2m13s
px-csi-replicated-encrypted                  Delete          Immediate              true                   2m12s
px-db                           Delete          Immediate              true                   2m13s
px-db-cloud-snapshot            Delete          Immediate              true                   2m13s
px-db-cloud-snapshot-encrypted     Delete          Immediate              true                   2m13s
px-db-encrypted                 Delete          Immediate              true                   2m13s
px-db-local-snapshot            Delete          Immediate              true                   2m13s
px-db-local-snapshot-encrypted     Delete          Immediate              true                   2m13s
px-replicated                   Delete          Immediate              true                   2m13s
px-replicated-encrypted         Delete          Immediate              true                   2m13s
stork-snapshot-sc                    stork-snapshot                    Delete          Immediate              true                   2m19s

pxctl status

Using pxctl status we can see a summary of the deployment.

% PX_POD=$(kubectl get pods -l name=portworx -n portworx -o jsonpath='{.items[0]}')
% kubectl exec -it $PX_POD -n portworx -- /opt/pwx/bin/pxctl status                             
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Status: PX is operational
Telemetry: Disabled or Unhealthy
Metering: Disabled or Unhealthy
License: Trial (expires in 31 days)
Node ID: 93b4f96a-292c-4e1a-8184-0399288ff823
 	Local Storage Pool: 1 pool
	0	HIGH		raid0		150 GiB	9.5 GiB	Online	UK-LONDON-1-AD-2	uk-london-1
	Local Storage Devices: 1 device
	Device	Path		Media Type		Size		Last-Scan
	0:1	/dev/sdb	STORAGE_MEDIUM_SSD	150 GiB		27 Feb 23 16:37 UTC
	total			-			150 GiB
	Cache Devices:
	 * No cache devices
	Kvdb Device:
	Device Path	Size
	/dev/sdc	150 GiB
	 * Internal kvdb on this node is using this dedicated kvdb device to store its data.
Cluster Summary
	Cluster ID: px-cluster-888de70e-bdc2-4761-99db-7c306df72e97
	Cluster UUID: 9354debc-188a-4bf6-8470-015daf6af996
	Scheduler: kubernetes
	Nodes: 3 node(s) with storage (3 online)
	IP		ID					SchedulerNodeName	Auth		StorageNode	Used	Capacity	StatuStorageStatus	Version		Kernel					OS	c4b3011d-0a65-433c-869f-176b0f0bdacc		Disabled	Yes		9.5 GiB	150 GiB		OnlinUp		2.13.0-9793db5	5.4.17-2136.314.6.2.el8uek.x86_64	Oracle Linux Server 8.6	93b4f96a-292c-4e1a-8184-0399288ff823		Disabled	Yes		9.5 GiB	150 GiB		OnlinUp (This node)	2.13.0-9793db5	5.4.17-2136.314.6.2.el8uek.x86_64	Oracle Linux Server 8.6	6bbf1b2c-054e-4739-a464-7b2669595cad		Disabled	Yes		9.5 GiB	150 GiB		OnlinUp		2.13.0-9793db5	5.4.17-2136.314.6.2.el8uek.x86_64	Oracle Linux Server 8.6
Global Storage Pool
	Total Used    	:  29 GiB
	Total Capacity	:  450 GiB

pxctl cluster

The pxctl cluster provision-status command lists the Portworx storage pools in the cluster.

% pxctl cluster provision-status
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
6bbf1b2c-054e-4739-a464-7b2669595cad	Up		0 ( 829b1d90-20cb-450c-85fa-422c620faa93 )	Online		HIGH		150 GiB	140 GiB		9.5 GiB	0 B		UK-LONDON-1-AD-3	uk-london-1	default
93b4f96a-292c-4e1a-8184-0399288ff823	Up		0 ( 41f1e1a9-a00f-4a04-bb5e-506dc90453f2 )	Online		HIGH		150 GiB	140 GiB		9.5 GiB	0 B		UK-LONDON-1-AD-2	uk-london-1	default
c4b3011d-0a65-433c-869f-176b0f0bdacc	Up		0 ( 75bd75e6-14ee-4689-a59e-5482ca878c18 )	Online		HIGH		150 GiB	140 GiB		9.5 GiB	0 B		UK-LONDON-1-AD-1	uk-london-1	default

pxctl clouddrive

The pxctl clouddrive command is useful for getting more insight into the OCI Block volumes provisioned by Portworx, including the OCID for each block volume.

% pxctl clouddrive list
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Cloud Drives Summary
	Number of nodes in the cluster:  3
	Number of storage nodes:  3
	List of storage nodes:  [6bbf1b2c-054e-4739-a464-7b2669595cad 93b4f96a-292c-4e1a-8184-0399288ff823 c4b3011d-0a65-433c-869f-176b0f0bdacc]
	List of storage less nodes:  []

Drive Set List
	NodeID					InstanceID											Zone	State	Drive IDs	
	6bbf1b2c-054e-4739-a464-7b2669595cad	iXvO:UK-LONDON-1-AD-3	In Use,
	93b4f96a-292c-4e1a-8184-0399288ff823	iXvO:UK-LONDON-1-AD-2	In Use,
	c4b3011d-0a65-433c-869f-176b0f0bdacc	iXvO:UK-LONDON-1-AD-1	In Use,

oci bv

Portworx has automatically created a block volume in each of my three Availability Domains and attached it to the worker nodes as per the diagram below.

Use oci bv volume list command with –lifecyle-state “Available” to list OCI Block Volumes in a JSON format, for example to see only display-name.

% oci bv volume list --compartment-id <compartment id> --lifecycle-state "Available" | grep display-name
      "display-name": "PX-DO-NOT-DELETE-px-cluster-888de70e-bdc2-4761-99db-7c306df72e97-b283075a-9654-465e-ad98-2ec24ddbf964",
      "display-name": "PX-DO-NOT-DELETE-px-cluster-888de70e-bdc2-4761-99db-7c306df72e97-00766bb2-7b80-495f-9e0e-efef58093e8b",
      "display-name": "PX-DO-NOT-DELETE-px-cluster-888de70e-bdc2-4761-99db-7c306df72e97-75797196-bfe2-42ae-ac44-81ca5779c919",
      "display-name": "PX-DO-NOT-DELETE-px-cluster-888de70e-bdc2-4761-99db-7c306df72e97-43ba8ca6-29a0-4d24-8510-c7b4d86cdb5c",
      "display-name": "PX-DO-NOT-DELETE-px-cluster-888de70e-bdc2-4761-99db-7c306df72e97-960dd658-7fe0-4d58-8e11-50d0fd1bd1c9",
      "display-name": "PX-DO-NOT-DELETE-px-cluster-888de70e-bdc2-4761-99db-7c306df72e97-6a198ba1-f5c7-4a4f-b6a8-e53faa11544e",

Alternatively, return to the OCI console to see the newly created Block Volumes

Block Volumes

And navigate to the compute instance, to confirm the Block Volume, Device Path, how it’s attached, Access mode, Size and volume performance (VPU).

Attached block volumes


In this post I have shared how to install Porworx Enterprise 2.13 on OKE, and demonstrated the automated provisioning of OCI Block Storage which is now possible du to the support of Portworx Cloud Drives.

Exit mobile version