Site icon Ron Ekins' – Oracle Technology, DevOps and Kubernetes Blog

Protect Kubernetes Portworx volumes with CloudSnap to OCI S3 Object Storage

In this post I will share how we can use Portworx CloudSnap to backup a Kubernetes Portworx volume to an Oracle Cloud Infrastructure (OCI) Object Storage S3 Bucket.

OCI Object Storage

Let’s start by logging into an Oracle Cloud Infrastructure (OCI) account, and from the Dashboard selecting Object Storage or via the hamburger menu navigate to Storage -> Object Storage -> Buckets.

Create Bucket

Select your required Compartment, and click Create Bucket

Object Storage & Archive Storage

Provide a Bucket Name, and confirm storage options, for this post I will use the Standard Default Storage Tier. OCI also provides an archive storage tier for long-term storage which may be of interest as it is one-tenth of the cost of Standard object storage.

Buckets in Compartment

Note: Bucket names must be unique, if you have used the name even in another Compartment expect to see an error message.

By default an OCI Object Storage bucket is Private, however if required you can change the visibility by clicking on the 3 vertical dots on the right hand side and updating it to Public.

Customer Secret Key

Before we can backup our Portworx Volume to OCI we need to generate authentication details.

To do this from the OCI dashboard, Click on your Profile icon (top right), navigate to User Settings -> Customer Secret Keys and click Generate Secret Key providing a descriptive name.

Customer Secret Keys

Enter secret key Name

Generate Secret Key

Once generated you will need to select Copy, this will be the s3_secret_key, which we will be using later.

Generate Secret Key

Now, click on the newly created Secret Name to obtain the s3_access_key, again save this for later.

Object Storage Namespace Name

From the OCI dashboard, Click on your Profile icon (top right), navigate to Tenancy

The Object Storage Namespace and default S3 compartments should be visible.

Tenancy information

Alternatively, you can use the OCI command line interface, this will be used within the s3-endpoint.

% oci os ns get                                                                                                                                    
{
  "data": "<Object Storage Namespace>"
}

OCI Region

The OCI region is visible from within the OCI console but we can also determine the region using the OCI command line interface, this will be the s3-region.

% oci iam region-subscription list 
{
  "data": [
    {
      "is-home-region": true,
      "region-key": "LHR",
      "region-name": "uk-london-1",
      "status": "READY"
    }
  ]
}

Kubernetes Cluster

Returning to our Kubernetes cluster setup an alias for pxctl as below, note in this example Portworx is installed in the portworx namespace.

[root@master-1 ~]# export PX_POD=$(kubectl get pods -l name=portworx -n portworx -o jsonpath='{.items[0].metadata.name}')

[root@master-1 ~]# alias pxctl='kubectl exec -n portworx ${PX_POD} -it -- /opt/pwx/bin/pxctl'

[root@master-1 ~]# pxctl -v
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
pxctl version 2.12.0-02bd5b0

Portworx OCI Credentials

Create a Porworx credentials using pxctl credentials create providing OCI access key, secret, region, endpoint, bucket name and credentials name, for example.

[root@master-1 ~]# pxctl credentials create \
--provider s3 \
--s3-access-key <OCI Access Key> \
--s3-secret-key <OCI Secret Key> \
--s3-region "uk-london-1" \
--s3-endpoint <OCI Object storage namespace>.compat.objectstorage.uk-london-1.oraclecloud.com  \
--bucket "PX-Bucket" \
px-oci-cred
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Credentials created successfully, UUID:16c1626d-c5d7-4ebd-a169-6f236de1a11e

Use pxctl credentials validate to check all is ok.

[root@master-1 ~]# pxctl credentials validate px-oci-cred
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Credential validated successfully

And pxctl credentials list to see details, for example.

[root@master-1 ~]# pxctl credentials list
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar

S3 Credentials
UUID						NAME			REGION			ENDPOINT								ACCESS KEY		SSL ENABLED		ENCRYPTION		BUCKET			WRITE THROUGHPUT (KBPS)
ffffffff-ffff-ffff-1111-ffffffffffff		px-oci-cred		uk-london-1		lrlr3o3lr8lr.compat.objectstorage.uk-london-1.oraclecloud.com		AAAAAAAAAAAAAAAAAAAA		true			false			PX-Bucket		11167

CloudSnap

We are now ready to perform a Portworx CloudSnap to our OCI Object Storage bucket.

Identify Volume

Using kubectl get pvc to identify required volume.

[root@master-1 ~]# kubectl get pvc
NAME   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pstg   Bound    pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c   8Gi        RWO            px-csi-db      24h

And pxctl volume list to confirm volume name.

[root@master-1 ~]# pxctl volume list
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
ID			NAME						SIZE	HA	SHARED	ENCRYPTED	PROXY-VOLUME	IO_PRIORITY	STATUS		SNAP-ENABLED	
971790877302845090	pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c	8 GiB	3	no	no		no		LOW		up - attached on 10.225.115.191	no

CloudSnap Backup

Use pxctl cloudsnap backup providing volume name and credentials to perform a CloudSnap to the target s3 bucket.

[root@master-1 ~]# pxctl cloudsnap backup pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c --cred-id px-oci-cred
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
Cloudsnap backup started successfully with id: b2c9f747-8c56-4e1b-982c-6fe86dcd6c54

CloudSnap Status

We can determine the status of the cloud backups with pxctl cloudsnap status, once complete the state will report Backup-Done.

[root@master-1 ~]# pxctl cloudsnap status --name b2c9f747-8c56-4e1b-982c-6fe86dcd6c54
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
NAME					SOURCEVOLUME		STATE		NODE		BYTES-PROCESSED	TIME-ELAPSED	ETA/COMPLETED	
b2c9f747-8c56-4e1b-982c-6fe86dcd6c54	971790877302845090	Backup-Done	10.225.115.190	4.1 GiB		36.692566911s	Tue, 13 Dec 2022 13:12:22 UTC

CloudSnap List

Use the pxctl cloudsnap list command to list the cloud backups

[root@master-1 ~]# pxctl cloudsnap list
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
SOURCEVOLUME						SOURCEVOLUMEID			CLOUD-SNAP-ID						CREATED-TIME				TYPE		STATUS		BELONGS-TO-CLUSTER		NAMESPACE
pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c		971790877302845090		PX-Bucket/971790877302845090-687196659438285600		Tue, 13 Dec 2022 13:11:54 UTC		Manual		Done		Yes	oracle-namespace

Oracle Object Storage

Returning to OCI and navigating to Object Storage -> Buckets, select bucket to see the newly created folder and objects.

Object Storage – Bucket Details

In the above we can see the OCI Object Storage Bucket Folder Name is the same as the pxctl cloudsnap list CLOUD-SNAP-ID


Summary

In this post I have shared how we can use Portworx CloudSnap backup to an Kubernetes Portworx volume to an Oracle Cloud Infrastructure (OCI) S3 Object Storage bucket.

The forever free Portworx Essentials licence is limited to a single CloudSnap of a volume per day, if you require more frequent CloudSnaps you will need to consider a Portworx Enterprise licence.

Exit mobile version