In this post I will share how we can use Portworx CloudSnap to backup a Kubernetes Portworx volume to an Oracle Cloud Infrastructure (OCI) Object Storage S3 Bucket.
OCI Object Storage
Let’s start by logging into an Oracle Cloud Infrastructure (OCI) account, and from the Dashboard selecting Object Storage or via the hamburger menu navigate to Storage -> Object Storage -> Buckets.
Create Bucket
Select your required Compartment, and click Create Bucket
Provide a Bucket Name, and confirm storage options, for this post I will use the Standard Default Storage Tier. OCI also provides an archive storage tier for long-term storage which may be of interest as it is one-tenth of the cost of Standard object storage.
Note: Bucket names must be unique, if you have used the name even in another Compartment expect to see an error message.
By default an OCI Object Storage bucket is Private, however if required you can change the visibility by clicking on the 3 vertical dots on the right hand side and updating it to Public.
Customer Secret Key
Before we can backup our Portworx Volume to OCI we need to generate authentication details.
To do this from the OCI dashboard, Click on your Profile icon (top right), navigate to User Settings -> Customer Secret Keys and click Generate Secret Key providing a descriptive name.
Enter secret key Name
Once generated you will need to select Copy, this will be the s3_secret_key, which we will be using later.
Now, click on the newly created Secret Name to obtain the s3_access_key, again save this for later.
Object Storage Namespace Name
From the OCI dashboard, Click on your Profile icon (top right), navigate to Tenancy
The Object Storage Namespace and default S3 compartments should be visible.
Alternatively, you can use the OCI command line interface, this will be used within the s3-endpoint.
% oci os ns get
{
"data": "<Object Storage Namespace>"
}
OCI Region
The OCI region is visible from within the OCI console but we can also determine the region using the OCI command line interface, this will be the s3-region.
% oci iam region-subscription list
{
"data": [
{
"is-home-region": true,
"region-key": "LHR",
"region-name": "uk-london-1",
"status": "READY"
}
]
}
Kubernetes Cluster
Returning to our Kubernetes cluster setup an alias for pxctl as below, note in this example Portworx is installed in the portworx namespace.
[root@master-1 ~]# export PX_POD=$(kubectl get pods -l name=portworx -n portworx -o jsonpath='{.items[0].metadata.name}')
[root@master-1 ~]# alias pxctl='kubectl exec -n portworx ${PX_POD} -it -- /opt/pwx/bin/pxctl'
[root@master-1 ~]# pxctl -v
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
pxctl version 2.12.0-02bd5b0
Portworx OCI Credentials
Create a Porworx credentials using pxctl credentials create providing OCI access key, secret, region, endpoint, bucket name and credentials name, for example.
[root@master-1 ~]# pxctl credentials create \ --provider s3 \ --s3-access-key <OCI Access Key> \ --s3-secret-key <OCI Secret Key> \ --s3-region "uk-london-1" \ --s3-endpoint <OCI Object storage namespace>.compat.objectstorage.uk-london-1.oraclecloud.com \ --bucket "PX-Bucket" \ px-oci-cred Defaulted container "portworx" out of: portworx, csi-node-driver-registrar Credentials created successfully, UUID:16c1626d-c5d7-4ebd-a169-6f236de1a11e
Use pxctl credentials validate to check all is ok.
[root@master-1 ~]# pxctl credentials validate px-oci-cred Defaulted container "portworx" out of: portworx, csi-node-driver-registrar Credential validated successfully
And pxctl credentials list to see details, for example.
[root@master-1 ~]# pxctl credentials list Defaulted container "portworx" out of: portworx, csi-node-driver-registrar S3 Credentials UUID NAME REGION ENDPOINT ACCESS KEY SSL ENABLED ENCRYPTION BUCKET WRITE THROUGHPUT (KBPS) ffffffff-ffff-ffff-1111-ffffffffffff px-oci-cred uk-london-1 lrlr3o3lr8lr.compat.objectstorage.uk-london-1.oraclecloud.com AAAAAAAAAAAAAAAAAAAA true false PX-Bucket 11167
CloudSnap
We are now ready to perform a Portworx CloudSnap to our OCI Object Storage bucket.
Identify Volume
Using kubectl get pvc to identify required volume.
[root@master-1 ~]# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE pstg Bound pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c 8Gi RWO px-csi-db 24h
And pxctl volume list to confirm volume name.
[root@master-1 ~]# pxctl volume list Defaulted container "portworx" out of: portworx, csi-node-driver-registrar ID NAME SIZE HA SHARED ENCRYPTED PROXY-VOLUME IO_PRIORITY STATUS SNAP-ENABLED 971790877302845090 pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c 8 GiB 3 no no no LOW up - attached on 10.225.115.191 no
CloudSnap Backup
Use pxctl cloudsnap backup providing volume name and credentials to perform a CloudSnap to the target s3 bucket.
[root@master-1 ~]# pxctl cloudsnap backup pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c --cred-id px-oci-cred Defaulted container "portworx" out of: portworx, csi-node-driver-registrar Cloudsnap backup started successfully with id: b2c9f747-8c56-4e1b-982c-6fe86dcd6c54
CloudSnap Status
We can determine the status of the cloud backups with pxctl cloudsnap status, once complete the state will report Backup-Done.
[root@master-1 ~]# pxctl cloudsnap status --name b2c9f747-8c56-4e1b-982c-6fe86dcd6c54 Defaulted container "portworx" out of: portworx, csi-node-driver-registrar NAME SOURCEVOLUME STATE NODE BYTES-PROCESSED TIME-ELAPSED ETA/COMPLETED b2c9f747-8c56-4e1b-982c-6fe86dcd6c54 971790877302845090 Backup-Done 10.225.115.190 4.1 GiB 36.692566911s Tue, 13 Dec 2022 13:12:22 UTC
CloudSnap List
Use the pxctl cloudsnap list command to list the cloud backups
[root@master-1 ~]# pxctl cloudsnap list Defaulted container "portworx" out of: portworx, csi-node-driver-registrar SOURCEVOLUME SOURCEVOLUMEID CLOUD-SNAP-ID CREATED-TIME TYPE STATUS BELONGS-TO-CLUSTER NAMESPACE pvc-4f2e08d1-43b6-457f-9bc9-797ed9a1ff2c 971790877302845090 PX-Bucket/971790877302845090-687196659438285600 Tue, 13 Dec 2022 13:11:54 UTC Manual Done Yes oracle-namespace
Oracle Object Storage
Returning to OCI and navigating to Object Storage -> Buckets, select bucket to see the newly created folder and objects.
In the above we can see the OCI Object Storage Bucket Folder Name is the same as the pxctl cloudsnap list CLOUD-SNAP-ID
Summary
In this post I have shared how we can use Portworx CloudSnap backup to an Kubernetes Portworx volume to an Oracle Cloud Infrastructure (OCI) S3 Object Storage bucket.
The forever free Portworx Essentials licence is limited to a single CloudSnap of a volume per day, if you require more frequent CloudSnaps you will need to consider a Portworx Enterprise licence.