In this post I will share how you can get started with containerising Oracle databases and how to use a Kubernetes Container Storage Interface (CSI) driver to provision dynamic persistent storage to the container.
For this post I will use Oracle 18c Express Edition (XE) the ‘Free Database for Everyone’ and Portworx Essentials Forever Free edition.
Docker Build scripts
Let’s start by logging onto a private registry server and pulling the latest Oracle container build script from GitHub.
# git clone https://github.com/oracle/docker-images.git Cloning into 'docker-images'... remote: Enumerating objects: 14145, done. remote: Counting objects: 100% (159/159), done. remote: Compressing objects: 100% (122/122), done. remote: Total 14145 (delta 56), reused 91 (delta 28), pack-reused 13986 Receiving objects: 100% (14145/14145), 9.92 MiB | 1.46 MiB/s, done. Resolving deltas: 100% (8326/8326), done.
Note: the build script requires a minimum docker version of 17.09, so before we begin it’s best to check the Docker version, for example:
# docker version --format '{{.Server.Version | printf "%.5s" }}'
20.10
Oracle 18c XE Docker image
Navigate to the container build script just downloaded and execute the buildContainerImage script.
# cd docker-images/OracleDatabase/SingleInstance/dockerfiles
Note: if version 18.4.0 is specified the build script automatically downloads the free Oracle 18c Express Edition rpm. https://download.oracle.com/otn-pub/otn_software/db-express/oracle-database-xe-18c-1.0-1.x86_64.rpm
# ./buildContainerImage.sh -v 18.4.0 -x
...
Successfully built 78adacd54ec5
Successfully tagged oracle/database:18.4.0-xe
Oracle Database container image for 'xe' version 18.4.0 is ready to be extended:
--> oracle/database:18.4.0-xe
Build completed in 828 seconds.
The build script automates the creation of a container, however depending upon your network you may need to be patient as the download is ~5.5GB.
# docker images oracle/database:18.4.0-xe
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/database 18.4.0-xe 78adacd54ec5 5 minutes ago 5.89GB
List repositories
To assist with the sharing of the image I have set-up a private registry, as you can see from the below it’s currently empty.
# curl -X GET http://10.225.115.68:5000/v2/_catalog
{"repositories":[]}
Tag the image
Before we can push the image to the repository we need to tag it.
# docker tag 78adacd54ec5 10.225.115.68:5000/oracledb:18.4.0-xe
Now push the image into the private registry
# docker push 10.225.115.68.:5000/oracledb:18.4.0-xe The push refers to repository [10.225.115.68:5000/oracledb] d8873d52115d: Pushed 6c6f43805655: Pushed 59759852752f: Pushed 18.4.0-xe: digest: sha256:a0bcdaf2afbd03dd2560ad84d4d3d9cd5600a0608530136eeccade61a9f1d10f size: 951
Confirm registry
Before moving on let’s confirm that the Kubernetes master can access the repository.
[root@master-1 Oracle-on-Kubernetes]# curl -X GET http://10.225.115.68:5000/v2/_catalog
{"repositories":["oracledb"]}
[root@master-1 Oracle-on-Kubernetes]# curl -X GET http://10.225.115.68:5000/v2/oracledb/tags/list
{"name":"oracledb","tags":["18.4.0-xe"]}
We can also use curl to review detailed information about the image, including variables, mount points, directory structures and scripts used with the image.
[root@master-1 Oracle-on-Kubernetes]# curl -X GET http://10.225.115.68:5000/v2/oracledb/manifests/18.4.0-xe
{
"schemaVersion": 1,
"name": "oracledb",
"tag": "18.4.0-xe",
"architecture": "amd64",
"fsLayers": [
...
Oracle 18c XE database
Pull manifest files.
If you want to follow this post or just interested in the Kubernetes manifest files you can find them here.
[root@master-1 Oracle-on-Kubernetes]# git clone https://github.com/PureStorage-OpenConnect/Oracle-on-Kubernetes.git Cloning into 'Oracle-on-Kubernetes'... remote: Enumerating objects: 67, done. remote: Counting objects: 100% (67/67), done. remote: Compressing objects: 100% (60/60), done. remote: Total 67 (delta 36), reused 13 (delta 6), pack-reused 0 Unpacking objects: 100% (67/67), done.
[root@master-1 Oracle-on-Kubernetes]# cd Oracle-on-Kubernetes
Create dedicated namespace
If you don’t already have an oracle namespace, time to create one.
[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f oracle-namepace.yaml namespace/oracle-namespace created
Create configmap
And again, create the common configmap if you have not previously followed my posts.
[root@master-1 Oracle-on-Kubernetes]# kubectl create configmap oradb --from-env-file=oracle.properties -n oracle-namespace configmap/oradb created
Dynamic Persistent Volume Claims
Before we create our database let’s look at how we can tell Kubernetes to dynamically create the required persistent volumes.
The Oracle 18x XE container supports three mount points oradata, setup and startup.
In the picture above you can see a POD with the three VolumeMounts and corresponding Volumes.
Volume Mounts
For ease of reading we will focus on the oradata volume only, as the setup is the same for setup and startup.
Our Oracle datafiles will be stored in a mountPath of /opt/oracle/oradata this has a name of ora-data184.
volumeMounts: - name: ora-data184 mountPath: "/opt/oracle/oradata"
Volumes
The volumes references the persistent volume claim name, for example ora-data184-claim
volumes: - name: ora-data184 persistentVolumeClaim: claimName: ora-data184-claim
Persistent Volume Claim
The persistent volume claim requests storage from the storage class.
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: ora-data184-claim labels: app: database version: 18.4.0.1 spec: storageClassName: px-ora-csi-sc accessModes: - ReadWriteOnce resources: requests: storage: 10Gi
From the above we can see that the persistent volume claim ora-data-184-claim is using a storageClassName of px-ora-cs-cs
StorageClass
The Portworx CSI storage class dynamically provisions the physical storage.
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: px-ora-csi-sc provisioner: pxd.portworx.com parameters: repl: "1" allowVolumeExpansion: true reclaimPolicy: Delete
Oracle 18c XE database
We are now ready to create a containerised Oracle 18c XE database.
[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f 18xe_deployment_PX-CSI.yaml storageclass.storage.k8s.io/px-ora-csi-sc created deployment.apps/oracle18xe created persistentvolumeclaim/ora-data184-claim created persistentvolumeclaim/ora-setup184-claim created persistentvolumeclaim/ora-startup184-claim created service/oracle18xe created
Deployment
Use kubectl get deployment or deploy to list deployment and check image, for example.
[root@master-1 Oracle-on-Kubernetes]# kubectl get deploy -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR oracle18xe 1/1 1 1 66s oracle18xe 10.225.115.68:5000/oracledb:18.4.0-xe app=database,version=18.4.0.1
[root@master-1 Oracle-on-Kubernetes]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES oracle18xe-bc86774b6-dvtv7 1/1 Running 0 2m35s 10.244.1.24 node-1-2 <none> <none>
Docker Image
We can also check the correct image has been pulled by logging onto the worker node and confirming the image id.
[root@node-1-2 ~]# docker images 10.225.115.68:5000/oracledb REPOSITORY TAG IMAGE ID CREATED SIZE 10.225.115.68:5000/oracledb 18.4.0-xe 78adacd54ec5 2 days ago 5.89 GB
Build Log
Use kubectl logs to check progress, the –follow option provides a useful way of watching the build.
The 18c XE image does not support database or pluggable name parameters, and creates a database called XE and PDB XEPDB1.
[root@master-1 Oracle-on-Kubernetes]# kubectl logs pods/oracle18xe-bc86774b6-dvtv7 ORACLE PASSWORD FOR SYS AND SYSTEM: Kube#2020 Specify a password to be used for database accounts. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9]. Note that the same password will be used for SYS, SYSTEM and PDBADMIN accounts: Confirm the password: Configuring Oracle Listener. Listener configuration succeeded. Configuring Oracle Database XE. Enter SYS user password: ******** Enter SYSTEM user password: *********** Enter PDBADMIN User Password: ********** Prepare for db operation 7% complete Copying database files 29% complete Creating and starting Oracle instance 30% complete 31% complete 34% complete 38% complete 41% complete 43% complete Completing Database Creation 47% complete 50% complete Creating Pluggable Databases 54% complete 71% complete Executing Post Configuration Actions 93% complete Running Custom Scripts 100% complete Database creation complete. For details check the logfiles at: /opt/oracle/cfgtoollogs/dbca/XE. Database Information: Global Database Name:XE System Identifier(SID):XE Look at the log file "/opt/oracle/cfgtoollogs/dbca/XE/XE.log" for further details. Connect to Oracle Database using one of the connect strings: Pluggable database: oracle18xe-bc86774b6-dvtv7/XEPDB1 Multitenant container database: oracle18xe-bc86774b6-dvtv7 Use https://localhost:5500/em to access Oracle Enterprise Manager for Oracle Database XE The Oracle base remains unchanged with value /opt/oracle ######################### DATABASE IS READY TO USE! ######################### The following output is now a tail of the alert.log: 2021-07-05T11:42:11.428709+00:00 XEPDB1(3):Resize operation completed for file# 10, old size 358400K, new size 368640K 2021-07-05T11:42:14.274158+00:00 XEPDB1(3):CREATE SMALLFILE TABLESPACE "USERS" LOGGING DATAFILE '/opt/oracle/oradata/XE/XEPDB1/users01.dbf' SIZE 5M REUSE AUTOEXTEND ON NEXT 1280K MAXSIZE UNLIMITED EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO XEPDB1(3):Completed: CREATE SMALLFILE TABLESPACE "USERS" LOGGING DATAFILE '/opt/oracle/oradata/XE/XEPDB1/users01.dbf' SIZE 5M REUSE AUTOEXTEND ON NEXT 1280K MAXSIZE UNLIMITED EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO XEPDB1(3):ALTER DATABASE DEFAULT TABLESPACE "USERS" XEPDB1(3):Completed: ALTER DATABASE DEFAULT TABLESPACE "USERS" 2021-07-05T11:42:15.659094+00:00 ALTER PLUGGABLE DATABASE XEPDB1 SAVE STATE Completed: ALTER PLUGGABLE DATABASE XEPDB1 SAVE STATE
Database Container
If you’re interested to see what our Oracle container looks like, we can shell into our database container using kubectl pods/<pod name> exec -it — /bin/bash we can see the persistent volumes presented to the container.
bash-4.2# df -Th
Filesystem Type Size Used Avail Use% Mounted on
overlay overlay 49G 14G 36G 27% /
tmpfs tmpfs 3.9G 0 3.9G 0% /dev
tmpfs tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
tmpfs tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/sda2 xfs 49G 14G 36G 27% /etc/hosts
/dev/pxd/pxd830774027675457210 ext4 9.8G 2.6G 6.7G 28% /opt/oracle/oradata
/dev/pxd/pxd1087869865371744604 ext4 976M 2.5M 907M 1% /opt/oracle/scripts/startup
/dev/pxd/pxd730447982461238045 ext4 976M 2.5M 907M 1% /opt/oracle/scripts/setup
tmpfs tmpfs 3.9G 12K 3.9G 1% /run/secrets/kubernetes.io/serviceaccount
tmpfs tmpfs 3.9G 0 3.9G 0% /proc/acpi
tmpfs tmpfs 3.9G 0 3.9G 0% /proc/scsi
tmpfs tmpfs 3.9G 0 3.9G 0% /sys/firmware
Persistent Volume Claims
Use kubectl get persistentvolumeclaims or pvc to list the volumes used.
To limit the number of persistent volumes returned consider using filtering with labels -l, for example.
[root@master-1 Oracle-on-Kubernetes]# kubectl get pvc -l version=18.4.0.1 NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE ora-data184-claim Bound pvc-bae9dc20-6444-4b40-b703-d97c9d08b552 10Gi RWO px-ora-csi-sc 81s ora-setup184-claim Bound pvc-eff35d27-87af-4543-9150-fc29cbe4b9b9 1Gi RWO px-ora-csi-sc 81s ora-startup184-claim Bound pvc-ed8a0061-6830-45e6-97fe-73fb8debf7c1 1Gi RWO px-ora-csi-sc 81s
Portworx Volumes
We can also see the three Portworx-managed volumes by using pxctl volume list with -v for volumes and -l to filter returned rows, for example.
[root@master-1 Oracle-on-Kubernetes]# pxctl volume list -v -l version=18.4.0.1 Defaulting container name to portworx. Use 'kubectl describe pod/px-deploy-1-g2rnm -n kube-system' to see all of the containers in this pod. ID NAME SIZE HA SHARED ENCRYPTED PROXY-VOLUME IO_PRIORITY STATUS SNAP-ENABLED 830774027675457210 pvc-bae9dc20-6444-4b40-b703-d97c9d08b552 10 GiB 1 no no no HIGH up - attached on 10.225.115.69 no 730447982461238045 pvc-ed8a0061-6830-45e6-97fe-73fb8debf7c1 1 GiB 1 no no no HIGH up - attached on 10.225.115.69 no 1087869865371744604 pvc-eff35d27-87af-4543-9150-fc29cbe4b9b9 1 GiB 1 no no no HIGH up - attached on 10.225.115.69 no
By using pxctl volume inspect <volume> we can see the Device Path visible from within the container, the pod using the volume as well as mount and usage details.
[root@master-1 Oracle-on-Kubernetes]# pxctl volume inspect pvc-bae9dc20-6444-4b40-b703-d97c9d08b552 Defaulting container name to portworx. Use 'kubectl describe pod/px-deploy-1-g2rnm -n kube-system' to see all of the containers in this pod. Volume : 830774027675457210 Name : pvc-bae9dc20-6444-4b40-b703-d97c9d08b552 Size : 10 GiB Format : ext4 HA : 1 IO Priority : HIGH Creation time : Jul 5 13:39:33 UTC 2021 Shared : no Status : up State : Attached: f9f9a668-499f-432a-966a-01c36ec7858c (10.225.115.80) Last Attached : Jul 5 13:55:35 UTC 2021 Device Path : /dev/pxd/pxd830774027675457210 Labels : app=database,namespace=oracle-namespace,pvc=ora-data184-claim,repl=1,version=18.4.0.1 Mount Options : discard Reads : 6629 Reads MS : 59970 Bytes Read : 523563008 Writes : 12893 Writes MS : 73831 Bytes Written : 142086144 IOs in progress : 0 Bytes used : 3.4 GiB Replica sets on nodes: Set 0 Node : 10.225.115.81 (Pool 111493ba-d0ea-45bc-84be-23d694a8fadd ) Replication Status : Up Volume consumers : - Name : oracle18xe-fc75dff95-92kq4 (0b7fc853-7334-4c4d-b69e-aa084897f96e) (Pod) Namespace : oracle-namespace Running on : node-1-4 Controlled by : oracle18xe-fc75dff95 (ReplicaSet)
Summary
In this blog post I have shared how we can download, build and deploy an Oracle 18c Express Edition (XE) database on Kubernetes using the Portworx Container Storage Interface (CSI) driver to deliver dynamic persistent storage.
The Container Storage Interface can do more than just provision storage, in my next posts I will share how we can also take snapshots and perform clones.
[twitter-follow screen_name=’RonEkins’ show_count=’yes’]