Over the last few years I have created a number of posts on deploying Oracle on Kubernetes including:
- Oracle 12c on a Kubernetes Cluster
- Oracle 12c on Minikube
- Oracle 18xe on a Kubernetes Cluster
- Oracle 19c on a Kubernetes Cluster
- Oracle 21c with the Oracle Kubernetes Operator
- and also how to install Portworx.
In this post I will share how we can now use the new Oracle 21c container image with Portworx storage.
Environment
For the post I will be deploying Oracle Enterprise Edition 21.3.0.0 on a 7 node Kubernetes 1.21.3 cluster running CentOS Linux 7 with Portworx 2.8 storage.
Kubernetes Version
[root@master-1 Oracle-on-Kubernetes]# kubectl version --short | awk -Fv '/Server Version: / {print $3}'
1.21.3
Kubernetes Nodes
[root@master-1 Oracle-on-Kubernetes]# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME master-1 Ready control-plane,master 9d v1.21.3 10.225.115.150 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1 node-1-1 Ready <none> 9d v1.21.3 10.225.115.151 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1 node-1-2 Ready <none> 9d v1.21.3 10.225.115.154 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1 node-1-3 Ready <none> 9d v1.21.3 10.225.115.155 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1 node-1-4 Ready <none> 9d v1.21.3 10.225.115.158 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1 node-1-5 Ready <none> 9d v1.21.3 10.225.115.157 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1 node-1-6 Ready <none> 9d v1.21.3 10.225.115.152 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1 node-1-7 Ready <none> 9d v1.21.3 10.225.115.153 <none> CentOS Linux 7 (Core) 3.10.0-1160.45.1.el7.x86_64 docker://1.13.1
Cluster Info
[root@master-1 Oracle-on-Kubernetes]# kubectl cluster-info Kubernetes control plane is running at https://10.225.115.150:6443 CoreDNS is running at https://10.225.115.150:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Portworx Version
[root@master-1 Oracle-on-Kubernetes]# PX_POD=$(kubectl get pods -l name=portworx -n portworx -o jsonpath='{.items[0].metadata.name}')
[root@master-1 Oracle-on-Kubernetes]# echo $PX_POD
px-deploy-1-26wmj
[root@master-1 Oracle-on-Kubernetes]# kubectl exec -n portworx ${PX_POD} -it -- /opt/pwx/bin/pxctl --version
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
pxctl version 2.8.1.2-c9b840b
Create Namespace
Create new Kubernetes namespace for ease of management, for example oracle-namespace.
We can create a new namespace with kubectl create namespace.
apiVersion: v1
kind: Namespace
metadata:
name: oracle-namespaceoracle-namespace.yaml
[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f oracle-namepace.yaml namespace/oracle-namespace created
Tip: To avoid having to specify our namespace with -n each time use the set-context to specify a default namespace.
[root@master-1 Oracle-on-Kubernetes]# kubectl config set-context --current --namespace=oracle-namespace Context "kubernetes-admin@kubernetes" modified.
Kubectl will now default to using the oracle-namespace the for example,
[root@master-1 Oracle-on-Kubernetes]# kubectl get pods No resources found in oracle-namespace namespace.
Create Secrets
For this post I will be using the official Oracle 21c Docker image which is available from Oracle Container Registry.
Please note: before you can pull this image you will need to accept the licence details and have a valid Oracle support account.
Logon to the Oracle Container Registry (OCR) using SSO credentials, for example
[root@master-1 Oracle-on-Kubernetes]# docker login container-registry.oracle.com Username: <SSO email> Password: <SSO password> Login Succeeded
Create Kubernetes secret using OCR credentials config.json file created by the previous step.
[root@master-1 Oracle-on-Kubernetes]# kubectl create secret generic regcred --from-file=.dockerconfigjson=$HOME/.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-namespace secret/regcred created
Create ConfigMap
For this Kubernetes build I will be using a ConfigMap to pass variables to my Oracle 21.3 container, this makes is very easy to change database name and passwords etc.
# Oracle 12.2
DB_SID=PSTG
DB_PDB=PSTGPDB1
DB_PASSWD=Kube#2020
DB_DOMAIN=localdomain
DB_BUNDLE=basic
DB_MEMORY=4g
# Oracle 18xe
ORACLE_CHARACTERSET=AL32UTF8
ORACLE_PWD=Kube#2020
# Oracle 19c
ORACLE_SID=ORCL
ORACLE_PDB=ORCLPDB1
ORACLE_EDITION=enterprise
ENABLE_ARCHIVELOG=true
# Optional
#INIT_SGA_SIZE=
#INIT_PGA_SIZE=oracle.properties
[root@master-1 Oracle-on-Kubernetes]# kubectl create configmap oradb --from-env-file=oracle.properties -n oracle-namespace configmap/oradb created
Create Storage Class
In this post, I will create a Portworx Storage Class that has a replication factor of 3 with I/O profile set to “db” and priority set to “high.”
This means that the storage will be optimized for low latency database workloads like Oracle and automatically placed on the highest performance storage available in the cluster.
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: px-ora-sc
provisioner: kubernetes.io/portworx-volume
parameters:
repl: "3"
io_profile: "db"
priority_io: "high"
allowVolumeExpansion: truepx-ora-sc.yaml
[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f px-ora-sc.yaml storageclass.storage.k8s.io/px-ora-sc created
We can view the Storage Class details using kubectl describe sc, for example
[root@master-1 Oracle-on-Kubernetes]# kubectl describe sc/px-ora-sc
Name: px-ora-sc
IsDefaultClass: No
Annotations: kubectl.kubernetes.io/last-applied-configuration={"allowVolumeExpansion":true,"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"px-ora-sc"},"parameters":{"io_profile":"db","priority_io":"high","repl":"3"},"provisioner":"kubernetes.io/portworx-volume"}
Provisioner: kubernetes.io/portworx-volume
Parameters: io_profile=db,priority_io=high,repl=3
AllowVolumeExpansion: True
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: Immediate
Events: <none>
Database Creation
We can now use kubectl apply to create an Oracle 21.3EE database on our Kubernetes.
[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f 21c_statefulset_PX.yaml statefulset.apps/oracle21c created service/oracle21c created
Statefulset
Use kubectl get statefulset to check status of the statefulset.
[root@master-1 Oracle-on-Kubernetes]# kubectl get statefulsets -o wide NAME READY AGE CONTAINERS IMAGES oracle21c 1/1 25s oracle21c container-registry.oracle.com/database/enterprise:21.3.0.0
Pods
Use kubectl get pods to get pod name and its status.
[root@master-1 Oracle-on-Kubernetes]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES oracle21c-0 1/1 Running 0 82s 10.244.1.7 node-1-3 <none> <none>
Now we know the pod name we can view / follow the log output during the build, for example
[root@master-1 Oracle-on-Kubernetes]# kubectl logs pods/oracle21c-0 --follow [2021:11:19 16:38:24]: Acquiring lock .ORCL.create_lck with heartbeat 30 secs [2021:11:19 16:38:24]: Lock acquired [2021:11:19 16:38:24]: Starting heartbeat [2021:11:19 16:38:24]: Lock held .ORCL.create_lck ORACLE EDITION: ENTERPRISE LSNRCTL for Linux: Version 21.0.0.0.0 - Production on 19-NOV-2021 16:38:24 Copyright (c) 1991, 2021, Oracle. All rights reserved. Starting /opt/oracle/product/21c/dbhome_1/bin/tnslsnr: please wait... TNSLSNR for Linux: Version 21.0.0.0.0 - Production System parameter file is /opt/oracle/homes/OraDB21Home1/network/admin/listener.ora Log messages written to /opt/oracle/diag/tnslsnr/oracle21c-0/listener/alert/log.xml Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1))) Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521))) Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1))) STATUS of the LISTENER ------------------------ Alias LISTENER Version TNSLSNR for Linux: Version 21.0.0.0.0 - Production Start Date 19-NOV-2021 16:38:25 Uptime 0 days 0 hr. 0 min. 0 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /opt/oracle/homes/OraDB21Home1/network/admin/listener.ora Listener Log File /opt/oracle/diag/tnslsnr/oracle21c-0/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521))) The listener supports no services The command completed successfully Prepare for db operation 8% complete Copying database files 31% complete Creating and starting Oracle instance 32% complete ... SQL> SQL> Disconnected from Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production Version 21.3.0.0.0 The Oracle base remains unchanged with value /opt/oracle Executing user defined scripts /opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/extensions/setup/savePatchSummary.sh /opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/extensions/setup/swapLocks.sh [2021:11:19 16:53:05]: Releasing lock .ORCL.create_lck [2021:11:19 16:53:05]: Lock released .ORCL.create_lck [2021:11:19 16:53:05]: Acquiring lock .ORCL.exist_lck with heartbeat 30 secs [2021:11:19 16:53:05]: Lock acquired [2021:11:19 16:53:05]: Starting heartbeat [2021:11:19 16:53:05]: Lock held .ORCL.exist_lck DONE: Executing user defined scripts The Oracle base remains unchanged with value /opt/oracle ######################### DATABASE IS READY TO USE! ######################### Executing user defined scripts /opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/extensions/startup/runDatapatch.sh Datafiles are already patched. Skipping datapatch run. ...
Persistent Volume Claims (PVCs)
[root@master-1 Oracle-on-Kubernetes]# kubectl get pvc -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE ora-data213-oracle21c-0 Bound pvc-f25fc9d0-7ce8-4bd5-9673-5120b9cc507c 25Gi RWO px-ora-sc 19m Filesystem ora-setup213-oracle21c-0 Bound pvc-9cf30d0a-79b6-4125-8d98-b5d3570d1da0 1Gi RWO px-ora-sc 19m Filesystem ora-startup213-oracle21c-0 Bound pvc-23f72131-a386-4407-ad82-fe75901d740d 1Gi RWO px-ora-sc 19m Filesystem
Services
[root@master-1 Oracle-on-Kubernetes]# kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
oracle21c NodePort 10.100.141.244 <none> 1521:31473/TCP,5500:32438/TCP 7m57s
[root@master-1 Oracle-on-Kubernetes]# kubectl get services oracle21c -o jsonpath={.spec.ports[?(@.port==1521)].nodePort}
31473
Database Connection
Let’s try and connect to our Oracle 21.3 instance using the external port number from my laptop.
rekins@rekins--MacBookPro15 ~ % sqlplus system/Kube#2020@//10.225.115.150:31473/ORCL @database_details SQL*Plus: Release 19.0.0.0.0 - Production on Fri Nov 19 17:16:37 2021 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Last Successful login time: Fri Nov 19 2021 17:16:04 +00:00 Connected to: Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production Version 21.3.0.0.0 Current Time : 19/11/21 17:16:42 Database Details =============================================== Hostname : oracle21c-0 Database Name : ORCL Date Created : 19/11/21 16:40:42 Date Started : 19/11/21 16:52:10 Resetlogs Date : 19/11/21 16:40:43 DB Status : OPEN Space Allocated: 1.98 GB Space Used : 1.94 GB
Oracle Enterprise Manager Database Express Console is also available from a browser using the external EM port number.
Clean-Up
Use kubectl delete to remove the statefulset
[root@master-1 Oracle-on-Kubernetes]# kubectl delete -f 21c_statefulset_PX.yaml statefulset.apps "oracle21c" deleted service "oracle21c" deleted
Use kubectl delete pvc -all to remove all Persistent Volume Claims
[root@master-1 Oracle-on-Kubernetes]# kubectl delete pvc --all -n oracle-namespace persistentvolumeclaim "ora-data213-oracle21c-0" deleted persistentvolumeclaim "ora-setup213-oracle21c-0" deleted persistentvolumeclaim "ora-startup213-oracle21c-0" deleted
Summary
In this post I have shared how we can deploy an Oracle 21.3 database on Kubernetes with Portworx storage.
If you want to try this for yourself you can find the manifests used within this post at my GitHub site.
[twitter-follow screen_name=’RonEkins’ show_count=’yes’]
Leave a Reply