Getting started with Oracle 21c on Kubernetes with Portworx

Over the last few years I have created a number of posts on deploying Oracle on Kubernetes including:

In this post I will share how we can now use the new Oracle 21c container image with Portworx storage.

Oracle 21c Docker Image

Environment

For the post I will be deploying Oracle Enterprise Edition 21.3.0.0 on a 7 node Kubernetes 1.21.3 cluster running CentOS Linux 7 with Portworx 2.8 storage.

Kubernetes Version

[root@master-1 Oracle-on-Kubernetes]# kubectl version --short | awk -Fv '/Server Version: / {print $3}'
1.21.3

Kubernetes Nodes

[root@master-1 Oracle-on-Kubernetes]# kubectl get nodes -o wide
NAME       STATUS   ROLES                  AGE   VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME
master-1   Ready    control-plane,master   9d    v1.21.3   10.225.115.150   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1
node-1-1   Ready    <none>                 9d    v1.21.3   10.225.115.151   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1
node-1-2   Ready    <none>                 9d    v1.21.3   10.225.115.154   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1
node-1-3   Ready    <none>                 9d    v1.21.3   10.225.115.155   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1
node-1-4   Ready    <none>                 9d    v1.21.3   10.225.115.158   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1
node-1-5   Ready    <none>                 9d    v1.21.3   10.225.115.157   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1
node-1-6   Ready    <none>                 9d    v1.21.3   10.225.115.152   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1
node-1-7   Ready    <none>                 9d    v1.21.3   10.225.115.153   <none>        CentOS Linux 7 (Core)   3.10.0-1160.45.1.el7.x86_64   docker://1.13.1

Cluster Info

[root@master-1 Oracle-on-Kubernetes]# kubectl cluster-info
Kubernetes control plane is running at https://10.225.115.150:6443
CoreDNS is running at https://10.225.115.150:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Portworx Version

[root@master-1 Oracle-on-Kubernetes]# PX_POD=$(kubectl get pods -l name=portworx -n portworx -o jsonpath='{.items[0].metadata.name}')

[root@master-1 Oracle-on-Kubernetes]# echo $PX_POD
px-deploy-1-26wmj

[root@master-1 Oracle-on-Kubernetes]# kubectl exec -n portworx ${PX_POD}  -it -- /opt/pwx/bin/pxctl --version
Defaulted container "portworx" out of: portworx, csi-node-driver-registrar
pxctl version 2.8.1.2-c9b840b

Create Namespace

Create new Kubernetes namespace for ease of management, for example oracle-namespace.

We can create a new namespace with kubectl create namespace.

apiVersion: v1
kind: Namespace
metadata:
  name: oracle-namespace

oracle-namespace.yaml

[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f oracle-namepace.yaml 
namespace/oracle-namespace created

Tip: To avoid having to specify our namespace with -n each time use the set-context to specify a default namespace.

[root@master-1 Oracle-on-Kubernetes]# kubectl config set-context --current --namespace=oracle-namespace
Context "kubernetes-admin@kubernetes" modified.

Kubectl will now default to using the oracle-namespace the for example,

[root@master-1 Oracle-on-Kubernetes]# kubectl get pods
No resources found in oracle-namespace namespace.

Create Secrets

For this post I will be using the official Oracle 21c Docker image which is available from Oracle Container Registry. 

Please note: before you can pull this image you will need to accept the licence details and have a valid Oracle support account.

Logon to the Oracle Container Registry (OCR) using SSO credentials, for example

[root@master-1 Oracle-on-Kubernetes]# docker login container-registry.oracle.com
Username: <SSO email>
Password: <SSO password>
Login Succeeded

Create Kubernetes secret using OCR credentials config.json file created by the previous step.

[root@master-1 Oracle-on-Kubernetes]# kubectl create secret generic regcred --from-file=.dockerconfigjson=$HOME/.docker/config.json  --type=kubernetes.io/dockerconfigjson -n oracle-namespace
secret/regcred created

Create ConfigMap

For this Kubernetes build I will be using a ConfigMap to pass variables to my Oracle 21.3 container, this makes is very easy to change database name and passwords etc.

# Oracle 12.2
DB_SID=PSTG
DB_PDB=PSTGPDB1
DB_PASSWD=Kube#2020
DB_DOMAIN=localdomain
DB_BUNDLE=basic
DB_MEMORY=4g
# Oracle 18xe
ORACLE_CHARACTERSET=AL32UTF8
ORACLE_PWD=Kube#2020
# Oracle 19c
ORACLE_SID=ORCL
ORACLE_PDB=ORCLPDB1
ORACLE_EDITION=enterprise
ENABLE_ARCHIVELOG=true
# Optional
#INIT_SGA_SIZE=
#INIT_PGA_SIZE=

oracle.properties

[root@master-1 Oracle-on-Kubernetes]# kubectl create configmap oradb --from-env-file=oracle.properties -n oracle-namespace
configmap/oradb created

Create Storage Class

In this post, I will create a Portworx Storage Class that has a replication factor of 3 with I/O profile set to “db” and priority set to “high.”

This means that the storage will be optimized for low latency database workloads like Oracle and automatically placed on the highest performance storage available in the cluster.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: px-ora-sc
provisioner: kubernetes.io/portworx-volume
parameters:
  repl:        "3"
  io_profile:  "db"
  priority_io: "high"
allowVolumeExpansion: true

px-ora-sc.yaml

[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f px-ora-sc.yaml 
storageclass.storage.k8s.io/px-ora-sc created

We can view the Storage Class details using kubectl describe sc, for example

[root@master-1 Oracle-on-Kubernetes]# kubectl describe sc/px-ora-sc
Name:            px-ora-sc
IsDefaultClass:  No
Annotations:     kubectl.kubernetes.io/last-applied-configuration={"allowVolumeExpansion":true,"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"px-ora-sc"},"parameters":{"io_profile":"db","priority_io":"high","repl":"3"},"provisioner":"kubernetes.io/portworx-volume"}

Provisioner:           kubernetes.io/portworx-volume
Parameters:            io_profile=db,priority_io=high,repl=3
AllowVolumeExpansion:  True
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>

Database Creation

We can now use kubectl apply to create an Oracle 21.3EE database on our Kubernetes.

[root@master-1 Oracle-on-Kubernetes]# kubectl apply -f 21c_statefulset_PX.yaml
statefulset.apps/oracle21c created
service/oracle21c created

Statefulset

Use kubectl get statefulset to check status of the statefulset.

[root@master-1 Oracle-on-Kubernetes]# kubectl get statefulsets -o wide
NAME        READY   AGE   CONTAINERS   IMAGES
oracle21c   1/1     25s   oracle21c    container-registry.oracle.com/database/enterprise:21.3.0.0

Pods

Use kubectl get pods to get pod name and its status.

[root@master-1 Oracle-on-Kubernetes]# kubectl get pods -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES
oracle21c-0   1/1     Running   0          82s   10.244.1.7   node-1-3   <none>           <none>

Now we know the pod name we can view / follow the log output during the build, for example

[root@master-1 Oracle-on-Kubernetes]# kubectl logs pods/oracle21c-0 --follow
[2021:11:19 16:38:24]: Acquiring lock .ORCL.create_lck with heartbeat 30 secs
[2021:11:19 16:38:24]: Lock acquired
[2021:11:19 16:38:24]: Starting heartbeat
[2021:11:19 16:38:24]: Lock held .ORCL.create_lck
ORACLE EDITION: ENTERPRISE
LSNRCTL for Linux: Version 21.0.0.0.0 - Production on 19-NOV-2021 16:38:24
Copyright (c) 1991, 2021, Oracle.  All rights reserved.
Starting /opt/oracle/product/21c/dbhome_1/bin/tnslsnr: please wait...
TNSLSNR for Linux: Version 21.0.0.0.0 - Production
System parameter file is /opt/oracle/homes/OraDB21Home1/network/admin/listener.ora
Log messages written to /opt/oracle/diag/tnslsnr/oracle21c-0/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 21.0.0.0.0 - Production
Start Date                19-NOV-2021 16:38:25
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/oracle/homes/OraDB21Home1/network/admin/listener.ora
Listener Log File         /opt/oracle/diag/tnslsnr/oracle21c-0/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))
The listener supports no services
The command completed successfully
Prepare for db operation
8% complete
Copying database files
31% complete
Creating and starting Oracle instance
32% complete
...
SQL> SQL> Disconnected from Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0
The Oracle base remains unchanged with value /opt/oracle
Executing user defined scripts
/opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/extensions/setup/savePatchSummary.sh
/opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/extensions/setup/swapLocks.sh
[2021:11:19 16:53:05]: Releasing lock .ORCL.create_lck
[2021:11:19 16:53:05]: Lock released .ORCL.create_lck
[2021:11:19 16:53:05]: Acquiring lock .ORCL.exist_lck with heartbeat 30 secs
[2021:11:19 16:53:05]: Lock acquired
[2021:11:19 16:53:05]: Starting heartbeat
[2021:11:19 16:53:05]: Lock held .ORCL.exist_lck
DONE: Executing user defined scripts
The Oracle base remains unchanged with value /opt/oracle
#########################
DATABASE IS READY TO USE!
#########################
Executing user defined scripts
/opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/extensions/startup/runDatapatch.sh
Datafiles are already patched. Skipping datapatch run.
...

Persistent Volume Claims (PVCs)

[root@master-1 Oracle-on-Kubernetes]# kubectl get pvc -o wide
NAME                         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE   VOLUMEMODE
ora-data213-oracle21c-0      Bound    pvc-f25fc9d0-7ce8-4bd5-9673-5120b9cc507c   25Gi       RWO            px-ora-sc      19m   Filesystem
ora-setup213-oracle21c-0     Bound    pvc-9cf30d0a-79b6-4125-8d98-b5d3570d1da0   1Gi        RWO            px-ora-sc      19m   Filesystem
ora-startup213-oracle21c-0   Bound    pvc-23f72131-a386-4407-ad82-fe75901d740d   1Gi        RWO            px-ora-sc      19m   Filesystem

Services

[root@master-1 Oracle-on-Kubernetes]# kubectl get services
NAME        TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                         AGE
oracle21c   NodePort   10.100.141.244   <none>        1521:31473/TCP,5500:32438/TCP   7m57s

[root@master-1 Oracle-on-Kubernetes]# kubectl get services oracle21c -o jsonpath={.spec.ports[?(@.port==1521)].nodePort}
31473

Database Connection

Let’s try and connect to our Oracle 21.3 instance using the external port number from my laptop.

rekins@rekins--MacBookPro15 ~ % sqlplus system/Kube#2020@//10.225.115.150:31473/ORCL @database_details    

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Nov 19 17:16:37 2021
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Last Successful login time: Fri Nov 19 2021 17:16:04 +00:00

Connected to:
Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0

Current Time   : 19/11/21 17:16:42

Database Details
===============================================
Hostname       : oracle21c-0
Database Name  : ORCL
Date Created   : 19/11/21 16:40:42
Date Started   : 19/11/21 16:52:10
Resetlogs Date : 19/11/21 16:40:43
DB Status      : OPEN
Space Allocated:	1.98 GB
Space Used     :	1.94 GB

Oracle Enterprise Manager Database Express Console is also available from a browser using the external EM port number.

Enterprise Manager Express Database

Clean-Up

Use kubectl delete to remove the statefulset 

[root@master-1 Oracle-on-Kubernetes]# kubectl delete -f 21c_statefulset_PX.yaml
statefulset.apps "oracle21c" deleted
service "oracle21c" deleted

Use kubectl delete pvc -all to remove all Persistent Volume Claims

[root@master-1 Oracle-on-Kubernetes]# kubectl delete pvc --all -n oracle-namespace
persistentvolumeclaim "ora-data213-oracle21c-0" deleted
persistentvolumeclaim "ora-setup213-oracle21c-0" deleted
persistentvolumeclaim "ora-startup213-oracle21c-0" deleted

Summary

In this post I have shared how we can deploy an Oracle 21.3 database on Kubernetes with Portworx storage.

If you want to try this for yourself you can find the manifests used within this post at my GitHub site.

[twitter-follow screen_name=’RonEkins’ show_count=’yes’]

Published by @RonEkins

Ron is an Oracle ACE Director, TOGAF Certified Enterprise Architect, and Oracle and Azure Cloud certified Architect with over 30 years experience in design, development and delivery of large enterprise solutions, from conception through to handover and closure. Ron is a leader in data management and inventor with 5 patents filed in data security, governance, management, and ERG (environmental, social governance). Ron is a Principal Field Solutions Architect, working in the Office of the CTO at Pure Storage, and contributor on Kubernetes content at Portworx. Ron is an active member of the Oracle community, and regular presenter at technology conferences across the globe including Oracle Open World, Oracle Ground Breakers Tours (AIOUG & APACOUC), UK, German (DOAG), Irish, Scottish, Israeli, Finnish, Dutch, Spanish, Norwegian, Bulgarian, and Polish OUGs.

Containers, Oracle Database

, ,

Leave a Reply

Create a website or blog at WordPress.com

Up ↑

%d bloggers like this: