In this Blog I can going to show how you can run an Oracle 19c Database on a Kubernetes cluster using a Network File System (NFS) server for Persistent Storage.
In this example I will be using the Pure Storage Container Storage Interface (CSI) Driver Pure Service Orchestrator (PSO) to present a persistent storage to my Oracle Database container using a FlashBlade NFS (Network File System) server in my lab.
If you have not yet created an Oracle 19c Kubernetes image from the official Oracle Docker build scripts you can find detailed steps here.
Persistent NFS Storage
Let’s start by creating our database file systems, the Oracle Docker image can use 3 filesystems for storing persistent data.
- /opt/oracle/oradata – used for Oracle database files
- /opt/oracle/scripts/setup – used for custom scripts run on initial setup
- /opt/oracle/scripts/startup – used for custom script run on database startup
Below is an extract from my pvc-nfs.yaml file which we will use to create our FlashBlade NFS filesystem using the PSO CSI Driver.
---
### Oracle 19c NFS
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: ora-data193-nfs
namespace: oracle-namespace
labels:
app: database
version: 19.3.0.1
spec:
storageClassName: pure-file
accessModes:
- ReadWriteMany
resources:
requests:
storage: 20GiCreate Persistent Volume Claim (PVC)
$ kubectl apply -f pvc-nfs.yaml -n oracle-namespace persistentvolumeclaim/ora-data193-nfs created persistentvolumeclaim/ora-startup193-nfs created persistentvolumeclaim/ora-setup193-nfs created
Get Persistent Volume Claim (PVC)
To help the track the PVC’s I will highlight the ‘/opt/oracle/oradata’ filesystem throughout this blog post.
$ kubectl get pvc -n oracle-namespace
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS
ora-data193-nfs Bound pvc-797e64df-4b12-4cf6-9e08-a6fa51bb4408 20Gi RWX pure-file
ora-setup193-nfs Bound pvc-35ced0e5-ec71-4454-8fa6-5cc1476a268e 1Gi RWX pure-file
ora-startup193-nfs Bound pvc-73b5c2a8-080b-4979-b69b-e1206189167b 1Gi RWX pure-file
Describe Persistent Volume Claim (PVC)
$ kubectl describe pvc/ora-data193-nfs -n oracle-namespace Name: ora-data193-nfs Namespace: oracle-namespace StorageClass: pure-file Status: Bound Volume: pvc-797e64df-4b12-4cf6-9e08-a6fa51bb4408 Labels: app=database version=19.3.0.1 Annotations: pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes volume.beta.kubernetes.io/storage-provisioner: pure-csi Finalizers: [kubernetes.io/pvc-protection] Capacity: 20Gi Access Modes: RWX VolumeMode: Filesystem
Delete Persistent Volume Claim (PVC)
If you no longer need your file system you can delete with kubectl delete persistentvolumeclaim or slightly easier kubectl delete pvc.
$ kubectl delete -n oracle-namespace persistentvolumeclaim ora-data193-nfs
Oracle 19c Database Creation
Using the previously created Kubernetes Oracle Docker image, ConfigMap and my Oracle 19c Database yaml file I can quickly spin up a new database with kubectl apply.
Create Oracle 19c NFS Deployment
$ kubectl apply -f database19c-nfs.yaml -n oracle-namespace deployment.apps/oracle19c-nfs created service/oracle19c-nfs created
List Running Database
List running databases with kubectl get pods
$ kubectl get pods -n oracle-namespace -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
oracle19c-nfs-69b87cb877-lf4w8 1/1 Running 0 58s 10.1.92.87 z-re-uk8s02 <none>
Database Creation Log
We can check progress of our database creation by reviewing the log output with kubectl logs
$ kubectl logs oracle19c-69b87cb877-lf4w8 -n oracle-namespace ORACLE PASSWORD FOR SYS, SYSTEM AND PDBADMIN: Kube#2020 LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 18-MAY-2020 14:16:48 Copyright (c) 1991, 2019, Oracle. All rights reserved. Starting /opt/oracle/product/19c/dbhome_1/bin/tnslsnr: please wait... TNSLSNR for Linux: Version 19.0.0.0.0 - Production System parameter file is /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora Log messages written to /opt/oracle/diag/tnslsnr/oracle19c-69b87cb877-lf4w8/listener/alert/log.xml Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1))) Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521))) Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1))) STATUS of the LISTENER ------------------------ Alias LISTENER Version TNSLSNR for Linux: Version 19.0.0.0.0 - Production Start Date 18-MAY-2020 14:16:49 Uptime 0 days 0 hr. 0 min. 0 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora Listener Log File /opt/oracle/diag/tnslsnr/oracle19c-69b87cb877-lf4w8/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521))) The listener supports no services The command completed successfully Prepare for db operation 8% complete Copying database files 31% complete Creating and starting Oracle instance 32% complete 36% complete ...
FlashBlade Network File System (NFS)
Once the database build has completed we can connect to our Kubernetes container using kubectl exec , from inside our container we can see the FlashBlade hosted NFS mount points.
$ kubectl exec -it oracle19c-nfs-69b87cb877-lf4w8 -n oracle-namespace -- /bin/bash [oracle@oracle19c-nfs-69b87cb877-lf4w8 ~]$ df -h Filesystem Size Used Avail Use% Mounted on overlay 197G 23G 166G 13% / tmpfs 64M 0 64M 0% /dev tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup tmpfs 7.9G 0 7.9G 0% /dev/shm /dev/sda2 197G 23G 166G 13% /etc/hosts 192.168.4.150:/rek8s-pvc-797e64df-4b12-4cf6-9e08-a6fa51bb4408 20G 3.9G 17G 20% /opt/oracle/oradata 192.168.4.150:/rek8s-pvc-35ced0e5-ec71-4454-8fa6-5cc1476a268e 1.0G 0 1.0G 0% /opt/oracle/scripts/startup 192.168.4.150:/rek8s-pvc-73b5c2a8-080b-4979-b69b-e1206189167b 1.0G 0 1.0G 0% /opt/oracle/scripts/setup tmpfs 7.9G 12K 7.9G 1% /run/secrets/kubernetes.io/serviceaccount tmpfs 7.9G 0 7.9G 0% /proc/acpi tmpfs 7.9G 0 7.9G 0% /proc/scsi tmpfs 7.9G 0 7.9G 0% /sys/firmware
By logging onto my FlashBlade I can see my 20G /opt/oracle/oradata file system has consumed 3.85GB with a Data Reduction of 4.1 : 1
Oracle Database Management
We can stop and start our database with kubectl scale –replicas , setting to zero to stop, or 1 to restart.
Stop Database
$ kubectl scale -n oracle-namespace deployment oracle19c-nfs --replicas=0
Start Database
$ kubectl scale -n oracle-namespace deployment oracle19c-nfs --replicas=0
Database Move / Relocation
We can also move / relocate our database to other nodes within our Kubernetes cluster to perfom maintenance etc. using kubectl cordon. I have already blogged on the use of cordon / uncordon, and if your interested you can read about it here.
Connect to Database
Before we can use sqlplus or other database tools, we need to know which port out listener (1521) has been mapped to by Kubernetes, we can do this with kubectl get svc
$ kubectl get svc/oracle19c-nfs -n oracle-namespace
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
oracle19c-nfs NodePort 10.152.183.196 <none> 1521:31413/TCP,5500:31836/TCP 4d5h
Now, from my desktop I can connect directly to my database using sqlplus and the Kubernetes service port.
$ sqlplus system/Kube#2020@z-re-uk8s01:31413/ORCL @database_details.sql SQL*Plus: Release 19.0.0.0.0 - Production on Tue May 19 09:41:43 2020 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Last Successful login time: Tue May 19 2020 09:40:49 +01:00 Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 Current Time : 19/05/20 08:41:48 Database Details =============================================== Hostname : oracle19c-nfs-69b87cb877-lf4w8 Database Name : ORCL Date Created : 18/05/20 20:25:40 Date Started : 18/05/20 23:28:28 Resetlogs Date : 18/05/20 20:25:42 DB Status : OPEN Space Allocated: 1.63 GB Space Used : 1.39 GB Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0
[twitter-follow screen_name=’RonEkins’ show_count=’yes’]
Leave a Reply