Oracle 19c on Docker and Kubernetes Part 3 – Using Minikube and VirtualBox

Background

If you have followed Part 1 and Part 2 you should have a already downloaded the Oracle 19.3 Docker image from the Oracle Container Registry and have tried it out in your environment.

In this post I will be using Minikube and VirtualBox to run an Oracle 19c database within Kubernetes environment.

Kubernetes – Minikube

If you have not already got MiniKube installed you get it following the instructions here, also for VirtualBox follow this link.

Before we start I will increase the default Minikube VM memory size from 2GB to 10GB, this is required to accommodate my Oracle 19c database, which requires a minimum of 8GB. I will also set the MiniKube VM driver to use VirtualBox this can all be done by changing the Minikube configuration.

% minikube version
minikube version: v1.13.1
commit: 1fd1f67f338cbab4b3e5a6e4c71c551f522ca138

% minikube config set memory 10240
❗  These changes will take effect upon a minikube delete and then a minikube start

% minikube config view
- disk-size: 40GB
- memory: 10240
- vm-driver: virtualbox

I will also enable the minikube metrics server to provide a CPU and Memory Usage graphs, these can be done after minikube has started.

% minikube addons enable metrics-server
🌟  The 'metrics-server' addon is enabled

To check what add-ons are enabled use minikube addons list

% minikube addons list
|-----------------------------|----------|--------------|
|         ADDON NAME          | PROFILE  |    STATUS    |
|-----------------------------|----------|--------------|
| ambassador                  | minikube | disabled     |
| csi-hostpath-driver         | minikube | disabled     |
| dashboard                   | minikube | enabled ✅   |
| default-storageclass        | minikube | enabled ✅   |
| efk                         | minikube | disabled     |
| freshpod                    | minikube | disabled     |
| gcp-auth                    | minikube | disabled     |
| gvisor                      | minikube | disabled     |
| helm-tiller                 | minikube | disabled     |
| ingress                     | minikube | disabled     |
| ingress-dns                 | minikube | disabled     |
| istio                       | minikube | disabled     |
| istio-provisioner           | minikube | disabled     |
| kubevirt                    | minikube | disabled     |
| logviewer                   | minikube | disabled     |
| metallb                     | minikube | disabled     |
| metrics-server              | minikube | enabled ✅   |
| nvidia-driver-installer     | minikube | disabled     |
| nvidia-gpu-device-plugin    | minikube | disabled     |
| olm                         | minikube | disabled     |
| pod-security-policy         | minikube | disabled     |
| registry                    | minikube | disabled     |
| registry-aliases            | minikube | disabled     |
| registry-creds              | minikube | disabled     |
| storage-provisioner         | minikube | enabled ✅   |
| storage-provisioner-gluster | minikube | disabled     |
| volumesnapshots             | minikube | disabled     |
|-----------------------------|----------|--------------|

We are now ready to start minikube

% minikube start
😄  minikube v1.13.1 on Darwin 10.14.6
✨  Using the virtualbox driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing virtualbox VM for "minikube" ...
🐳  Preparing Kubernetes v1.19.2 on Docker 19.03.8 ...
🔎  Verifying Kubernetes components...
🌟  Enabled addons: dashboard, default-storageclass, metrics-server, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" by default

Let’s shell into our minikube VM and see if we can see our 19c database image.

% docker images
REPOSITORY                              TAG      IMAGE ID    CREATED             SIZE
k8s.gcr.io/kube-proxy                   v1.19.2  d373dd5a8593 4 weeks ago         118MB
k8s.gcr.io/kube-apiserver               v1.19.2  607331163122 4 weeks ago         119MB
k8s.gcr.io/kube-controller-manager      v1.19.2  8603821e1a7a 4 weeks ago         111MB
k8s.gcr.io/kube-scheduler               v1.19.2  2f32d66b884f 4 weeks ago         45.7MB
gcr.io/k8s-minikube/storage-provisioner v3       bad58561c4be 6 weeks ago         29.7MB
k8s.gcr.io/etcd                         3.4.13-0 0369cf4303ff 7 weeks ago         253MB
kubernetesui/dashboard                  v2.0.3   503bc4b7440b 3 months ago        225MB
k8s.gcr.io/coredns                      1.7.0    bfe3a36ebd25 3 months ago        45.2MB
k8s.gcr.io/kube-proxy                   v1.18.3  3439b7546f29 4 months ago        117MB
<none>                                  <none>   da26705ccb4b 4 months ago        162MB
<none>                                  <none>   76216c34ed0c 4 months ago        95.3MB
<none>                                  <none>   7e28efa976bd 4 months ago        173MB
kubernetesui/metrics-scraper            v1.0.4   86262685d9ab 6 months ago        36.9MB
k8s.gcr.io/pause                        3.2      80d28bedfe5d 8 months ago        683kB
k8s.gcr.io/coredns                      1.6.7    67da37a9a360 8 months ago        43.8MB
<none>                                  <none>   303ce5db0e90 11 months ago       288MB
k8s.gcr.io/metrics-server-amd64         v0.2.1   9801395070f3  2 years ago         42.5MB

As expected we can’t see our Oracle 19c image as we have not pushed it into our minikube VM.

Minikube – ssh

There are a number of ways of pushing images into minikube, however the best approach for performance is a pull from within the minikube ssh.

So let’s ssh into the minikube VM and pull the image from the oracle container registry using our Oracle SSO credentials.

% minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ docker login container-registry.oracle.com 
Username: <Email>
Password: <Password>
WARNING! Your password will be stored unencrypted in /home/docker/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

Now we have logged on to the Oracle container registry let’s pull the image.

$ docker pull container-registry.oracle.com/database/enterprise:latest
latest: Pulling from database/enterprise
...
Digest: sha256:9b28cbc568bc58fb085516664369930efbd943d22fa24299c68651586e3ef668
Status: Downloaded newer image for container-registry.oracle.com/database/enterprise:latest
container-registry.oracle.com/database/enterprise:latest

If all has gone well you should now be able to see the downloaded Oracle 19.3 Docker image, be patient it’s fairly large ~2.7GB.

$ docker images 
REPOSITORY                                        TAG    IMAGE ID    CREATED    SIZE
container-registry.oracle.com/database/enterprise latest 2e375ab66980 2 months ago 6.66GB

Getting Started with Kubernetes

Kubernetes Namespace

For this blog I use a new Kubernetes namespace called oracle-namespace.

We can create a new namespace with kubectl create namespace.

% kubectl create namespace oracle-namespace
namespace/oracle-namespace created

We can see what namespace are available with kubectl get namespace.

% kubectl get namespace oracle-namespace
NAME               STATUS   AGE
oracle-namespace   Active   19m

Kubernetes ConfigMap

For my Kubernetes build I will be using a ConfigMap to pass variables to my Oracle 19.3 container.

% kubectl create configmap oradb --from-env-file=oracle.properties -n oracle-namespace
configmap/oradb created

You can obtain a copy of the oracle.properies from my GitHub site.

Kubernetes Deployment

We can create our Oracle 19.3 database Kubernetes deployment with kubectl apply, if you want to use my yaml file you can find a copy here.

% kubectl apply -f database_193.yaml -n oracle-namespace
deployment.apps/oracle193 created
service/oracle193 created

Use kubectl get deployments to check status of deployment.

% kubectl get deployments -n oracle-namespace
NAME        READY   UP-TO-DATE   AVAILABLE   AGE
oracle193   1/1     1            0           3m3s

Use kubectl get pods to get pod name and its status.

% kubectl get pods -n oracle-namespace
NAME                       READY STATUS  RESTARTS AGE
oracle193-84fd658fcb-92dgn 1/1   Running 0        56s

Now we know the pod name we can view the log output during the build.

% kubectl logs pods/oracle193-84fd658fcb-92dgn -n oracle-namespace --follow
[2020:10:19 12:13:23]: Acquiring lock on /opt/oracle/oradata/.ORCL.create_lck
[2020:10:19 12:13:23]: Lock acquired on /opt/oracle/oradata/.ORCL.create_lck
[2020:10:19 12:13:23]: Holding on to the lock using /tmp/.ORCL.create_lck
ORACLE EDITION: ENTERPRISE
ORACLE PASSWORD FOR SYS, SYSTEM AND PDBADMIN: Kube#2020

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 19-OCT-2020 12:13:23

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Starting /opt/oracle/product/19c/dbhome_1/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 19.0.0.0.0 - Production
System parameter file is /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora
Log messages written to /opt/oracle/diag/tnslsnr/oracle193-84fd658fcb-92dgn/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                19-OCT-2020 12:13:23
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora
Listener Log File         /opt/oracle/diag/tnslsnr/oracle193-84fd658fcb-92dgn/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))
The listener supports no services
The command completed successfully
Prepare for db operation
8% complete
Copying database files
31% complete
Creating and starting Oracle instance
32% complete
36% complete
40% complete
43% complete
46% complete
Completing Database Creation
51% complete
54% complete
Creating Pluggable Databases
58% complete
77% complete
Executing Post Configuration Actions
100% complete
Database creation complete. For details check the logfiles at:
 /opt/oracle/cfgtoollogs/dbca/ORCL.
Database Information:
Global Database Name:ORCL
System Identifier(SID):ORCL
Look at the log file "/opt/oracle/cfgtoollogs/dbca/ORCL/ORCL.log" for further details.

SQL*Plus: Release 19.0.0.0.0 - Production on Mon Oct 19 12:25:51 2020
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

SQL> 
System altered.

SQL> 
System altered.

SQL> 
Pluggable database altered.

SQL> 
PL/SQL procedure successfully completed.

SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

Executing user defined scripts
/opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/setup/savePatchSummary.sh

/opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/setup/swapLocks.sh
[2020:10:19 12:25:53]: Connecting to the lock process /tmp/.ORCL.create_lck
[2020:10:19 12:25:53]: Lock released on /opt/oracle/oradata/.ORCL.create_lck
[2020:10:19 12:25:53]: Acquiring lock on /opt/oracle/oradata/.ORCL.exist_lck
[2020:10:19 12:25:53]: Lock acquired on /opt/oracle/oradata/.ORCL.exist_lck
[2020:10:19 12:25:53]: Holding on to the lock using /tmp/.ORCL.exist_lck

DONE: Executing user defined scripts

The Oracle base remains unchanged with value /opt/oracle
#########################
DATABASE IS READY TO USE!
#########################

Executing user defined scripts
/opt/oracle/runUserScripts.sh: running /opt/oracle/scripts/startup/runDatapatch.sh
Datafiles are already patched. Skipping datapatch run.

DONE: Executing user defined scripts

The following output is now a tail of the alert.log:
ORCLPDB1(3):ALTER DATABASE DEFAULT TABLESPACE "USERS"
ORCLPDB1(3):Completed: ALTER DATABASE DEFAULT TABLESPACE "USERS"
2020-10-19T12:25:51.524442+00:00
ALTER SYSTEM SET control_files='/opt/oracle/oradata/ORCL/control01.ctl' SCOPE=SPFILE;
2020-10-19T12:25:51.528797+00:00
ALTER SYSTEM SET local_listener='' SCOPE=BOTH;
   ALTER PLUGGABLE DATABASE ORCLPDB1 SAVE STATE
Completed:    ALTER PLUGGABLE DATABASE ORCLPDB1 SAVE STATE

XDB initialized.

Connecting Remotely

The first we need to do it determine the Minikube IP address

% minikube service oracle193 -n oracle-namespace --url
http://192.168.99.100:32271
http://192.168.99.100:32411

And Kubernete Listener Port (1521)

% kubectl get svc -n oracle-namespace
NAME      TYPE     CLUSTER-IP   EXTERNAL-IP PORT(S)                   
oracle193 NodePort 10.110.67.63 <none>      1521:32271/TCP,5500:32411/TCP   

So from the above we can see our Oracle listener is available externally on Port 32271 on IP 192.168.99.10

sqlplus system/Kube#2020@//192.168.99.100:32271/ORCL @database_details

SQL*Plus: Release 19.0.0.0.0 - Production on Mon Oct 19 14:16:01 2020
Version 19.3.0.0.0

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

Current Time   : 19/10/20 13:16:01

Database Details
===============================================
Hostname       : oracle193-84fd658fcb-92dgn
Database Name  : ORCL
Date Created   : 19/10/20 12:15:17
Date Started   : 19/10/20 12:25:36
Resetlogs Date : 19/10/20 12:15:19
DB Status      : OPEN
Space Allocated: 1.71 GB
Space Used     : 1.68 GB

MiniKube Dashboard

The MiniKube dashboard provides a GUI interface to manage your Kubernetes environment, this can be started using the minikube dashboard command, this will start a browser tab ready and you’re ready to go and explore.

% minikube dashboard
🤔  Verifying dashboard health ...
🚀  Launching proxy ...
🤔  Verifying proxy health ...
🎉  Opening http://127.0.0.1:54600/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser...
Kubectl Dashboard

Container Persistence

What happens if we delete our pod ?

% kubectl get pods -n oracle-namespace
NAME                         READY   STATUS    RESTARTS   AGE
oracle193-84fd658fcb-92dgn   1/1     Running   0          97m

% kubectl delete pod/oracle193-84fd658fcb-92dgn -n oracle-namespace
pod "oracle193-84fd658fcb-92dgn" deleted

% kubectl get pods -n oracle-namespace                             
NAME                         READY   STATUS    RESTARTS   AGE
oracle193-84fd658fcb-psthv   1/1     Running   0          2m13s

From the above we can see we have a new pod name, if we check the log for the new container we can see it’s being recreated and lost any updates.

% kubectl logs pods/oracle193-84fd658fcb-psthv -n oracle-namespace
[2020:10:19 13:51:45]: Acquiring lock on /opt/oracle/oradata/.ORCL.create_lck
[2020:10:19 13:51:45]: Lock acquired on /opt/oracle/oradata/.ORCL.create_lck
[2020:10:19 13:51:45]: Holding on to the lock using /tmp/.ORCL.create_lck
ORACLE EDITION: ENTERPRISE
ORACLE PASSWORD FOR SYS, SYSTEM AND PDBADMIN: Kube#2020

LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 19-OCT-2020 13:51:45

Copyright (c) 1991, 2019, Oracle.  All rights reserved.

Starting /opt/oracle/product/19c/dbhome_1/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 19.0.0.0.0 - Production
System parameter file is /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora
Log messages written to /opt/oracle/diag/tnslsnr/oracle193-84fd658fcb-psthv/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1)))
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date                19-OCT-2020 13:51:45
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/oracle/product/19c/dbhome_1/network/admin/listener.ora
Listener Log File         /opt/oracle/diag/tnslsnr/oracle193-84fd658fcb-psthv/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))
The listener supports no services
The command completed successfully
Prepare for db operation
8% complete
Copying database files
31% complete
Creating and starting Oracle instance
32% complete

In Part 4 & 5, I will show how we can use a Kubernetes cluster to run our Oracle 19c database and protect our data with persistent storage.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a website or blog at WordPress.com

Up ↑

%d bloggers like this: