Introduction
In a previous post I have shared how to install and configure the Oracle ZFS Storage Appliance simulator on VirtualBox, and also on VMware Fusion.
In this post I am going to show how we can use an Oracle ZFS Storage Appliance simulator to provide an environment to learn about NFSv4.1 and Oracle Direct NFS (dNFS).
Note: this configuration is for purely educational and training purposes and should not be considered suitable for Production use.
Oracle ZFS Storage VM
On my laptop I have started an Oracle ZFS storage appliance simulator on VirtualBox , and have created a 75G share.
Linux Server
For my database server I am using a Vagrant built Oracle Linux 7.9 server, the Vagrant and Ansible playbooks are available on my GitHub repo.
[oracle@oracle19c CDB1]$ cat /etc/oracle-release Oracle Linux Server release 7.9
Using the Linux showmount command we can see the available exported shares.
[oracle@oracle19c ~]$ showmount -e zfsa Export list for zfsa: /export/oradata (everyone)
The rpcinfo command provides a convenient way of checking supported NFS versions, for example.
[root@oracle19c ~]# rpcinfo zfsa |egrep "service|nfs"
program version netid address service owner
100003 3 udp6 ::.8.1 nfs superuser
100227 3 udp6 ::.8.1 nfs_acl superuser
100003 3 tcp6 ::.8.1 nfs superuser
100003 4 tcp6 ::.8.1 nfs superuser
100227 3 tcp6 ::.8.1 nfs_acl superuser
100003 3 udp 0.0.0.0.8.1 nfs superuser
100227 3 udp 0.0.0.0.8.1 nfs_acl superuser
100003 3 tcp 0.0.0.0.8.1 nfs superuser
100003 4 tcp 0.0.0.0.8.1 nfs superuser
100227 3 tcp 0.0.0.0.8.1 nfs_acl superuser
The above confirms the ZFSA simulator is exporting an NFSv4.1 share and it’s mountable.
Looking at the Oracle Linux database servers /var/log/messages file we can see NFSv4.1 is registered.
[root@oracle19c ~]# grep NFSv4.1 /var/log/messages Mar 14 08:56:33 oracle19c kernel: RPC: Registered tcp NFSv4.1 backchannel transport module.
The mount command can be used to show type and version for example.
[root@oracle19c ~]# mount -t nfs4 zfsa:/export/oradata on /u01/oradata type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.56.20,local_lock=none,addr=192.168.56.101)
Use df -t nfs4 to report file system usage, limit output with type to NFS4, for example.
[root@oracle19c ~]# df -h -t nfs4 -T Filesystem Type Size Used Avail Use% Mounted on zfsa:/export/oradata nfs4 74G 3.4G 71G 5% /u01/oradata
Oracle Database
The Oracle database reads the oranfstab to obtain information about the required dNFS mounts, during the database start-up, therefore any change to the oranfstab file requires a database bounce.
[oracle@oracle19c ~]$ cat $ORACLE_HOME/dbs/oranfstab server: zfsa local: 192.168.56.20 path: 192.168.56.101 export: zfsa:/export/oradata mount: /u01/oradata nfs_version: NFS4.1 security_default: sys
Note: the oranfstab file is very particular in its parsing and requires the local parameter to be defined before the path, they can be on the same line or split over two.
Alert Log File
The Oracle alert.log file should include an entry confirming the Oracle Direct NFS ODM library is running (Oracle 19c uses ODM v6.0), and the channels used.
Confirm that path IP address, filer name and the local IP address are as expected.
... Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 6.0 ... Direct NFS: mount complete dir /export/oradata on zfsa nfsport 2049 using NFS version 4.1 ... Direct NFS: channel id [0] path [192.168.56.101] to filer [zfsa] via local [192.168.56.20] is UP ..
Note: if filer is reporting an IP address rather than a server name, there is likely an issue with the formatting for the oranfstab file.
Oracle 19c Database
From our Oracle 19c database we can also confirm Oracle dNFS is configured correcting by logging on to the database and querying the v$dnfs_servers and v$dns_channels views.
[oracle@oracle19c CDB1]$ sqlplus / as sysdba SQL*Plus: Release 19.0.0.0.0 - Production on Fri Mar 11 15:27:52 2022 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0 SQL> COLUMN svrname FORMAT A10 SQL> COLUMN dirname FORMAT A20 SQL> SELECT svrname, dirname, nfsversion FROM v$dnfs_servers; SVRNAME DIRNAME NFSVERSION ---------- -------------------- ---------------- zfsa /export/oradata NFSv4.1 SQL> COLUMN path FORMAT A20 SQL> SELECT distinct svrname, path FROM v$dnfs_channels; SVRNAME PATH ---------- -------------------- zfsa 192.168.56.101
dNFS views
Oracle provides a number of v$ views specifically for the management of Oracle dNFS, these can be used to obtain further details and metrics, for example.
- v$dnfs_stats – shows dNFS performance statistics
- v$dnfs_channels – shows network channels is use by dNFS
- v$dnfs_servers – shows dNFS servers
- v$dnfs_files – shows files open for dNFS
Oracle dNFS tests
Logon, open and change to the newly created pluggable database (pdb).
SQL> show con_name;
CON_NAME
------------------------------
CDB$ROOT
SQL> show pdbs;
CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ----------
2 PDB$SEED READ ONLY NO
3 PDB1 MOUNTED
SQL> ALTER PLUGGABLE DATABASE pdb1 open;
Pluggable database altered.
SQL> ALTER PLUGGABLE DATABASE pdb1 SAVE STATE;
Pluggable database altered.
SQL> ALTER SESSION SET CONTAINER=pdb1;
Session altered.
Oracle dNFS Test 1
The first test we will try is creating datafile in mounted directory.
SQL> create tablespace test1 datafile '/u01/oradata/test1.dbf' size 1g; Tablespace created.
Oracle dNFS Test 2
The second test I will try is creating a datafile in a sub-directory.
SQL> create tablespace test2 datafile '/u01/oradata/CDB1/pdb1/test2.dbf' size 2g; Tablespace created.
Oracle dNFS Test 3
The third test I will try is creating bigfile tablespace.
SQL> create bigfile tablespace test3 datafile '/u01/oradata/CDB1/pdb1/test3.dbf' size 3g; Tablespace created.
Oracle dNFS test results
Using the Oracle v$dnfs_files view we can confirm the datafile files have been successfully created.
SQL> column filename heading 'File Name' format a50 SQL> column filesize heading 'File Size MB' format a10 SQL> select filename, to_char(filesize/1024/1024,'9,999,999') as filesize from v$dnfs_files; File Name File Size -------------------------------------------------- ---------- ... /u01/oradata/test1.dbf 1,024 /u01/oradata/CDB1/pdb1/test2.dbf 2,048 /u01/oradata/CDB1/pdb1/test3.dbf 3,072
Checking the Oracle altert.log file we can confirm there were no dNFS issues reported.
... PDB1(3):create tablespace test1 datafile '/u01/oradata/test1.dbf' size 1g 2022-03-14T11:32:00.151542+00:00 PDB1(3):Completed: create tablespace test1 datafile '/u01/oradata/test1.dbf' size 1g 2022-03-14T11:32:54.580086+00:00 ... PDB1(3):create tablespace test2 datafile '/u01/oradata/CDB1/pdb1/test2.dbf' size 2g 2022-03-14T11:33:29.464193+00:00 PDB1(3):Completed: create tablespace test2 datafile '/u01/oradata/CDB1/pdb1/test2.dbf' size 2g 2022-03-14T11:35:06.973992+00:00 ... PDB1(3):create bigfile tablespace test3 datafile '/u01/oradata/CDB1/pdb1/test3.dbf' size 3g 2022-03-14T11:36:01.222563+00:00 PDB1(3):Completed: create bigfile tablespace test3 datafile '/u01/oradata/CDB1/pdb1/test3.dbf' size 3g ...
Finally, returning to the Oracle ZFS storage VM, we can see the usage.
Drop Tablespace
SQL> drop tablespace test1 including contents and datafiles; Tablespace dropped.
Summary
In this post I have demonstrated how the Oracle ZFS Storage VM can be used to deliver NFS4.1 shares, and shown how an Oracle 19c database can be configured to use Oracle dNFS with NFSv4.1.
[twitter-follow screen_name=’RonEkins’ show_count=’yes’]