Getting started with Oracle 19c ASM Filter Driver (ASMFD)

Before I start any new software install I find it’s good practice to check the Oracle Certification Matrix on My Oracle Support (MOS).

The ASM Filter Driver MOS document is -> ASMFD (ASM Filter Driver) Support on OS Platforms (Certification Matrix). ( Doc ID 2034681.1 )

...
Oracle Linux – RedHat Compatible Kernel 7 3.10.0-123 and later RedHat kernels X86_64 Base 
Oracle Linux - Unbreakable Enterprise Kernel 7 All Updates, 4.1 and later UEK 4.1 kernels X86_64 Base
... 

My 2 node cluster has been built with Oracle 7.6, so the above says I am OK to proceed.

Volume Creation

Let’s start by creating our new ASMFD Volumes for my Grid Infrastructure and ASM database. I will create a 2 Volume for GI and 4 for my ASM database, I have prefixed my Volume names with my RAC cluster name to help identification. e.g

• z-rac_dg_rac_crs
• z-rac_dg_rac_gimr
• z-rac_dg_rac_data01
• z-rac_dg_rac_data02
• z-rac_dg_rac_control_redo
• z-rac_dg_rac_fra

I repeated the above for my 6 volumes and connected them to my two Oracle 19c RAC Linux database servers (z-rac1 & z-rac2).

Multipath Configuration

Now we have connected our newly created ASMFD Volumes I will add entries to  ‘/etc/multipath.conf’ on both RAC nodes, this will allow us to work with more human readable names.

In the example the wwid (World Wide Identifier) is set to Vendor ID + Serial number. e.g. ‘3624a9370’ (for Pure Storage) + ‘50c939582b0f46c00031e3f4’.
Note, the ‘wwid’ needs to be in lowercase and the ‘alias’ name for ASM disks needs to be less than 30 characters, alphanumeric and only use the ‘_ ‘ special character. 

multipaths {
      multipath {
             wwid        3624a937050c939582b0f46c00031e3f4
             alias       dg_rac_data01
      }
             wwid        3624a937050c939582b0f46c00031e418
             alias       dg_rac_data02
      }
      multipath {
             wwid        3624a937050c939582b0f46c00031e3f7
             alias       dg_rac_fra
      }
      multipath {
             wwid        3624a937050c939582b0f46c00031e3fa
             alias       dg_rac_control_redo
      }
      multipath {
             wwid        3624a937050c939582b0f46c00031e3fd
             alias       dg_rac_crs
      }
      multipath {
             wwid        3624a937050c939582b0f46c00031e410
             alias       dg_rac_gimr
      }     
 }

Re-scan SCSI Bus

# rescan-scsi-bus.sh -a

Reload multipath configuration:

# service multipathd reload

Load and display multipath configuration, device mapper and other components

# multipath -ll

Identify Volumes

 [root@z-rac1 mapper]# ls -l dg*
 lrwxrwxrwx. 1 root root 7 Jun  5 10:42 dg_rac_control_redo -> ../dm-4
 lrwxrwxrwx. 1 root root 7 Jun  5 10:42 dg_rac_crs -> ../dm-5
 lrwxrwxrwx. 1 root root 7 Jun  5 10:42 dg_rac_data01 -> ../dm-3
 lrwxrwxrwx. 1 root root 7 Jun  5 10:42 dg_rac_data02 -> ../dm-7
 lrwxrwxrwx. 1 root root 7 Jun  5 10:42 dg_rac_fra -> ../dm-8
 lrwxrwxrwx. 1 root root 7 Jun  5 10:42 dg_rac_gimr -> ../dm-9

Disable avahi-daemon

[root@z-rac1 bin]# service avahi-daemon stop
 Redirecting to /bin/systemctl stop avahi-daemon.service
 Warning: Stopping avahi-daemon.service, but it can still be activated by:
   avahi-daemon.socket

 [root@z-rac1 bin]# chkconfig avahi-daemon off
 Note: Forwarding request to 'systemctl disable avahi-daemon.service'.
 Removed symlink /etc/systemd/system/multi-user.target.wants/avahi-daemon.service.
 Removed symlink /etc/systemd/system/sockets.target.wants/avahi-daemon.socket.
 Removed symlink /etc/systemd/system/dbus-org.freedesktop.Avahi.service.

Oracle database pre-install package

[root@z-rac1] #cd /etc/yum.repos.d/ 
[root@z-rac1] #wget http://yum.oracle.com/public-yum-ol7.repo
[root@z-rac1] #yum install oracle-database-preinstall-19c 

Unzip Oracle 19c Grid Infrastructure media

Configure Oracle yum repository, if not already configured and use yum to install oracle-database-preinstall-19c e.g.

[grid@z-rac1 ~]$ cd $GRID_HOME
[grid@z-rac1 grid]$ pwd
/u01/app/19.0.0/grid

[grid@z-rac1 grid]$ unzip /mnt/oracle/LINUX.X64_193000_grid_home.zip 

Initialise Volumes for ASMFD

[root@z-rac1 bin] # $ORACLE_HOME/bin/asmcmd afd_label 'CRS' '/dev/mapper/dg_rac_crs' --init

[root@z-rac1 bin] # $ORACLE_HOME/bin/asmcmd afd_label 'GIMR' '/dev/mapper/dg_rac_gimr' --init

[root@z-rac1 bin] # $ORACLE_HOME/bin/asmcmd afd_label 'FRA' '/dev/mapper/dg_rac_fra' --init

[root@z-rac1 bin] # $ORACLE_HOME/bin/asmcmd afd_label 'DATA' '/dev/mapper/dg_rac_data01' --init

[root@z-rac1 bin]# $ORACLE_HOME/bin/asmcmd afd_label 'DATA' '/dev/mapper/dg_rac_data02' --init

[root@z-rac1 bin]# $ORACLE_HOME/bin/asmcmd afd_label 'CONTROL_REDO' '/dev/mapper/dg_rac_control_redo' --init

Verify ASMFD Labels

We can now use the ASMCMD afd_lslbl command to verify our devices has been marked for use with Oracle ASMFD.

[root@z-rac1 bin]# ./asmcmd afd_lslbl /dev/mapper/
----------------------------------------------------------------------
Label                     Duplicate  Path ======================================================================
CONTROL_REDO                          /dev/mapper/dg_rac_control_redo
CRS                                   /dev/mapper/dg_rac_crs
DATA                              Y   /dev/mapper/dg_rac_data01
DATA                              Y   /dev/mapper/dg_rac_data02
FRA                                   /dev/mapper/dg_rac_fra
GIMR                                  /dev/mapper/dg_rac_gimr                                 

Everything looks good to this point.

Issue with UEK5

It was at this point I hit an issue, my 2 node Oracle 19c cluster is running Oracle Linux 7.6 which use the Oracle UEK5 Kernel.

UEK4 is using the 4.1 Kernel, UEK uses the 4.14 Kernel. You can check your Oracle Linux version and Kernel using the following.

[oracle@z-rac1 SLOB]$ uname -a 
Linux z-rac1.uklab.purestorage.com 4.14.35-1902.0.18.el7uek.x86_64 #2 SMP Thu May 23 16:44:07 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux

[oracle@z-rac1 SLOB]$ cat /etc/*release
Oracle Linux Server release 7.6
NAME="Oracle Linux Server"
VERSION="7.6"
ID="ol"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.6"
PRETTY_NAME="Oracle Linux Server 7.6"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:7:6:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://bugzilla.oracle.com/"
ORACLE_BUGZILLA_PRODUCT="Oracle Linux 7"
ORACLE_BUGZILLA_PRODUCT_VERSION=7.6
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=7.6
Red Hat Enterprise Linux Server release 7.6 (Maipo)
Oracle Linux Server release 7.6

Unfortunately the Oracle Certification Matrix is not correct, there is no UKE5 Kernel support for ASMFD, you can create the devices but you run into issues using them.

ASMFD not supported on UEK5 at this time. BUG 29008427 – 19.2_AFD_UEK5: AFD NOT SUPPORT IN UEK5 – unpublished bug but, it also applies to 18 or any version. It is the kernel that it is not supported for ASMFD. 

When Oracle development provide UEK5 kernel supports I will revisit ASMFD and this Blog, but until then I will continue to use UDEV rules.

[twitter-follow screen_name=’RonEkins’ show_count=’yes’]