I have previously shared how to install the Oracle ADM Filter Driver (ADMFD) using the CLI and also using the Oracle Installer graphical UI.
In the post I will share how to migrate existing Oracle ASM UDEV managed disks to use the newer ASM Filter Driver (ASMFD).
To confirm ASM Filter Driver has already been installed, loaded, supported and current version, use afddriverstate installed | loaded | supported | version, for example.
[root@z-rac1:~]# afddriverstate installed AFD-9203: AFD device driver installed status: 'true' [root@z-rac1:~]# afddriverstate loaded AFD-9205: AFD device driver loaded status: 'true' [root@z-rac1:~]# afddriverstate supported AFD-9200: Supported [root@z-rac1:~]# afddriverstate version AFD-9325: Driver OS kernel version = 5.4.17-2011.0.7.el7uek.x86_64. AFD-9326: Driver build number = 220322.1. AFD-9231: Driver build version = 19.0.0.0.0 (19.15.0.0.0). AFD-9547: Driver available build number = 220322.1. AFD-9232: Driver available build version = 19.0.0.0.0 (19.15.0.0.0).
To migrate the disks to ASMFS we first need to stop the Oracle High Availability Service using $GRID_HOME/crsctl stop has. (Note: repeat on all nodes within a RAC Cluster)
[root@z-rac1:~]# /u01/app/19.0.0/grid/bin/crsctl stop has CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'z-rac1' CRS-2673: Attempting to stop 'ora.crsd' on 'z-rac1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'z-rac1' .. CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'z-rac2' has completed CRS-4133: Oracle High Availability Services has been stopped.
Configure ASMFD and enable using $GRID_HOME/ascmcd afd_configure -e (repeat for RAC)
[root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_configure -e AFD-627: AFD distribution files found. AFD-634: Removing previous AFD installation. AFD-635: Previous AFD components successfully removed. AFD-9294: updating file /etc/sysconfig/oracledrivers.conf AFD-636: Installing requested AFD software. AFD-637: Loading installed AFD drivers. AFD-9321: Creating udev for AFD. AFD-9323: Creating module dependencies - this may take some time. AFD-9154: Loading 'oracleafd.ko' driver. AFD-649: Verifying AFD devices. AFD-9156: Detecting control device '/dev/oracleafd/admin'. AFD-638: AFD installation correctness verified. Modifying resource dependencies - this may take some time.
Confirm that the ASM Filer Driver Kernel module has loaded successfully using lsmod | grep afd.
[root@z-rac1:~]# lsmod | grep afd oracleafd 221184 0 [root@z-rac2 ~]# lsmod | grep afd oracleafd 233472 0
Check the status of the ASMFD using $GRID_HOME/asmcmd afd_state.
[root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_state ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'z-rac1.uklab.purestorage.com' [root@z-rac2 ~]# /u01/app/19.0.0/grid/bin/asmcmd afd_state ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'z-rac2.uklab.purestorage.com'
Before label the disks, lets use kfod to check the current labels are blank.
[root@z-rac1:~]# kfod OP=DISKS status=TRUE disks=ASM label=TRUE dscvgroup=TRUE -------------------------------------------------------------------------------- Disk Size Header Path Disk Group Site Failgroup Disk-Label User Group ================================================================================ 1: 10485760 MB MEMBER /dev/mapper/dg_racpod_acfs01 ACFS # # # oracle dba 2: 204800 MB MEMBER /dev/mapper/dg_racpod_control_redo CONTROL_REDO # # # oracle dba 3: 51200 MB MEMBER /dev/mapper/dg_racpod_crs OCR # # # oracle dba 4: 31457280 MB MEMBER /dev/mapper/dg_racpod_data01 DATA # # # oracle dba 5: 31457280 MB MEMBER /dev/mapper/dg_racpod_data02 DATA # # # oracle dba 6: 31457280 MB MEMBER /dev/mapper/dg_racpod_data03 DATA # # # oracle dba 7: 31457280 MB MEMBER /dev/mapper/dg_racpod_data04 DATA # # # oracle dba 8: 3145728 MB MEMBER /dev/mapper/dg_racpod_fra FRA # # # oracle dba 9: 102400 MB MEMBER /dev/mapper/dg_racpod_gimr MGMT # # # oracle dba
Label and migrate disks using $GRID_HOME/bin/asmcmd afd_label <label> <physical path> –migrate.
[root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label ACFS01 /dev/mapper/dg_racpod_acfs01 --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label CONTROL_REDO /dev/mapper/dg_racpod_control_redo --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label CRS /dev/mapper/dg_racpod_crs --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label DATA01 /dev/mapper/dg_racpod_data01 --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label DATA02 /dev/mapper/dg_racpod_data02 --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label DATA03 /dev/mapper/dg_racpod_data03 --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label DATA04 /dev/mapper/dg_racpod_data04 --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label FRA /dev/mapper/dg_racpod_fra --migrate [root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_label GIMR /dev/mapper/dg_racpod_gimr --migrate
Now when we use kfod label=TRUE and asm_diskstring=AFD:* we can see our disks have been re-labeled AFD:<label name>.
[root@z-rac1:~]# kfod OP=DISKS status=TRUE disks=ASM label=TRUE dscvgroup=TRUE asm_diskstring=AFD:* -------------------------------------------------------------------------------- Disk Size Header Path Disk Group Site Failgroup Disk-Label User Group ================================================================================ 1: 10485760 MB MEMBER AFD:ACFS01 ACFS # # ACFS01 2: 204800 MB MEMBER AFD:CONTROL_REDO CONTROL_REDO # # CONTROL_REDO 3: 51200 MB MEMBER AFD:CRS OCR # # CRS 4: 31457280 MB MEMBER AFD:DATA01 DATA # # DATA01 5: 31457280 MB MEMBER AFD:DATA02 DATA # # DATA02 6: 31457280 MB MEMBER AFD:DATA03 DATA # # DATA03 7: 31457280 MB MEMBER AFD:DATA04 DATA # # DATA04 8: 3145728 MB MEMBER AFD:FRA FRA # # FRA 9: 102400 MB MEMBER AFD:GIMR MGMT # # GIMR
Use asmcmd afd_lsdsk to list all ASM Filter driver disks, here we can see the physical paths.
[root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd afd_lsdsk -------------------------------------------------------------------------------- Label Filtering Path ================================================================================ ACFS01 ENABLED /dev/mapper/dg_racpod_acfs01 CONTROL_REDO ENABLED /dev/mapper/dg_racpod_control_redo CRS ENABLED /dev/mapper/dg_racpod_crs DATA01 ENABLED /dev/mapper/dg_racpod_data01 DATA02 ENABLED /dev/mapper/dg_racpod_data02 DATA03 ENABLED /dev/mapper/dg_racpod_data03 DATA04 ENABLED /dev/mapper/dg_racpod_data04 FRA ENABLED /dev/mapper/dg_racpod_fra GIMR ENABLED /dev/mapper/dg_racpod_gimr
Using asmcmd lsdsk we can see the ASM disks but this time showing the ASMFD label name.
[root@z-rac1:~]# /u01/app/19.0.0/grid/bin/asmcmd lsdsk Path AFD:ACFS01 AFD:CONTROL_REDO AFD:CRS AFD:DATA01 AFD:DATA02 AFD:DATA03 AFD:DATA04 AFD:FRA AFD:GIMR
Using ls -al we can see ASMFD has created some file in /dev/oracleafd/disks
[root@z-rac1:~]# ls -al /dev/oracleafd/disks total 36 drwxrwx---. 2 oracle dba 220 Apr 5 10:40 . drwxrwx---. 3 oracle dba 80 Apr 3 15:50 .. -rw-rw-r--. 1 oracle oinstall 29 Apr 5 10:37 ACFS01 -rw-rw-r--. 1 oracle oinstall 35 Apr 5 10:39 CONTROL_REDO -rw-rw-r--. 1 oracle oinstall 26 Apr 5 10:39 CRS -rw-rw-r--. 1 oracle oinstall 29 Apr 3 15:50 DATA01 -rw-rw-r--. 1 oracle oinstall 29 Apr 3 15:50 DATA02 -rw-rw-r--. 1 oracle oinstall 29 Apr 3 15:50 DATA03 -rw-rw-r--. 1 oracle oinstall 29 Apr 3 15:50 DATA04 -rw-rw-r--. 1 oracle oinstall 26 Apr 5 10:40 FRA -rw-rw-r--. 1 oracle oinstall 27 Apr 5 10:40 GIMR
If we check the contents of the above we can see the file references the physical device.
[root@z-rac1:~]# cat /dev/oracleafd/disks/DATA01 /dev/mapper/dg_racpod_data01
Now, remove UDEV rules from the /etc/udev/rules.d/ directory and reload UDEV rules.
Re-start the Oracle High Availability Services
[root@z-rac1:~]# crsctl start has CRS-4123: Oracle High Availability Services has been started.
As the Grid user, updated the Oracle ASM Filter Driver discovery diskstring value, thus:
[oracle@z-rac1 ~]$ asmcmd dsset 'AFD:*' [oracle@z-rac1 ~]$ asmcmd dsget parameter:AFD:* profile:AFD:*
Using sqlplus connect to the ASM database as sysasm, and perform the following:
Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.15.0.0.0 SQL> show parameter string NAME TYPE VALUE ------------------- ----------- ------------ asm_diskstring string AFD:* SQL> select name, state from v$asm_diskgroup; NAME STATE ------------------------------ ----------- ACFS MOUNTED CONTROL_REDO MOUNTED DATA MOUNTED FRA MOUNTED MGMT MOUNTED OCR MOUNTED 6 rows selected. SQL> select path from v$asm_disk; PATH ------------------------------------------ AFD:DATA01 AFD:DATA02 AFD:DATA03 AFD:DATA04 AFD:GIMR AFD:FRA AFD:CRS AFD:CONTROL_REDO AFD:ACFS01 9 rows selected.
To confirm the ASM Filter Driver is being used we can use the following SQL.
set feed off
set pages 100
set lines 200
col group_number heading 'Grp' format 9
col name heading 'Name' format a18
col state heading 'State' format a7
col header_status heading 'Header' format a7
col mount_status heading 'Mount' format a7
col path heading 'Path' format a16
col library heading 'Library' format a44
Prompt ASMFD Configuration
select group_number, name, state, header_status, mount_status, path, library
from v$asm_disk order by group_number, name;
And the output.
SQL> @asmfd ASMFD Configuration Grp Name State Header Mount Path Library --- ------------------ ------- ------- ------- ---------------- -------------------------------------------- 1 ACFS_0000 NORMAL MEMBER CACHED AFD:ACFS01 AFD Library - Generic , version 3 (KABI_V3) 2 CONTROL_REDO_0000 NORMAL MEMBER CACHED AFD:CONTROL_REDO AFD Library - Generic , version 3 (KABI_V3) 3 DATA_0000 NORMAL MEMBER CACHED AFD:DATA01 AFD Library - Generic , version 3 (KABI_V3) 3 DATA_0001 NORMAL MEMBER CACHED AFD:DATA02 AFD Library - Generic , version 3 (KABI_V3) 3 DATA_0002 NORMAL MEMBER CACHED AFD:DATA03 AFD Library - Generic , version 3 (KABI_V3) 3 DATA_0003 NORMAL MEMBER CACHED AFD:DATA04 AFD Library - Generic , version 3 (KABI_V3) 4 FRA_0000 NORMAL MEMBER CACHED AFD:FRA AFD Library - Generic , version 3 (KABI_V3) 5 MGMT_0000 NORMAL MEMBER CACHED AFD:GIMR AFD Library - Generic , version 3 (KABI_V3) 6 OCR_0000 NORMAL MEMBER CACHED AFD:CRS AFD Library - Generic , version 3 (KABI_V3)
In the above I have demonstrated how to migrate an Oracle ASM UDEV to the ASM Filter Driver (ASMFD).
Leave a Reply