I was recently asked ‘How can I check to see if my storage supports 4K sector sizes, and if my Oracle 19c database is using it’ ?
In this post I will share how to check disk sector sizes from Linux, from Oracle Automatic Storage Management (ASM) and via the Oracle kfed utility.
Traditional disk based storage systems use a sector size of 512 bytes, but if you are using hybrid or an all-flash-array you may want to confirm what is being used.
Storage Terminology
Before we look at some of the Linux OS commands we can use to inspect our disks, we need to understand some of terminology used, for example.
Sector Sizes
The physical and logical sector size of a disk is set by the storage / disk manufacturer and can not be changed, when requested the block sizes are read from the disks.
Non-Advanced Format
The logical block and physical block sizes are both 512 bytes each.
Advanced Format
Advanced Format (AF) is a generic term for block devices with physical sectors larger than the traditional 512 bytes, these generally have a physical sector size of 4K.
512e
512 byte emulation where the logical (addressable) sector is 512 bytes in length (8:1 logical:physical sector ratio
4kN
4096 byte native mode where the logical (addressable) sector is physical 4kb sector size in length (1:1 logical:physical sector ratio), 4kN are also referred to as 4K native, or True 4k.
Linux Server
To check the disk sector sizes, logon to the Linux server as root, we can then use the blockdev command with report to provide a summary of a block devices, for example.
[root@z-oracle:~]# blockdev --report /dev/oracleasm/dg_oradata3 RO RA SSZ BSZ StartSec Size Device rw 8192 512 4096 0 1099511627776 /dev/oracleasm/dg_oradata3
Check Logical Sector Size
Use the getss option to limit output of blockdev to logical block sectors.
[root@z-oracle:~]# blockdev -v --getss /dev/oracleasm/dg_oradata3 get logical block (sector) size: 512
Check Physical Sector Size
And with the getpbsz to see physical block sector size.
[root@z-oracle:~]# blockdev -v --getpbsz /dev/oracleasm/dg_oradata3 get physical block (sector) size: 512
Check Block Size
Use blockdev to see the blocksize using getbsz, for example.
[root@z-oracle:~]# blockdev -v --getbsz /dev/oracleasm/dg_oradata3 get blocksize: 4096
Alternatively, we can also use sg_readcap command to see the logical and physical block sizes, if using sg_readcap pay close attention to the value of exponent to determine physical size
For example, if an logical block lenght=512 and exponent is 3 = 2³ x 512 = 8 x 512 = 4,096 bytes.
[root@z-oracle:~]# sg_readcap --long /dev/oracleasm/dg_oradata3 Read Capacity results: Protection: prot_en=0, p_type=0, p_i_exponent=0 Logical block provisioning: lbpme=1, lbprz=1 Last logical block address=2147483647 (0x7fffffff), Number of logical blocks=2147483648 Logical block length=512 bytes Logical blocks per physical block exponent=0 Lowest aligned logical block address=0 Hence: Device size: 1099511627776 bytes, 1048576.0 MiB, 1099.51 GB
The above confirms the disk has be presented with a sector size of 512 bytes
Oracle ASM
If your storage supports 4K sector sizes you should familiarise yourself with the Oracle Support Note: Using 4k Redo Logs on Flash, 4k-Disk and SSD-based Storage (Doc ID 1681266.1).
Within the Oracle the Oracle ASM documentation section on specifying the logical and physical sector size the following note is included:
To specify the logical and physical sector size, the disks must support sector size emulation.
In the above, I have confirmed I the disks presented to my database server have a sector size of 512 bytes, rather than 4K or 512e.
So what happens if we try and create an ASM diskgroup with a 4k sector size ?
Create diskgroup with a 4K sector size
If the sector_size attribute is explicitly specified when creating a diskgroup, then Oracle ASM attempts to verify that all disks discovered have a sector size equal to the value provided, for example.
SQL> create diskgroup DATA_4K EXTERNAL REDUNDANCY DISK '/dev/oracleasm/dg_oradata3' ATTRIBUTE 'au_size'='4M', 'sector_size'='4096';
create diskgroup DATA_4K EXTERNAL REDUNDANCY DISK '/dev/oracleasm/dg_oradata3' ATTRIBUTE 'au_size'='4M', 'sector_size'='4096';
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15038: disk '/dev/oracleasm/dg_oradata3' mismatch on 'Sector Size' with
target disk group [512] [4096]
From the above we can see an Oracle has raised an ORA-15038 error.
This is an expected behaviour as my disk has a sector size of 512 bytes rather than 4k.
For the purpose of this post I will ask ASM to ignore the validation by setting “_disk_sector_size_override” to True.
Note: the use of any underscore parameters in Production should only be used under the of direction of Oracle support.
SQL> alter system set "_disk_sector_size_override"=TRUE scope=both;
System altered.
Repeat, creation of diskgroup with a 4K sector size
SQL> create diskgroup DATA_4K EXTERNAL REDUNDANCY DISK '/dev/oracleasm/dg_oradata3' ATTRIBUTE 'au_size'='4M', 'sector_size'='4096';
Diskgroup created
Check v$asm_diskgroup
SQL> select name, state, sector_size, group_number from v$asm_diskgroup;
NAME STATE SECTOR_SIZE GROUP_NUMBER
------------------------------ ----------- ----------- ------------
CONTROL_REDO1 MOUNTED 512 1
CONTROL_REDO2 MOUNTED 512 2
CONTROL_REDO MOUNTED 512 4
DATA1 MOUNTED 512 5
DATA2 MOUNTED 512 6
DATA_4K MOUNTED 4096 3
DATA MOUNTED 512 8
FRA1 MOUNTED 512 9
FRA2 MOUNTED 512 10
FRA MOUNTED 512 12
10 rows selected.
Using asmcmd to check ASM disk groups.
[oracle@z-oracle ~]$ asmcmd lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 512 4096 4194304 1048576 1048488 0 1048488 0 N CONTROL_REDO/
MOUNTED EXTERN N 512 512 4096 4194304 1048576 1047840 0 1047840 0 N CONTROL_REDO1/
MOUNTED EXTERN N 512 512 4096 4194304 1048576 1048484 0 1048484 0 N CONTROL_REDO2/
MOUNTED EXTERN N 512 512 4096 4194304 1048576 893788 0 893788 0 N DATA/
MOUNTED EXTERN N 512 512 4096 4194304 1048576 1043048 0 1043048 0 N DATA1/
MOUNTED EXTERN N 512 512 4096 4194304 1048576 1048484 0 1048484 0 N DATA2/
MOUNTED EXTERN N 4096 4096 4096 4194304 1048576 1048492 0 1048492 0 N DATA_4K/
MOUNTED EXTERN N 512 512 4096 4194304 10485760 10326616 0 10326616 0 N FRA/
MOUNTED EXTERN N 512 512 4096 4194304 10485760 10450316 0 10450316 0 N FRA1/
MOUNTED EXTERN N 512 512 4096 4194304 10485760 10464140 0 10464140 0 N FRA2/
Oracle kfed
I have previously blogged on how to analyse Oracle ASM metadata with kfed, we can also use the Oracle kfed utility inspect the sector size, for example.
[oracle@z-oracle ~]$ kfed read /dev/oracleasm/dg_oradata3 | egrep 'secsize|dskname|grpname' kfdhdb.dskname: DATA31 ; 0x028: length=6 kfdhdb.grpname: DATA_4K ; 0x048: length=7 kfdhdb.secsize: 4096 ; 0x0b8: 0x1000
Summary
In this post I have shared how we can check disk sector sizes from Linux, and how to check and set sector size from the Oracle 19c database.
Some hybrid and all-flash-arrays (AFAs) see performance benefits from using a 4K block size for redo logs, this is due to the way they architected to work with fixed 4K geometry.
However, this is not the case for a Pure Storage FlashArray as it was architected to use a variable sector size with the smallest being 512 bytes.
If you are interested in seeing some test results this blog post has some useful data points.