Prior to Oracle Grid Infrastructure (GI) release 12.1.0.2, ASM Library (ASMLIB) was the only method to access storage devices by means of ASM Disks and Diskgroups. There were serious limitations to this method. The ASM disks were owned by GI home owner and both the owner and any users belonging to an operating system group designated as the OSASM group could read from and write to ASM disks using OS commands like strings, echo, dd etc. This led to compromising the security of data as well as accidental corruption of the data in the ASM disks and ASM disks themselves.
With Oracle Grid Infrastructure release 12.1.0.2, Oracle has introduced a new component called Oracle Automatic Storage Management Filter Driver (Oracle ASMFD) which is installed with an Oracle Grid Infrastructure installation. It is intended to overcome the limitations mentioned earlier and replace traditional ASMLIB. ASMFD is currently available only on the Linux operating system and can be completely managed by the ASMCMD command Line interface.
Oracle ASMFD is a kernel module that resides in the I/O path of the Oracle ASM disks. ASMFD uses the filter driver to validate write I/O requests to Oracle ASM disks. Any write I/O requests that are not issued by Oracle software (e.g. using dd command) are not committed to ASM disks. Thus it helps to prevent users, including those with administrative privileges, from inadvertently overwriting Oracle ASM disks, thereby preventing corruption of the data stored on ASM disk(s) within the disk group(s). In case ASM disks have been configured on disk partitions, the filter protects the area on the disk managed by Oracle ASMFD, assuming the partition table is left untouched by the user.
Besides, Oracle ASMFD eliminates the need to rebind disk devices used with Oracle ASM each time the system is restarted.
If you have an existing Oracle ASM library driver (Oracle ASMLIB) configuration, and you want to employ ASMFD, you would need to configure Oracle ASMFD. This will remove Oracle ASMLIB and Oracle ASM devices will be configured to use Oracle ASMFD. In this article, I will demonstrate configuration of Oracle ASMFD in a standalone environment.
Currently, I have Oracle ASMLIB configured on my standalone system. OS user oracle is the owner of both GI home and Database home.
Overview:
- Verify that currently ASMLIB has been configured and ASMFD is not configured yet
- Create a table in an ASM tablespace
- Migrate ASM disks to use ASMFD
- Configure ASMFD
- Start Clusterware stack
- Verify that ASMFD has been successfully configured and ASM disks have been migrated
- Set ASM_DISKSTRING and AFD_DISKSTRING to ‘AFD:*’
- Validate that ASM Disk groups have migrated and are using ASMFD
- Verify that our data is intact after migration
- Verify that when filter is enabled it does not allow OS commands to alter the contents of ASM disks
Implementation:
Verify that currently ASMLIB has been configured and ASMFD is not configured yet
We can verify that Oracle ASMLIB is configured and driver for oracleasm is currently loaded.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
[root@host01 ~]# lsmod | grep oracle <span style="font-weight: bold; color: red;">oracleasm</span> 48660 1 [root@host01 ~]# oracleasm scandisks [root@host01 ~]# oracleasm listdisks ASMDISK01 ASMDISK02 ASMDISK03 ASMDISK04 ASMDISK05 ASMDISK06 ASMDISK07 ASMDISK08 ASMDISK09 ASMDISK10 |
Let’s check the permissions and location of ASM Disks.
The ASM Disks are block devices in /dev/oracleasm/disks owned by the GI owner “oracle” user. OS user oracle and anyone belonging to OS group “dba” (which maps to OSASM privilege) can read from and write to ASM disks. Thus the ASM disks are vulnerable to accidental Disk and or Data corruption.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
[root@host01 disks]# ls -lrt /dev/oracleasm/disks total 0 brw-rw---- 1 oracle dba 8, 6 Apr 20 11:02 ASMDISK01 brw-rw---- 1 oracle dba 8, 7 Apr 20 11:02 ASMDISK02 brw-rw---- 1 oracle dba 8, 8 Apr 20 11:02 ASMDISK03 brw-rw---- 1 oracle dba 8, 9 Apr 20 11:02 ASMDISK04 brw-rw---- 1 oracle dba 8, 10 Apr 20 11:02 ASMDISK05 brw-rw---- 1 oracle dba 8, 11 Apr 20 11:02 ASMDISK06 brw-rw---- 1 oracle dba 8, 12 Apr 20 11:02 ASMDISK07 brw-rw---- 1 oracle dba 8, 13 Apr 20 11:02 ASMDISK08 brw-rw---- 1 oracle dba 8, 14 Apr 20 11:02 ASMDISK09 brw-rw---- 1 oracle dba 8, 15 Apr 20 11:02 ASMDISK10 |
We can verify that ASMFD is not configured.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
[oracle@host01 ~]$ asmcmd afd_lsdsk <span style="font-weight: bold; color: red;">Failed to load AFD library.</span> [oracle@host01 disks]$ asmcmd afd_state <span style="font-weight: bold; color: red;">ASMCMD-9526: The AFD state is 'NOT INSTALLED' and filtering is 'DEFAULT' on host 'host01.example.com'</span> ASM>SELECT SYS_CONTEXT('SYS_ASMFD_PROPERTIES', 'AFD_STATE') FROM DUAL; SYS_CONTEXT('SYS_ASMFD_PROPERTIES','AFD_STATE') -------------------------------------------------------------------------------- <span style="font-weight: bold; color: red;">NOT AVAILABLE</span> |
Create a table in an ASM tablespace
Let’s find out candidate disks which are not part of any disk group yet.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
[oracle@host01 ~]$ asmcmd lsdsk --candidate Path ORCL:ASMDISK05 ORCL:ASMDISK06 ORCL:ASMDISK07 ORCL:ASMDISK08 ORCL:ASMDISK09 ORCL:ASMDISK10 ASM> create diskgroup test external redundancy disk 'ORCL:ASMDISK10'; ORCL>create tablespace ASMTBS datafile '+TEST'; ORCL>create table hr.asm_tab tablespace ASMTBS as select * from hr.employees; select count(*) from hr.asm_tab where last_name = 'King'; COUNT(*) ---------- 2 |
Migrate ASM disks to use ASMFD
Let’s find out disks in DATA and TEST disk groups
|
1 2 3 4 5 6 7 8 9 10 11 |
[oracle@host01 disks]$ asmcmd lsdsk -G DATA Path ORCL:ASMDISK01 ORCL:ASMDISK02 ORCL:ASMDISK03 ORCL:ASMDISK04 [oracle@host01 disks]$ asmcmd lsdsk -G TEST Path ORCL:ASMDISK10 |
Let’s make a note of current disk discovery path. It currently points to ASMLIB disks.
|
1 2 3 |
[oracle@host01 disks]$ asmcmd dsget parameter:<span style="font-weight: bold; color: red;">ORCL:*</span> profile:ORCL:* |
Make sure ASM is using SPFILE
|
1 2 3 4 5 6 7 |
[oracle@host01 disks]$ srvctl config asm ASM home: Password file: +DATA/orapwasm ASM listener: LISTENER <span style="font-weight: bold; color: red;">Spfile: +DATA/ASM/ASMPARAMETERFILE/registry.253.872686839</span> ASM diskgroup discovery string: ORCL:* |
Now, let’s start with the migration.
Update the ASM Disk discovery path to enable ASMFD lookup
Once we label the disk, it will show up as AFD: and that name will be used for disk discovery.
AS GI owner , set the ASM discovery string to include AFD.* as well:
|
1 2 3 4 5 6 |
[oracle@host01 root]$ asmcmd dsset 'ORCL:*','AFD:*' [oracle@host01 disks]$ asmcmd dsget parameter:ORCL:*, AFD:* profile:ORCL:*,AFD:* |
We have updated ASM disk discovery string. Now, when we configure ASMFD it would be able to discover and migrate the existing disks.
Bring down the Clusterware stack
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
ORCL>shu immediate; [root@host01 dev]# srvctl stop asm -f [root@host01 dev]# crsctl stop has CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'host01' CRS-2673: Attempting to stop 'ora.evmd' on 'host01' CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'host01' CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'host01' succeeded CRS-2677: Stop of 'ora.evmd' on 'host01' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'host01' CRS-2677: Stop of 'ora.cssd' on 'host01' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'host01' has completed CRS-4133: Oracle High Availability Services has been stopped. |
Now, we are ready for the migration process.
Configure ASMFD
This step will deconfigure ASMLIB, configure ASMFD and migrate the disks from ASMLIB to ASMFD.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
[root@host01 ~]# asmcmd afd_configure Connected to an idle instance. AFD-627: AFD distribution files found. AFD-636: Installing requested AFD software. AFD-637: Loading installed AFD drivers. AFD-9321: Creating udev for AFD. AFD-9323: Creating module dependencies - this may take some time. AFD-9154: Loading 'oracleafd.ko' driver. AFD-649: Verifying AFD devices. AFD-9156: Detecting control device '/dev/oracleafd/admin'. AFD-638: AFD installation correctness verified. Modifying resource dependencies - this may take some time. ASMCMD-9524: AFD configuration failed 'ERROR: OHASD start failed' Modifying resource dependencies - this may take some time. ASMCMD-9524: AFD configuration failed 'ERROR: OHASD start failed' |
We can ignore the last error for now but we do have a working ASMFD now.
|
1 2 3 4 5 |
[root@host01 ~]# asmcmd afd_state Connected to an idle instance. <span style="font-weight: bold; color: red;">ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'DEFAULT' on host 'host01.example.com'</span> |
Note that the folder for oracleasm has been emptied and ASMFD has created a folder /dev/oracleafd where all the ASM disks are visible such that root is the owner of disks and only root can read from or write to ASM disks.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
[[root@host01 dev]# cd /dev ls -l |grep oraclea drwxrwx--- 3 oracle dba 80 Apr 20 17:22 <span style="font-weight: bold; color: red;">oracleafd</span> drwxr-xr-x 2 root root 40 Apr 20 17:15 oracleasm [root@host01 oracleasm]# ls -l /dev/oracleasm/* <span style="font-weight: bold; color: red;">ls: /dev/oracleasm/*: No such file or directory</span> [root@host01 dev]# ls -l /dev/oracleafd/disks total 40 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 10 Apr 20 17:22 ASMDISK01 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 10 Apr 20 17:22 ASMDISK02 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 10 Apr 20 17:22 ASMDISK03 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 10 Apr 20 17:22 ASMDISK04 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 11 Apr 20 17:22 ASMDISK05 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 11 Apr 20 17:22 ASMDISK06 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 11 Apr 20 17:22 ASMDISK07 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 11 Apr 20 17:22 ASMDISK08 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 11 Apr 20 17:22 ASMDISK09 <span style="font-weight: bold; color: red;">-rw-r--r-- 1 root root</span> 11 Apr 20 17:22 ASMDISK10 |
The disks can now be viewed from ASM Filter Driver Module.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
[root@host01 oracleasm]# asmcmd afd_lsdsk Connected to an idle instance. -------------------------------------------------------------------------------- Label Filtering Path ================================================================================ ASMDISK01 ENABLED /dev/sda6 ASMDISK02 ENABLED /dev/sda7 ASMDISK03 ENABLED /dev/sda8 ASMDISK04 ENABLED /dev/sda9 ASMDISK05 ENABLED /dev/sda10 ASMDISK06 ENABLED /dev/sda11 ASMDISK07 ENABLED /dev/sda12 ASMDISK08 ENABLED /dev/sda13 ASMDISK09 ENABLED /dev/sda14 ASMDISK10 ENABLED /dev/sda15 |
We can verify that the configuration process has stopped the ASMLIB kernel module
|
1 2 3 4 |
[root@host01 dev]# service oracleasm status Checking if ASM is loaded: <span style="font-weight: bold; color: red;">no</span> Checking if /dev/oracleasm is mounted: <span style="font-weight: bold; color: red;">no</span> |
We are almost done with the migration as ASMFD is loaded and the existing disks are already mapped. Once we start the Clusterware stack, we will update few configuration settings to make ASMFD persistent.
Start Clusterware stack
|
1 2 |
[root@host01 disks]# crsctl start has CRS-4123: Oracle High Availability Services has been started. |
We can verify that driver for ASMFD has been loaded.
|
1 2 3 4 5 |
root@host01 oracleasm]# lsmod | grep oracle <span style="font-weight: bold; color: red;">oracleafd</span> 221120 1 root@host01 dev]# srvctl start asm |
Verify that ASMFD has been successfully configured and ASM disks have been migrated
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
ASM> SELECT SYS_CONTEXT('SYS_ASMFD_PROPERTIES', 'AFD_STATE') FROM DUAL; SYS_CONTEXT('SYS_ASMFD_PROPERTIES','AFD_STATE') ----------------------------------------------------------------------------------------------------------- <span style="font-weight: bold; color: red;">CONFIGURED</span> ASM> select name, path, total_mb, free_mb from v$asm_disk; NAME PATH TOTAL_MB FREE_MB --------------- -------------------- ---------- ---------- AFD:ASMDISK09 0 0 AFD:ASMDISK08 0 0 AFD:ASMDISK05 0 0 AFD:ASMDISK07 0 0 AFD:ASMDISK06 0 0 ASMDISK01 AFD:ASMDISK01 3914 3865 ASMDISK02 AFD:ASMDISK02 3914 3862 ASMDISK03 AFD:ASMDISK03 3914 3867 ASMDISK04 AFD:ASMDISK04 3914 3871 ASMDISK10 AFD:ASMDISK10 3914 3761 |
Set ASM_DISKSTRING and AFD_DISKSTRING to ‘AFD:*’
Currently, parameter ASM_DISKSTRING points to both ASMLIB and ASMFD disks. Since ASMLIB has been deconfigured, we can remove ORCL:* from the parameter so that the parameter points to ASMFD disks only.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
ASMFD disks only. ASM>sho parameter diskstring NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ asm_diskstring string <span style="font-weight: bold; color: red;">ORCL:*, AFD:*</span> [oracle@host01 ~]$ asmcmd dsget <span style="font-weight: bold; color: red;">parameter:ORCL:*, AFD:*</span> profile:ORCL:*,AFD:* ASM>ALTER SYSTEM set asm_diskstring = 'AFD:*'; [oracle@host01 ~]$ asmcmd dsget <span style="font-weight: bold; color: red;">parameter:AFD:*</span> profile:AFD:* |
We can also set discovery string for ASMFD disks to ‘AFD:*’
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
ASM>ALTER SYSTEM AFD_DISKSTRING SET 'AFD:*'; [oracle@host01 ~]$ asmcmd afd_dsget AFD discovery string: <span style="font-weight: bold; color: red;">'AFD:*'</span> ASM>SELECT SYS_CONTEXT('SYS_ASMFD_PROPERTIES', 'AFD_DISKSTRING') FROM DUAL; SYS_CONTEXT('SYS_ASMFD_PROPERTIES','AFD_DISKSTRING') ----------------------------------------------------------------------------------------------------------- <span style="font-weight: bold; color: red;">'AFD:*'</span> [oracle@host01 ~]$ srvctl config asm ASM home: Password file: +DATA/orapwasm ASM listener: LISTENER Spfile: +DATA/ASM/ASMPARAMETERFILE/registry.253.872686839 <span style="font-weight: bold; color: red;">ASM diskgroup discovery string: AFD:*</span> |
Validate that ASM Diskgroups have migrated and are using ASMFD
|
1 2 3 4 5 |
[oracle@host01 ~]$ srvctl status diskgroup -diskgroup DATA <span style="font-weight: bold; color: red;">Disk Group DATA is running on host01</span> [oracle@host01 ~]$ srvctl status diskgroup -diskgroup TEST <span style="font-weight: bold; color: red;">Disk Group TEST is running on host01</span> |
Validate if the diskgroups are migrated and using ASMFD
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
[oracle@host01 ~]$ asmcmd lsdsk -G DATA Path AFD:ASMDISK01 AFD:ASMDISK02 AFD:ASMDISK03 AFD:ASMDISK04 [oracle@host01 ~]$ asmcmd lsdsk -G TEST Path AFD:ASMDISK10 root@host01 disks]# crsctl stat res -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- <span style="font-weight: bold; color: red;">ora.DATA.dg</span> <span style="font-weight: bold; color: red;">ONLINE ONLINE</span> <span style="font-weight: bold; color: red;">host01</span> <span style="font-weight: bold; color: red;">STABLE</span> ora.LISTENER.lsnr ONLINE ONLINE host01 STABLE <span style="font-weight: bold; color: red;">ora.TEST.dg</span> <span style="font-weight: bold; color: red;">OFFLINE OFFLINE</span> <span style="font-weight: bold; color: red;">host01</span> <span style="font-weight: bold; color: red;">STABLE</span> ora.asm ONLINE ONLINE host01 Started,STABLE ora.ons OFFLINE OFFLINE host01 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.cssd 1 ONLINE ONLINE host01 STABLE ora.diskmon 1 OFFLINE OFFLINE STABLE ora.evmd 1 ONLINE ONLINE host01 STABLE ------------------------------------------------------------------------------- [oracle@host01 ~]$ asmcmd afd_scan [oracle@host01 ~]$ asmcmd afd_lsdsk -------------------------------------------------------------------------------- Label Filtering Path ================================================================================ ASMDISK01 ENABLED /dev/sda6 ASMDISK02 ENABLED /dev/sda7 ASMDISK03 ENABLED /dev/sda8 ASMDISK04 ENABLED /dev/sda9 ASMDISK05 ENABLED /dev/sda10 ASMDISK06 ENABLED /dev/sda11 ASMDISK07 ENABLED /dev/sda12 ASMDISK08 ENABLED /dev/sda13 ASMDISK09 ENABLED /dev/sda14 ASMDISK10 ENABLED /dev/sda15 |
Verify that our data is intact after migration
|
1 2 3 4 5 6 7 8 9 10 11 |
ORCL> select tablespace_name, file_name from dba_data_files where tablespace_name = 'ASMTBS' ; TABLESPACE_NAME FILE_NAME ------------------------------ ------------------------------ ASMTBS +TEST/ORCL/DATAFILE/asmtbs.256.877540769 ORCL> select count(*) from hr.asm_tab where last_name = 'King'; COUNT(*) ---------- 2 |
Let’s explore the disks located under /dev/oracleafd/disks/ .
These are actually just the text files owned by root pointing to the actual disks on which the ASM disks are configured as opposed to ASMLIB disks which were block devices owned by GI owner.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
[root@host01 disks]# file /dev/oracleafd/disks/ASM* /dev/oracleafd/disks/ASMDISK01: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK02: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK03: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK04: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK05: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK06: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK07: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK08: <span style="font-weight: bold; color: red;">ASCII text</span> /dev/oracleafd/disks/ASMDISK09: <span style="font-weight: bold; color: red;">ASCII text</span> [root@host01 disks]# ls -l /dev/oracleafd/disks/ASM* total 40 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 10 Apr 20 17:22 ASMDISK01 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 10 Apr 20 17:22 ASMDISK02 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 10 Apr 20 17:22 ASMDISK03 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 10 Apr 20 17:22 ASMDISK04 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 11 Apr 20 17:22 ASMDISK05 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 11 Apr 20 17:22 ASMDISK06 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 11 Apr 20 17:22 ASMDISK07 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 11 Apr 20 17:22 ASMDISK08 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 11 Apr 20 17:22 ASMDISK09 -rw-r--r-- 1 <span style="font-weight: bold; color: red;">root root</span> 11 Apr 20 17:22 ASMDISK10 [root@host01 disks]# cat /dev/oracleafd/disks/ASM* /dev/sda6 /dev/sda7 /dev/sda8 /dev/sda9 /dev/sda10 /dev/sda11 /dev/sda12 /dev/sda13 /dev/sda14 /dev/sda15 [root@host01 log]# ls -l /dev/sda* brw-r----- 1 root disk 8, 0 Apr 20 17:14 /dev/sda brw-r----- 1 root disk 8, 1 Apr 20 17:14 /dev/sda1 brw-r----- 1 root disk 8, 10 Apr 20 17:14 /dev/sda10 brw-r----- 1 root disk 8, 11 Apr 20 17:14 /dev/sda11 brw-r----- 1 root disk 8, 12 Apr 20 17:14 /dev/sda12 brw-r----- 1 root disk 8, 13 Apr 20 17:14 /dev/sda13 brw-r----- 1 root disk 8, 14 Apr 20 17:14 /dev/sda14 brw-r----- 1 root disk 8, 15 Apr 27 11:23 /dev/sda15 brw-r----- 1 root disk 8, 2 Apr 20 17:14 /dev/sda2 brw-r----- 1 root disk 8, 3 Apr 20 17:14 /dev/sda3 brw-r----- 1 root disk 8, 4 Apr 20 17:14 /dev/sda4 brw-r----- 1 root disk 8, 5 Apr 20 17:14 /dev/sda5 brw-r----- 1 root disk 8, 6 Apr 27 10:52 /dev/sda6 brw-r----- 1 root disk 8, 7 Apr 20 17:14 /dev/sda7 brw-r----- 1 root disk 8, 8 Apr 20 17:14 /dev/sda8 brw-r----- 1 root disk 8, 9 Apr 20 17:14 /dev/sda9 |
Verify that when filter is enabled it does not allow OS commands to alter the contents of ASM disks
We can check that filtering is enabled on AFD disks.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
[oracle@host01 ~]$ asmcmd afd_scan [oracle@host01 ~]$ asmcmd afd_lsdsk -------------------------------------------------------------------------------- Label Filtering Path ================================================================================ ASMDISK01 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda6 ASMDISK02 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda7 ASMDISK03 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda8 ASMDISK04 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda9 ASMDISK05 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda10 ASMDISK06 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda11 ASMDISK07 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda12 ASMDISK08 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda13 ASMDISK09 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda14 ASMDISK10 <span style="font-weight: bold; color: red;">ENABLED</span> /dev/sda15 |
Now non-root users have no way to read directly from ASM disks using OS command “strings”. Root user can still read the content of ASM disks cirectly.
|
1 2 3 4 5 6 7 8 |
[<span style="font-weight: bold; color: red;">oracle</span>@host01 disks]$ strings /dev/sda15 | grep King <span style="font-weight: bold; color: red;">strings: /dev/sda15: Permission denied</span> [<span style="font-weight: bold; color: red;">root</span>@host01 log]# strings /dev/sda15 | grep King <span style="font-weight: bold; color: red;">King</span> <span style="font-weight: bold; color: red;">King</span> |
For ASM disk writes, we get a layer of protection for both root and non-root users. I will demonstrate it for both “echo” and “dd” commands.
|
1 2 3 |
[<span style="font-weight: bold; color: red;">oracle</span>@host01 disks]$ echo 'corrupt' > /dev/sda15 <span style="font-weight: bold; color: red;">bash: /dev/sda15: Permission denied</span> |
When we try to write to underlying disk using “echo” as a root user, although the echo command is successful, the I/O were rejected by ASMFD which can be confirmed by looking into /var/log/messages.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[<span style="font-weight: bold; color: red;">root</span>@host01 log]# echo 'corrupt' > /dev/sda15 [root@host01 log]# more /var/log/messages .... quest_fn: <span style="font-weight: bold; color: red;">write IO on ASM managed device (major=8/minor=15) not supported i</span>=10 start=186868143 seccnt=4 pstart=186868143 pend=194884515 Apr 27 11:28:51 host01 kernel: Buffer I/O error on device sda15, logical block 0 Apr 27 11:28:51 host01 kernel: <span style="font-weight: bold; color: red;">lost page write due to I/O error on sda15</span> .... |
Similarly when we try to modify contents of ASM disk using ‘dd‘ command, GI owner is denied permission where command succeeds for root without any effect though as can again be verified from /var/log/messages.
|
1 2 3 4 5 6 7 8 9 |
[<span style="font-weight: bold; color: red;">oracle</span>@host01 disks]$ dd if=/dev/zero of=/dev/sda6 count=100 bs=2M <span style="font-weight: bold; color: red;">dd: opening `/dev/sda6': Permission denied</span> [<span style="font-weight: bold; color: red;">root</span>@host01 log]# dd if=/dev/zero of=/dev/sda15 count=10 bs=2M 10+0 records in 10+0 records out <span style="font-weight: bold; color: red;">20971520 bytes (21 MB) copied, 0.013702 seconds, 1.5 GB/s</span> [root@host01 log]# more /var/log/messages ... Apr 27 11:34:06 host01 kernel: F 4306319.693/150427060406 pdflush[363] afd_mkrequest_fn: <span style="font-weight: bold; color: red;">write IO on ASM managed device (major=8/minor=15) not supported</span> i=10 start=186868143 seccnt=4 pstart=186868143 pend=194884515 |
Let us disable filter for all disks
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
[oracle@host01 root]$ asmcmd afd_filter -d [oracle@host01 root]$ asmcmd afd_lsdsk -------------------------------------------------------------------------------- Label Filtering Path ================================================================================ ASMDISK01 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda6 ASMDISK02 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda7 ASMDISK03 <span style="font-weight: bold; color: red;"> DISABLED</span> /dev/sda8 ASMDISK04 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda9 ASMDISK05 <span style="font-weight: bold; color: red;"> DISABLED</span> /dev/sda10 ASMDISK06 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda11 ASMDISK07 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda12 ASMDISK08 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda13 ASMDISK09 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda14 ASMDISK10 <span style="font-weight: bold; color: red;">DISABLED</span> /dev/sda15 |
Now that filter is disabled, let us try to perform write I/O on ASMDISK10.
It can be seen although non-root user can still not write to ASM disk but root user can alter the contents of ASM Disks leading to corruption of Data in the ASM Disks and ASM disks themselves.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
[<span style="font-weight: bold; color: red;">oracle</span>@host01 disks]$ echo 'corrupt' > /dev/sda15 <span style="font-weight: bold; color: red;">bash: /dev/sda15: Permission denied</span> [<span style="font-weight: bold; color: red;">root</span>@host01 log]# echo 'corrupt' > /dev/sda15 [root@host01 ~]# strings /dev/sda15 | grep corrupt <span style="font-weight: bold; color: red;">corrupt</span> [<span style="font-weight: bold; color: red;">oracle</span>@host01 disks]$ dd if=/dev/zero of=/dev/sda15 count=100 bs=2M <span style="font-weight: bold; color: red;">dd: opening `/dev/sda6': Permission denied</span> [<span style="font-weight: bold; color: red;">root</span>@host01 ~]# dd if=/dev/zero of=/dev/sda15 count=1000 bs=2M 1000+0 records in 1000+0 records out <span style="font-weight: bold; color: red;">2097152000 bytes (2.1 GB) copied, 4.38152 seconds, 479 MB/s</span> ORCL>alter system flush buffer_cache; System altered. ORCL>select count(*) from hr.asm_tab where last_name = 'King'; select count(*) from hr.asm_tab where last_name = 'King' * ERROR at line 1: <span style="font-weight: bold; color: red;">ORA-01578: ORACLE data block corrupted (file # 2, block # 130)</span> <span style="font-weight: bold; color: red;">ORA-01110: data file 2: '+TEST/ORCL/DATAFILE/asmtbs.256.877540769'</span> ASM>select name, STATE from v$asm_diskgroup; NAME STATE ------------------------------ ----------- DATA MOUNTED ASM>alter diskgroup test mount; alter diskgroup test mount * ERROR at line 1: ORA-15032: not all alterations performed <span style="font-weight: bold; color: red;">ORA-15017: diskgroup "TEST" cannot be mounted</span> ORA-15040: diskgroup is incomplete ASM> select name, path, total_mb, free_mb from v$asm_disk; NAME PATH TOTAL_MB FREE_MB --------------- -------------------- ---------- ---------- <span style="font-weight: bold; color: red;">AFD:ASMDISK10 0 0</span> AFD:ASMDISK09 0 0 AFD:ASMDISK08 0 0 AFD:ASMDISK05 0 0 AFD:ASMDISK07 0 0 AFD:ASMDISK06 0 0 ASMDISK01 AFD:ASMDISK01 3914 3865 ASMDISK02 AFD:ASMDISK02 3914 3862 ASMDISK03 AFD:ASMDISK03 3914 3867 ASMDISK04 AFD:ASMDISK04 3914 3871 10 rows selected. |
Conclusion:
Oracle Automatic Storage Management Filter Driver (Oracle ASMFD) can overcome the limitations of and replace traditional ASMLIB. When filter is enabled,
– Contents of ASMFD disks can only be read using OS commands by root user
– Contents of ASMFD disks cannot be altered using OS commands by any user
Thus, any write I/O performed to ASM disks by will be validated by the filter and corruption of data on ASM disks by non-oracle software will be eliminated.
References:
- https://docs.oracle.com/database/121/LADBI/oraclerestart.htm#LADBI8076
- http://docs.oracle.com/cd/E55747_01/doc.41/e58244/GUID-3CFAEE85-7F6A-4B0A-91C8-E6A2CA221663.htm#DMRIG149
- http://flashdba.com/2014/07/25/oracle-12-1-0-2-asm-filter-driver-first-impressions/
- http://www.oraclebuffer.com/general-discussions/an-encounter-with-asm-filter-driver-asmfd-oracle-gi-12c-new-feature/
- https://rmchandran.wordpress.com/2014/12/30/first-look-at-asm-filter-driver-asmfd/
Load comments