In the last post I showed mathematically how ASM calculates the free usable space that it displays. However one of the first questions I received after presenting that internally was from someone concerned about all of the unusable space being saved just in case we lose a disk. (In my worked example it was 1/6 of the disk capacity! And what if we don’t care about restoring redundancy before we replace the failed disk?)

But it turns out that ASM actually handles this quite nicely – the “usable space” is used only for display and alerting. If a datafile needs to autoextend or if an archivelog needs to be created then ASM will keep using space until it completely exhausts everything physically available. It will go ahead and use the “reserved space” and just report a negative amount of free space! Here’s the output of my test confirming this behavior:

SQL> alter database datafile
2>  '+TEST01/js1_rh5lab20/datafile/filler.256.681667743' resize 18G;

Database altered.

.

rh5lab20$ asmcmd dgs
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB
MOUNTED  NORMAL  N         512   4096  1048576     38561     1529

  Req_mir_free_MB  Usable_file_MB  Offline_disks  Name
             7844           -3157              0  TEST01/

.

rh5lab20$ asmdf
Filesystem        SizeKB     UsedKB    AvailKB    Use% Mounted on
+TEST01         15727104   18959872   -3232768 120.56% TEST01

There is also a second way to get negative usable space in ASM. If you do experience a failure, then ASM will restore your redundancy level making new copies of the data that was on the failed disk(s). It will use the “reserved space” on your disks. After the rebuild is complete, ASM will recalculate the amount of reserved space it needs to protect from a second failure. If you were using most of the space before the first failure then ASM might report a negative amount of usable space after the rebuild.

It’s important for you to remember that negative free space doesn’t mean your data isn’t safe – you just won’t be able to restore redundancy after the failure. In other words, if you have negative free space then you cannot physically survive two subsequent failures without data loss (or three with high redundancy/triple mirroring). But ASM allows the negative numbers because it’s smart enough not to hang your database when it can be avoided (for example if archivelogs fill up the reported usable space).

This is quick and dirty. I hope to get a more polished write-up of what it means and how it works, but since I’m so busy right now I don’t know if I’ll get to it soon. In the meantime someone just might find this useful or informative so I’m just going to put it out there as it is.

This is an worked-through example of how ASM reports free space, taking into consideration “hot spare capacity” that is reserved to restore redundancy after the largest failure group dies.

ASM has 46 local disks attached to 6 controllers
- two controllers have 6 disks, other four have 8
- ASM has failure groups by controller

===========================================================================

Each disk is partitioned. Get the size of the ASM partition according to
the Linux kernel. All partitions are the same size.

# sfdisk -s /dev/sdav1
401624968

Now get the size info from ASM.

ASMCMD> lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB   Free_MB
MOUNTED  NORMAL  N         512   4096  1048576  18041706  18041326
Req_mir_free_MB  Usable_file_MB  Offline_disks  Name
        3137688         7451819              0  DATA01/

===========================================================================

-- size of individual disk in MB is kernel-reported KB divided by 1024
401624968 /1024 = 392211.8828125 M

-- ASM can't use partial AU's so truncate and we get Total_MB
392211 * 46 = 18041706

-- largest failure group [controller] has 8 disks so get Req_mir_free_MB
     (amount of "hot spare" capacity that is reserved to restore redundancy
     after a controller failure)
392211 * 8 = 3137688

-- asm reports that some space is used; use "Free_MB" to see total free space
= 18041326

-- with normal-mirroring, usable space is (totalfree-spare)/2
(18041326 -  3137688) / 2 = 7451819