How to monitor aggregate free space using OnCommand?

[jont]

Hi all,

I've been asked by a large customer to generate a report in OnCommand Core that will report on the performance-relevant aspects of capacity management within an aggregate - in other words, the % free blocks available for writes within an aggregate (ignoring all reservations, guarantees, or the like - just the free space available to the write allocator). The reason to do this is to be able to proactively order additional storage for an aggregate before write performance is impacted.

Has anyone found a way to monitor this value from within OC?

[agireesh]

Hi Jont,

Please use the below report to find the space available in aggregate.  Please use "dfm report view <report-name>" command to view the report data.

- aggregates-capacity-report (this report have "Aggregate Used Capacity" field)

- aggregates-capacity (you can use "Total" and "Used" filed to get the available space in aggregate)

or

create the custom report using below command with require fields.

dfm report create -R Aggregate -f "Aggregate.Id,Aggregate.FullName,Aggregate.TotalSpace,Aggregate.SpaceAvailable,Aggregate.AvailablePct,Aggregate.Used" aggr_report

[root@shoemake-rhel ~]# dfm report create -R Aggregate -f "Aggregate.Id,Aggregate.FullName,Aggregate.TotalSpace,Aggregate.SpaceAvailable,Aggregate.AvailablePct,Aggregate.Used" aggr_report

Report aggr_report created.

[root@shoemake-rhel ~]# dfm report view aggr_report

Aggregate Id Aggregate Full Name                 Aggregate Total Space Aggregate Space Available Aggregate Space Available in % Aggregate Used Capacity

------------ ----------------------------------- --------------------- ------------------------- ------------------------------ -----------------------

208          sc41a37-2b:aggr0_sc41a37_2a_2b_02_0             781088496                  31933084                            4.1               749155412

210          sc41a37-2a:aggr_hamlet1                         781088496                 773737672                           99.1                 7350824

Regards,

Gireesh

[jont]

Hi Gireesh and the Community,

I'm afraid those values don't describe what I'm looking to monitor. What I'm after is effectively the %used you would see for an aggregate whose contents are fully thin provisioned.

If I use your report for one of my customer's aggregates,

U:\>dfm report create -R Aggregate -f "Aggregate.Id,Aggregate.FullName,Aggregate.TotalSpace,Aggregate.SpaceAvailable,Aggregate.AvailablePct,Aggregate.Used" aggr_report

Report aggr_report created.

U:\>dfm report view aggr_report

Aggregate Id Aggregate Full Name                 Aggregate Total Space Aggregate Space Available Aggregate Space Available in % Aggregate Used Capacity

------------ ----------------------------------- --------------------- ------------------------- ------------------------------ -----------------------

...

1019         FILER:aggr3_SATA                            14227331328                 355366136                            2.5             13871965192

...

I see that this aggregate has about 2% "Space Available in %". This number is, however, irrelevant as far as performance is concerned since it could, for example, be space reserved to an empty volume.

If I look at the same aggregate using aggr show_space,

FILER> aggr show_space aggr3_SATA

Aggregate 'aggr3_SATA'

    Total space    WAFL reserve    Snap reserve    Usable space       BSR NVLOG           A-SIS

  16640153600KB    1664015360KB     748806912KB   14227331328KB             0KB      84634020KB

Space allocated to volumes in the aggregate

Volume                          Allocated            Used       Guarantee

xxxxxxxxxxxxxxxxxxxxxx        105440140KB       1136960KB          volume

xxxxxxxxxxx                  1992818688KB    1582846256KB          volume

xxxxxxxxxxxxxx               3279188424KB    2477813136KB          volume

xxxxxxxx                     1381265860KB    1071100556KB          volume

xxxxxxxxxxxx                 2551651440KB    2036075376KB          volume

xxxxxxxxxxxxxxx               105440140KB       1488120KB          volume

xxxxxxxxxxxxxxxx               52720068KB       4549660KB          volume

xxxxxxxxxxxxxxxxxxxxxxx        52720068KB       5763756KB          volume

xxxxxxxx                     4212333680KB    2718154172KB          volume

xxxxxxxxxxx                    52720068KB      13628444KB          volume

Aggregate                       Allocated            Used           Avail

Total space                 13786298576KB    9912556436KB     355362476KB

Snap reserve                  748806912KB       4668912KB     744138000KB

WAFL reserve                 1664015360KB     167033616KB    1496981744KB

First calculate the physical free space in the aggregate,

Total Space - Physical Used Space (BSR NVLOG + A-SIS + Total space Used + Snap reserve Used + WAFL reserve Used):

16640153600KB - 0KB - 84634020KB - 9912556436KB - 4668912KB - 167033616KB = 6471260616KB physical free space

Then the physical % free space in this aggregate is: physical free space / total physical space,

6471260616KB / 16640153600KB = 39% physical free space (not taking WAFL reserve into account).

Or, taking the WAFL reserve into account as we do in our recommendations on maximum (the way we measure things in our best practices):

(Total space - WAFL reserve - (BSR NVLOG + A-SIS + Total space Used + Snap reserve Used)) / (Total space - WAFL reserve )

(16640153600KB - 1664015360KB - 0KB - 84634020KB - 9912556436KB - 4668912KB) / (16640153600KB - 1664015360KB) = 33% physical free space.

To ask my question more precisely, Is there any way to get OnCommand to report on the physical free space available in an aggregate?

Thanks!

[jont]

I realize that the above is somewhat pedantic, a simple measure of

Physical Space Free == 1 - (Total Space Used / Usable Space)

would suffice for many cases... (it is 30% for the above example). However, I can't find any equivalent data for either Total Space Used or Usable Space in OnCommand.

[agireesh]

Hi Jont,

I think,you are doing wrong calculation.  Here, I  am going to defined each field and latter I'll calculate space available in aggregate.

Usually, we use "df -A <aggr_name>" and "aggr show_space <aggr_name>" commands to list the aggregate space attributes

df -A <aggr_name> command command will list aggregate capacity, used capacity and available capacity.

Similarly,  "aggr show_space <aggr-name>" command display space usage of aggregate in more detail.this command shows the space usage for each flexible volume within an aggregate.

Output of this command display below fields.

Total space: This is total size of aggregate.

Usable space: The capacity available to applications for storing data, this is also call aggregate capacity. you can also check the aggregate capacity space using df -A <aggr-name> command.

WAFL reserve and Snap reserve: WAFL reserve is WAFL metadata and user can not have any control on this.Snap reserve is allocated when user take the snaphot of aggregate.

A-SIS: I am not sure about this field but I think this is space saved in aggregate due to dedupe.

Allocated: The capacity allocated to applications or users for storing data, including effects of thin provisioning. For space guarantee as "volume" (thick provisioning), this field value equal to size of volume and for "none" and "file" space guarantee (thin provisioning), value should be the real data store in volume or volume used space.  

Used: This field show application data stored in volume.

"Allocated" and "Used" field's value are with respect to volume.

So, After defining above fields, we can relate the fields as below.

Total space = Usable space + WAFL reserve + Snap reserve

Usable space = Total space - (WAFL reserve + Snap reserve)

14227331328 = 16640153600 - (1664015360 + 748806912)

              AND

Aggregate                       Allocated                  Used                        Avail

Total space                 13786298576KB               9912556436KB                  355362476KB

                            (Sum of all volume sizes if all            (Application data stored on all volumes)           (Available space on aggregate)

                                                          volumes are thick provisioning)       

Free space = Avail/Usable space*100

             355362476/14227331328*100 = 2.4977451343993891698309649431396%

If you verify this data to report data then report also showing "Available in %" field's value as "2.5%".

So, you can use same report to check or monitor the free space available in aggregate.

Please revert for further clarification if any

Regards,

Gireesh

[amirm · Accepted solution]

Hi Jon,

It seems that DFM is not collecting/calculating the value for Total space used/size-volume-used. You can try the report:

#dfm report create -R Aggregate -f "Aggregate.ALL" aggr_report_ALL

This will eventually take all the available attributes collected for aggr object. In this list I could not locate anything equivalent of Total space used/size-volume-used that is recorded in DFM DB.  

There are 2 ways that may give the report that you are looking for:

Option#1: Using ZAPIs:

The “aggr-space-list-info” lists the info as(listing only values of interest in this case):

size-volume-allocated = 897802240

size-volume-used = 641814528

size-used = 898043904

size-free = 45674496

size-metadata = 5840896

size-nominal = 943718400

This gives : ====> 1 - (641814528/943718400) = 31.99%

Using new ZExplore Development Interface(ZEDI) shipped with NMSDK 5.0 release you can generate the code for getting this values from ONTAP quite easily:

https://communities.netapp.com/community/interfaces_and_tools/developer/zedi

Option#2: Using PowerShell cmd lets:

This is also pretty quick:

PS C:\Users\amirm> Get-NaAggrSpace |select @{Label="Aggr Name";Expression={$_.AggregateName}},@{Label="Free Space";Expression={[Math]::Round((1-($_.SizeVolumeUsed/$_.SizeNominal))*100,2)}} | format-table -AutoSize

Aggr Name Free Space

--------- ----------

aggr1 9.48

aggr2 6.74

aggr0 32.46

Values reported on controller:

vsim> aggr show_space aggr0

Aggregate 'aggr0'

    Total space    WAFL reserve    Snap reserve    Usable space       BSR NVLOG           A-SIS          Smtape

      1024000KB        102400KB             0KB        921600KB             0KB             0KB             0KB

Space allocated to volumes in the aggregate

Volume                          Allocated            Used       Guarantee

vol0                             876760KB        626040KB          volume

Aggregate                       Allocated            Used           Avail

Total space                      876760KB        626040KB         44608KB

Snap reserve                          0KB             0KB             0KB

WAFL reserve                     102400KB          7612KB         94788KB

===> 1 - (626040KB/921600KB) = 32.07%

Regards,

-Amir

[jont]

Hi Amir,

thanks for the pointers to the data, I'll looking into scripting a solution until DFM is able to collect these counters natively. Is there something I can do to help get the development in DFM moving (have the customer open an RFE / case / open a BURT)?

Cheers,

--Jon

[amirm]

Sure Jon you can open an RFE for this.

Regards,

-Amir