Active IQ Unified Manager Discussions
Active IQ Unified Manager Discussions
Hi all,
I've been asked by a large customer to generate a report in OnCommand Core that will report on the performance-relevant aspects of capacity management within an aggregate - in other words, the % free blocks available for writes within an aggregate (ignoring all reservations, guarantees, or the like - just the free space available to the write allocator). The reason to do this is to be able to proactively order additional storage for an aggregate before write performance is impacted.
Has anyone found a way to monitor this value from within OC?
Solved! See The Solution
Hi Jon,
It seems that DFM is not collecting/calculating the value for Total space used/size-volume-used. You can try the report:
#dfm report create -R Aggregate -f "Aggregate.ALL" aggr_report_ALL
This will eventually take all the available attributes collected for aggr object. In this list I could not locate anything equivalent of Total space used/size-volume-used that is recorded in DFM DB.
There are 2 ways that may give the report that you are looking for:
Option#1: Using ZAPIs:
The “aggr-space-list-info” lists the info as(listing only values of interest in this case):
size-volume-allocated = 897802240
size-volume-used = 641814528
size-used = 898043904
size-free = 45674496
size-metadata = 5840896
size-nominal = 943718400
This gives : ====> 1 - (641814528/943718400) = 31.99%
Using new ZExplore Development Interface(ZEDI) shipped with NMSDK 5.0 release you can generate the code for getting this values from ONTAP quite easily:
https://communities.netapp.com/community/interfaces_and_tools/developer/zedi
Option#2: Using PowerShell cmd lets:
This is also pretty quick:
PS C:\Users\amirm> Get-NaAggrSpace |select @{Label="Aggr Name";Expression={$_.AggregateName}},@{Label="Free Space";Expression={[Math]::Round((1-($_.SizeVolumeUsed/$_.SizeNominal))*100,2)}} | format-table -AutoSize
Aggr Name Free Space
--------- ----------
aggr1 9.48
aggr2 6.74
aggr0 32.46
Values reported on controller:
vsim> aggr show_space aggr0
Aggregate 'aggr0'
Total space WAFL reserve Snap reserve Usable space BSR NVLOG A-SIS Smtape
1024000KB 102400KB 0KB 921600KB 0KB 0KB 0KB
Space allocated to volumes in the aggregate
Volume Allocated Used Guarantee
vol0 876760KB 626040KB volume
Aggregate Allocated Used Avail
Total space 876760KB 626040KB 44608KB
Snap reserve 0KB 0KB 0KB
WAFL reserve 102400KB 7612KB 94788KB
===> 1 - (626040KB/921600KB) = 32.07%
Regards,
-Amir
Hi Jont,
Please use the below report to find the space available in aggregate. Please use "dfm report view <report-name>" command to view the report data.
- aggregates-capacity-report (this report have "Aggregate Used Capacity" field)
- aggregates-capacity (you can use "Total" and "Used" filed to get the available space in aggregate)
or
create the custom report using below command with require fields.
dfm report create -R Aggregate -f "Aggregate.Id,Aggregate.FullName,Aggregate.TotalSpace,Aggregate.SpaceAvailable,Aggregate.AvailablePct,Aggregate.Used" aggr_report
[root@shoemake-rhel ~]# dfm report create -R Aggregate -f "Aggregate.Id,Aggregate.FullName,Aggregate.TotalSpace,Aggregate.SpaceAvailable,Aggregate.AvailablePct,Aggregate.Used" aggr_report
Report aggr_report created.
[root@shoemake-rhel ~]# dfm report view aggr_report
Aggregate Id Aggregate Full Name Aggregate Total Space Aggregate Space Available Aggregate Space Available in % Aggregate Used Capacity
------------ ----------------------------------- --------------------- ------------------------- ------------------------------ -----------------------
208 sc41a37-2b:aggr0_sc41a37_2a_2b_02_0 781088496 31933084 4.1 749155412
210 sc41a37-2a:aggr_hamlet1 781088496 773737672 99.1 7350824
Regards,
Gireesh
Hi Gireesh and the Community,
I'm afraid those values don't describe what I'm looking to monitor. What I'm after is effectively the %used you would see for an aggregate whose contents are fully thin provisioned.
If I use your report for one of my customer's aggregates,
U:\>dfm report create -R Aggregate -f "Aggregate.Id,Aggregate.FullName,Aggregate.TotalSpace,Aggregate.SpaceAvailable,Aggregate.AvailablePct,Aggregate.Used" aggr_report
Report aggr_report created.
U:\>dfm report view aggr_report
Aggregate Id Aggregate Full Name Aggregate Total Space Aggregate Space Available Aggregate Space Available in % Aggregate Used Capacity
------------ ----------------------------------- --------------------- ------------------------- ------------------------------ -----------------------
...
1019 FILER:aggr3_SATA 14227331328 355366136 2.5 13871965192
...
I see that this aggregate has about 2% "Space Available in %". This number is, however, irrelevant as far as performance is concerned since it could, for example, be space reserved to an empty volume.
If I look at the same aggregate using aggr show_space,
FILER> aggr show_space aggr3_SATA
Aggregate 'aggr3_SATA'
Total space WAFL reserve Snap reserve Usable space BSR NVLOG A-SIS
16640153600KB 1664015360KB 748806912KB 14227331328KB 0KB 84634020KB
Space allocated to volumes in the aggregate
Volume Allocated Used Guarantee
xxxxxxxxxxxxxxxxxxxxxx 105440140KB 1136960KB volume
xxxxxxxxxxx 1992818688KB 1582846256KB volume
xxxxxxxxxxxxxx 3279188424KB 2477813136KB volume
xxxxxxxx 1381265860KB 1071100556KB volume
xxxxxxxxxxxx 2551651440KB 2036075376KB volume
xxxxxxxxxxxxxxx 105440140KB 1488120KB volume
xxxxxxxxxxxxxxxx 52720068KB 4549660KB volume
xxxxxxxxxxxxxxxxxxxxxxx 52720068KB 5763756KB volume
xxxxxxxx 4212333680KB 2718154172KB volume
xxxxxxxxxxx 52720068KB 13628444KB volume
Aggregate Allocated Used Avail
Total space 13786298576KB 9912556436KB 355362476KB
Snap reserve 748806912KB 4668912KB 744138000KB
WAFL reserve 1664015360KB 167033616KB 1496981744KB
First calculate the physical free space in the aggregate,
Total Space - Physical Used Space (BSR NVLOG + A-SIS + Total space Used + Snap reserve Used + WAFL reserve Used):
16640153600KB - 0KB - 84634020KB - 9912556436KB - 4668912KB - 167033616KB = 6471260616KB physical free space
Then the physical % free space in this aggregate is: physical free space / total physical space,
6471260616KB / 16640153600KB = 39% physical free space (not taking WAFL reserve into account).
Or, taking the WAFL reserve into account as we do in our recommendations on maximum (the way we measure things in our best practices):
(Total space - WAFL reserve - (BSR NVLOG + A-SIS + Total space Used + Snap reserve Used)) / (Total space - WAFL reserve )
(16640153600KB - 1664015360KB - 0KB - 84634020KB - 9912556436KB - 4668912KB) / (16640153600KB - 1664015360KB) = 33% physical free space.
To ask my question more precisely, Is there any way to get OnCommand to report on the physical free space available in an aggregate?
Thanks!
I realize that the above is somewhat pedantic, a simple measure of
Physical Space Free == 1 - (Total Space Used / Usable Space)
would suffice for many cases... (it is 30% for the above example). However, I can't find any equivalent data for either Total Space Used or Usable Space in OnCommand.
Hi Jont,
I think,you are doing wrong calculation. Here, I am going to defined each field and latter I'll calculate space available in aggregate.
Usually, we use "df -A <aggr_name>" and "aggr show_space <aggr_name>" commands to list the aggregate space attributes
df -A <aggr_name> command command will list aggregate capacity, used capacity and available capacity.
Similarly, "aggr show_space <aggr-name>" command display space usage of aggregate in more detail.this command shows the space usage for each flexible volume within an aggregate.
Output of this command display below fields.
Total space: This is total size of aggregate.
Usable space: The capacity available to applications for storing data, this is also call aggregate capacity. you can also check the aggregate capacity space using df -A <aggr-name> command.
WAFL reserve and Snap reserve: WAFL reserve is WAFL metadata and user can not have any control on this.Snap reserve is allocated when user take the snaphot of aggregate.
A-SIS: I am not sure about this field but I think this is space saved in aggregate due to dedupe.
Allocated: The capacity allocated to applications or users for storing data, including effects of thin provisioning. For space guarantee as "volume" (thick provisioning), this field value equal to size of volume and for "none" and "file" space guarantee (thin provisioning), value should be the real data store in volume or volume used space.
Used: This field show application data stored in volume.
"Allocated" and "Used" field's value are with respect to volume.
So, After defining above fields, we can relate the fields as below.
Total space = Usable space + WAFL reserve + Snap reserve
Usable space = Total space - (WAFL reserve + Snap reserve)
14227331328 = 16640153600 - (1664015360 + 748806912)
AND
Aggregate Allocated Used Avail
Total space 13786298576KB 9912556436KB 355362476KB
(Sum of all volume sizes if all (Application data stored on all volumes) (Available space on aggregate)
volumes are thick provisioning)
Free space = Avail/Usable space*100
355362476/14227331328*100 = 2.4977451343993891698309649431396%
If you verify this data to report data then report also showing "Available in %" field's value as "2.5%".
So, you can use same report to check or monitor the free space available in aggregate.
Please revert for further clarification if any
Regards,
Gireesh
Hi Jon,
It seems that DFM is not collecting/calculating the value for Total space used/size-volume-used. You can try the report:
#dfm report create -R Aggregate -f "Aggregate.ALL" aggr_report_ALL
This will eventually take all the available attributes collected for aggr object. In this list I could not locate anything equivalent of Total space used/size-volume-used that is recorded in DFM DB.
There are 2 ways that may give the report that you are looking for:
Option#1: Using ZAPIs:
The “aggr-space-list-info” lists the info as(listing only values of interest in this case):
size-volume-allocated = 897802240
size-volume-used = 641814528
size-used = 898043904
size-free = 45674496
size-metadata = 5840896
size-nominal = 943718400
This gives : ====> 1 - (641814528/943718400) = 31.99%
Using new ZExplore Development Interface(ZEDI) shipped with NMSDK 5.0 release you can generate the code for getting this values from ONTAP quite easily:
https://communities.netapp.com/community/interfaces_and_tools/developer/zedi
Option#2: Using PowerShell cmd lets:
This is also pretty quick:
PS C:\Users\amirm> Get-NaAggrSpace |select @{Label="Aggr Name";Expression={$_.AggregateName}},@{Label="Free Space";Expression={[Math]::Round((1-($_.SizeVolumeUsed/$_.SizeNominal))*100,2)}} | format-table -AutoSize
Aggr Name Free Space
--------- ----------
aggr1 9.48
aggr2 6.74
aggr0 32.46
Values reported on controller:
vsim> aggr show_space aggr0
Aggregate 'aggr0'
Total space WAFL reserve Snap reserve Usable space BSR NVLOG A-SIS Smtape
1024000KB 102400KB 0KB 921600KB 0KB 0KB 0KB
Space allocated to volumes in the aggregate
Volume Allocated Used Guarantee
vol0 876760KB 626040KB volume
Aggregate Allocated Used Avail
Total space 876760KB 626040KB 44608KB
Snap reserve 0KB 0KB 0KB
WAFL reserve 102400KB 7612KB 94788KB
===> 1 - (626040KB/921600KB) = 32.07%
Regards,
-Amir
Hi Amir,
thanks for the pointers to the data, I'll looking into scripting a solution until DFM is able to collect these counters natively. Is there something I can do to help get the development in DFM moving (have the customer open an RFE / case / open a BURT)?
Cheers,
--Jon
Sure Jon you can open an RFE for this.
Regards,
-Amir