Brainfreeze: Volume reporting with OnTap Powershell Toolkit

[piraconsk]

Hi Techexperts,

i have been tasked by my customer with creating a volume based report for him, that contains

- Name of the volume

- Total Volume Size

- Used Data Space

- Free Data Space

- Used Snapshot Space

- Free Snapshot Space (regarding the reserve)

- Overflow Snapshot Space (if existing)

Getting the volume's data confuses me, because the GUI is showing values that is not reflecting by Get-NcVol, Get-NcEfficiency, Get-NcSnapshot... somehow.

We are running Ontap 8.3.1, but when viewing bar char on the volume's storage allocation tab, it seems the values are not assigned correctly to the lines.

I was using the Ontap Powershell Tookit to retrieve the values but i have some difficulties accumulating the snapshot spaces correctly

Get-NcEfficiency | sort-object Name | Select-Object Name, TotalLunSize, Used, Free, @{Label="SnapshotUsed";Expression={(Get-NcSnapshot $($_.Name) | Measure-Object Total -Sum).Sum }}, SnapshotReserve, SnapshotOverflow | Export-Csv -NoTypeInformation -Path "C:\temp\volumereport.csv"

Please feel free to correct me, but i was assuming that SnapshotReserve + SnapshotUsed + SnapshotOverflow = SnapshotTotal and that SnapshotReserve - SnapshotUsed = SnapshotFree

So when I run the command, the result for the volume shown is:

Name             : D5
TotalLunSize     : 0
Used             : 660389388288
Free             : 206657134592
SnapshotUsed     : 278830014464
SnapshotReserve  : 45634027520
SnapshotOverflow : 227582599168

or with some more readable values:

Name             : D5
TotalLunSize     : 0
Used             : 1803,59
Free             : 238,91
SnapshotUsed     : 293,90
SnapshotReserve  : 107,50
SnapshotOverflow : 61,32

As you can see, there are huge differences between what is shown in GUI and what is reported by the command.

And to make this problem more complex: for some volumes (mostly those without overflow space) there seem to be the same values displayed in GUI and in the result of the command.

So my questions are: which values can I trust and how do fetch the (correct) required values for the customers report

[asulliva]

Hello @piraconsk,

I'm seeing equivalent values using this bit of script for display:

Get-NcEfficiency -Volume $volName | 
    Sort-Object Name | 
    Select-Object Name, `
        TotalLunSize, `
        @{ 'N'="Used"; "E"={ ConvertTo-FormattedNumber -Type DataSize -NumberFormatString "0.00" -Value $_.Used }}, `
        @{ 'N'="Free"; "E"={ ConvertTo-FormattedNumber -Type DataSize -NumberFormatString "0.00" -Value $_.Free }}, `
        @{ Label="SnapshotUsed"; Expression={ 
            ConvertTo-FormattedNumber -Type DataSize -NumberFormatString "0.00" -Value (Get-NcSnapshot -Volume $($_.Name) | Sort-Object -Property Created -Descending | Select-Object -First 1).CumulativeTotal 
        } }, `
        @{ 'N'="SnapshotReserve"; "E"={ ConvertTo-FormattedNumber -Type DataSize -NumberFormatString "0.00" -Value $_.SnapshotReserve }}, `
        @{ 'N'="SnapshotOverflow"; "E"={ ConvertTo-FormattedNumber -Type DataSize -NumberFormatString "0.00" -Value $_.SnapshotOverflow }}

Note that I changed a few things.  First, and most verbosely, I'm using the ConvertTo-FormattedNumber cmdlet to display back the data size values.  Second, I modified the snapshot used field to use the "cumulative total" property from the old snapshot on the volume rather than summing the size of the snapshots.

Also note that the "Used" value in the PowerShell output is going to be the size of the data + the size of the snapshot overflow, whereas System Manager reports them separately.

Andrew

[markweber · Accepted solution]

if i'm reading what you wrote correctly:

snapshotused = snapshottotal

SnapshotReserve - SnapshotUsed = SnapshotFree if snapshotused -lt snapshotreserve

i'm curious why you are doing the get-ncsnapshot and math to get the snapshot used space since get-ncefficiency returns snapshotused.

i ran your command on a single volume and it seemed to return the correct values for me.

personally, i would avoid get-ncefficiency and use get-ncvol since get-ncefficiency is a heavy and slow command.

something like:

$vols = get-ncvol | select name,@{n="totalsize";e={$_.VolumeSpaceAttributes.Size/1gb}},
@{n="free";e={$_.VolumeSpaceAttributes.SizeAvailable/1gb}},
@{n="data size";e={$_.VolumeSpaceAttributes.SizeTotal/1gb}},
@{n="data used";e={$_.VolumeSpaceAttributes.SizeUsed/1gb}},
@{n="snap size";e={$_.VolumeSpaceAttributes.SizeUsedBySnapshots/1gb}},
@{n="snap reserve";e={$_.VolumeSpaceAttributes.SnapshotReserveSize/1gb}},
@{n="snap overflow";e={if($_.VolumeSpaceAttributes.SizeUsedBySnapshots -gt $_.VolumeSpaceAttributes.SnapshotReserveSize){($_.VolumeSpaceAttributes.SizeUsedBySnapshots - $_.VolumeSpaceAttributes.SnapshotReserveSize)/1gb}else{0}}}

this is all in GB and the formatting is - well, there isn't any

hth

[piraconsk]

@markweber

"i'm curious why you are doing the get-ncsnapshot and math to get the snapshot used space since get-ncefficiency returns snapshotused."

Simple as that: the return values for snapshotused from Get-NcEfficiency are not equal to (Get-NcSnapshot D5 | Measure-Object Total -Sum).Sum.

[markweber]

interesting - they are for me:

PS E:\> (Get-NcEfficiency storage).SnapshotUsed
1083846656
PS E:\> (Get-NcSnapshot storage | Measure-Object Total -Sum).Sum
1083846656

are they the same for you when there isn't any snapshot overflow?

[piraconsk]

@markweber

Indeed, very interesting they are equal on your device. And yes, i have noticed the performance of get-ncefficiency is not as high as with get-ncvol.

On my device the result looks like this:

(Get-NcEfficiency D1).SnapshotUsed: 5618896896
(Get-NcSnapshot D1 | Measure-Object Total -Sum).Sum: 9651384320

I have noticed that there are a few "old" snapshots from a previous configuration existing. I am unable to see if, there is a baseline snapshot to this subset. Maybe the baseline has been deleted, so the SnapshotUsed differs from Sum?

I have run your code sample vs my environment. For my example volume the result are matching the most.

Result from your fast query:
Name          : D5
totalsize     : 850
free          : 192,476535797119140625
data size     : 807,5
data used     : 615,023464202880859375
snap size     : 260,0552215576171875
snap reserve  : 42,5
snap overflow : 217,5552215576171875

Result from my slow query:
Name             : D5
TotalSize        : 850
Data             : 807,5
Used             : 615,048320770263671875
Free             : 192,451679229736328125
SnapshotUsed     : 260,074733734131
SnapshotReserve  : 42,5
SnapshotOverflow : 211,876407623291015625

The results are mostly equal, a slight difference in overflow space, which is negligible imho.

I think i will stick with your Get-NcVol approach.

About the graphical display, I'm assuming, that these values are sometimes sorted wrong due to a bug in the release?

[RavikumarVR]

I need Powershell command which can pool datastore reached 80% from 7mode & C-mode.it sholud also send an email.

any one please give suggestion