Using SDK API's for 'sysstat -x' Consistency Point & Disk Utilization Info Without Math

[pclayton99]

Hello..

I admit to being a fan of the 'sysstat -x' output, especially for the columns of CP Time, CP Type and Overall Disk Utilization. No fuss, no mess, just the facts. And with very low impact to the filer to produce that data!

And I can do that command when logged into a 7-Mode or Cluster just fine.

I have a Powershell script that invokes the sysstat command against the 7Mode filers and then formats the output very nicely into centered columns and mails me the result. The script doesn't need to do lots and lots of calls to get counters, do math, average things out. The script uses the invoke-nassh command and then massages the resulting info into a nice table format that closely matches what we see when done on the command line.

When I try the same sequence against the clusters, the invoke-ncssh command detects the sysstat command and says:

"sysstat" is not supported: use the (privilege: advanced) "statistics { start | stop | show } -preset sysstat" command.

Why can't the CDOT API and filer code just allow this command to execute natively and save everyone, including the filer, a lot of overhead when we know a Powershell script isn't going to be able to do things on 1 second intervals over time so the numbers aren't going to come out right to start with?

I read through the following thread and it dealt with CP counters and not the CP types that can come to be.

http://community.netapp.com/t5/Software-Development-Kit-SDK-and-API-Discussions/CP-info/td-p/192

What are the suggested means to get this high level data, quickly, easily and at an accurate periodic?

Or maybe this needs to be an enhancement request to allow sysstat with the invoke-ncssh command?

My thanks for your time and thoughts.

pdc

[asulliva · Accepted solution]

It's not the PowerShell Toolkit which is giving you the error about "sysstat is not supported", that's clustered Data ONTAP.  You'll want to execute the command on a per-controller basis, not at the cluster level:

Get-NcNode | %{ 
    Invoke-NcSsh "node run -node $($_.Node) -command sysstat -x -c 5 1"
}

Make sure to use the "-c" parameter to limit the result so that the command doesn't hang waiting for a result.  Without the iteration limit it would sit waiting for the ssh command to finish until it times out and throws an error.

If you're looking for the non-CP related counters (e.g. IOPS, latency, etc.) then the best/easiest way is to use the "Invoke-NcSysstat" cmdlet.  PowerShell can do performance statistics calculations, even at 1 second intervals, though it is somewhat more elaborate than simply using the Invoke-NcSysstat cmdlet.  I wrote about how to do it here.

Hope that helps.

Andrew

[pclayton99]

Andrew..

My thanks for the information about the origin of the error message and providing the means to get the sysstat data from the controllers!

I will need to keep the scope of visibility/execution in mind when working Powershell scripts when it is not clearly spelled out in the 'family' column of the API's.

The web page pointer you provided with information on scoping out the various counters and how to get and manipulate the information is a help as well.

Take care.

pdc

[pclayton99]

By way of THANKS for this forum and Andrew's providing the answer to my cluster API problem, I am attaching two files.

The first is the resulting Powershell script that will execute a 'sysstat -x -c7 1' command against a list of 7Mode controllers or cDOT cluster names and email you the result of that data collection with the columns lining up correctly and sometimes having a box around it. I haven't figured out what is goofed up in the resulting HTML code that results in boxes being around every other controller. Not that big a deal as it is easy enough to tell one controller from another. Take a look at the "sysstat_collection_and_reporting.txt" file for the code. If this code is not allowed on the forum, my apologies, please delete at will.

The second is a PDF document of examples about how to format tables/columns with spaces, colors, alignments and such that I have been using to speed other things along. I thought this might prove useful to others that want to code something along these lines with tables/columns.

Take care.

pdc

[asulliva]

That's really great work, and thank you for contributing back to the community!  If you'll PM me your address I'd be happy to send you some NetApp stickers : )

Out of curiousity, when sending the email why not use a monospace font, the HTML pre tag, or send a plain text email?  Then you could simply dump the raw text from the sysstat result into the email and it would show up with correct alignment.  It's not as pretty as putting a border around the each cell in the output table, but is significantly less work overall, especially if you're not manipulating the data.

Also, you can use the ConvertTo-FormattedNumber cmdlet, which ships with the PowerShell Toolkit, to convert bytes to human readable formats easily:

PS C:\Users\Andrew> ConvertTo-FormattedNumber -Type DataSize -Value 1024 -NumberFormatString 0.0
1.0 KB

PS C:\Users\Andrew> ConvertTo-FormattedNumber -Type DataSize -Value 1024 -NumberFormatString 0
1 KB

PS C:\Users\Andrew> ConvertTo-FormattedNumber -Type DataSize -Value 1024 -NumberFormatString 0.00
1.00 KB

PS C:\Users\Andrew> ConvertTo-FormattedNumber -Type DataSize -Value 1024000 -NumberFormatString 0.00
1000.00 KB

PS C:\Users\Andrew> ConvertTo-FormattedNumber -Type DataSize -Value 10240000 -NumberFormatString 0.00
9.77 MB

PS C:\Users\Andrew> ConvertTo-FormattedNumber -Type DataSize -Value 10240000000 -NumberFormatString 0.00
9.54 GB

The cmdlet doesn't show up in the HTML help when you use "Show-NcHelp", but you can still see the help using "Get-Help -Full ConvertTo-FormattedNumber".  It'll also convert other values: raw count, RPMs, and percentages to easier to read versions.

Andrew

[pclayton99]

Well, I can think of several reasons for not taking the easy solution with monospace fonts.

1. Never occurred to me that their use is what made things line up to start with when sending the output to a SSH session for example. Figured there was some magic going on.

2. Wanted to try my hand with tables, HTML and various Powershell coding options to see if I understood what I was reading.

3. Newbie to the whole aspect of NetApp SDK/API calls and Powershell and have been crossing off tasks that we have been doing manually with our NetApps up till now.

Overall, it has been interesting and educational to get different scripts coded up and running the way we need them to. 

My thanks for the pointer to ConvertTo-FormattedNumber cmdlet as well!

I have been using a function call that I found in my travels to do something along that line.  I was searching around for the capabilities of 'Format' and checking for other community code dealing with sysstat and came to find a thread from lobanov at this address. 

Function Format-BigNumber () {
[cmdletbinding()]
Param ([long]$Type)
If ($Type -ge 1TB) {[string]::Format("{0:0.00} TB", $Type / 1TB)}
ElseIf ($Type -ge 1GB) {[string]::Format("{0:0.00} GB", $Type / 1GB)}
ElseIf ($Type -ge 1MB) {[string]::Format("{0:0.00} MB", $Type / 1MB)}
ElseIf ($Type -ge 1KB) {[string]::Format("{0:0.00} KB", $Type / 1KB)}
ElseIf ($Type -gt 0) {[string]::Format("{0:0.00} Bytes", $Type)}
Else {""}
} # End of function

Fun times..

pdc