Help with Storage Allocation custom reporting.

emanuel · ‎2009-07-20

Hello all

I want to create a special report for a customer that they will use on a monthly basis to track volume allocation ( including snap reserves ), grouped by filer and to be subtotalled per filer.

... if my memory serves me right, we dont do subtotalling????????....

however ... i believe we can create a custom report that can list out all the volumes with space allocation and snap reserves and a column that shows the controller name. This would be a simple custom report with a report name that i should be able to access externally by a web address and be able to import it into some other application ... like Excel.

and with some data massaging ... i should be able to group the filers and produce a report that gives me all the allocated volumes, sub totalled per filer, and grand totalled at the bottom.

my question then is ...

1. can OM do this? subtotalling?

2. does anyone here have an example of this or has done anything similar?

A curious thought is that the customer will use this to bill their internal storage users ( chargeback?? ) ... quotas are being used so that functionality is enabled and if #1 can happen, we can turn on chargebacks on this server.

Looking for any and all ideas to help out;.

Thank you

hiyer · ‎2009-07-21

You can create with the following fields:

[root@trinity ~]# dfm report volrep help
Warning: Use of this command for listing and viewing reports has been
deprecated by 'dfm report list' and 'dfm report view' commands respectively.

volrep Report (volrep)
Catalog Name: Volume

Display Tab: Filesystems

Catalog Field                                 Field Name                                    Format
--------------------------------------------- --------------------------------------------- ---------
Volume.FullName                               Volume Full Name
Volume.ReservedSpaceTotal                     Volume Reserved Space Total                   Auto-Scaled
Volume.ReservedSpaceUsed                      Volume Reserved Space Used                    Auto-Scaled
Volume.Used                                   Volume Used Capacity                          Auto-Scaled
Volume.Total                                  Volume Total Capacity                         Auto-Scaled
Appliance.FullName                            Appliance Full Name

Default sort order is Volume.FullName.

This report is created using the syntax:

dfm report create -R volume -f "fullname, reservedspacetotal,reservespaceused,used,total,aggregate.filer.fullname" volrep

You are correct in that DFM does not do any sub-totalling. However, as you mentioned, it should be possible to use the data from this report to group the data on a per-filer basis. An example of this, using awk, is presented below:

[root@trinity scripts]# dfm report view -q -F csv volrep | awk -F, '{total[$NF] = total[$NF]+$5; used[$NF] = used[$NF]+$4;} END {for (x in total) {printf("%s total %.1f used %.1f\n", x, total[x], used[x])}}'
total 2.6 used 448.0
chinki.lab.eng.btc.netapp.in total 2678.3 used 13199.9
asthana.lab.eng.btc.netapp.in total 89.6 used 6.9
f3050-181-246.lab.eng.btc.netapp.in total 1210.8 used 471.5
f3050-181-245.lab.eng.btc.netapp.in total 1314.2 used 1430.8
sohail.lab.eng.btc.netapp.in total 1435.3 used 8849.1
beckham.lab.eng.btc.netapp.in total 935.8 used 5805.2
alexander.foster.lab.eng.btc.netapp.in total 1562.0 used 3622.7
waqar.lab.eng.btc.netapp.in total 15.0 used 10.3
shortcircuit.lab.eng.btc.netapp.in total 4025.4 used 6968.5
maamu.lab.eng.btc.netapp.in total 40.3 used 250.5
miandad.lab.eng.btc.netapp.in total 5756.3 used 7821.6
shoaib.lab.eng.btc.netapp.in total 234.7 used 5055.9
ijaz.lab.eng.btc.netapp.in total 1109.2 used 2952.0
zidane.lab.eng.btc.netapp.in total 43.1 used 593.0
jameel.lab.eng.btc.netapp.in total 6955.3 used 10855.2
dawood.lab.eng.btc.netapp.in total 3464.0 used 20108.2
wasim.smad.lab.eng.btc.netapp.in total 3213.9 used 21782.4

kjag · ‎2009-07-22

Hi,

If you are looking for a report like the attached one, we are having a java script which generates the customized report in html and xls format. We need to modify the script little bit to adapt your requirements.

Thanks,

Kjag

emanuel · ‎2009-07-22

Hello .. the worksheet came out blank ... missing a file on the download. You can email to me directly emanuel@netapp.com

Here is what I have done ... I created a custom report that will list .... filer name, volume name, and total space. pretty simple. But in OM i cannot create a subtotal field per filer. check out my crude example here. I can get some of this to import into Excel but then i would need to work on my Excel Magic to create my example below.

What do you guys think?

OM produces this...........

FILER1 VOL0 200MB

FILER1 VOL1 500MB

FILER1 VOL2 1000MB

FILER2 VOL0 150MB

FILER2 VOL33 5000MB

FILER3 VOL0 210MB

TOTAL Space 7060MB

What I am hoping to see is....

FILER1 VOL0 200MB

FILER1 VOL1 500MB

FILER1 VOL2 1000MB

Filer 1 SubTotal 1700MB

FILER2 VOL0 150MB

FILER2 VOL33 5000MB

Filer 2 SubTotal 5150 MB

FILER3 VOL0 210MB

Filer 3 SubTotal 210MB

TOTAL Space 7060MB

and even more cool would be....

FILER1 VOL0 200MB

VOL1 500MB

VOL2 1000MB

Filer 1 SubTotal 1700MB

FILER2 VOL0 150MB

VOL33 5000MB

Filer 2 SubTotal 5150 MB

FILER3 VOL0 210MB

Filer 3 SubTotal 210MB

TOTAL Space 7060MB

rshiva · ‎2009-07-23

Hi,

Have you tried considering the creation of subgroups? That should solve your problem.

Re-iterating your example:

Group_A

=======

FILER1 VOL0 200MB

FILER1 VOL1 500MB

FILER1 VOL2 1000MB

Filer 1 SubTotal 1700MB

Group_B

=======

FILER2 VOL0 150MB

FILER2 VOL33 5000MB

Filer 2 SubTotal 5150 MB

Group_C

=======

FILER3 VOL0 210MB

Filer 3 SubTotal 210MB

Global

======

FILER1 VOL0 200MB

FILER1 VOL1 500MB

FILER1 VOL2 1000MB

FILER2 VOL0 150MB

FILER2 VOL33 5000MB

FILER3 VOL0 210MB

TOTAL Space 7060MB

With respect to your second example:

FILER1          VOL0          200MB
                     VOL1          500MB
                     VOL2          1000MB
               Filer 1 SubTotal 1700MB

FILER2          VOL0          150MB
                     VOL33         5000MB
               Filer 2 SubTotal 5150 MB

FILER3          VOL0          210MB
               Filer 3 SubTotal 210MB

These reports are available in Operations Manager. Get into the Appliance-Details page of a Filer a look out for a section called Storage capacity. Under this section you'll be able to see total count of volumes on that filer. Just click on "number” and you can see all kinds of volume based reports pertaining to that filer.

Thanks and regards
Shiva Raja

fenton · ‎2009-07-23

I'd also add to Shiva's response that if you use the Chargeback reports (either by Storage System or by Group) the data provided will be based on the average (allocated or used) for a given month (this month or last month);

Cheers

Rich

emanuel · ‎2009-07-23

I will look at these today ...

chargebacks are based on quotas being activiated ( you can probibly get away with just dashes in the /etc/quotass ) and a basic configuration in OM ; dollars, etc. Is this correct? I will look at docs later today.