Quota notification email

[igor]

Is it possible for a filer to send out an e-mail directly to the user, when they reach soft/hard quota limit? A script, perhaps?

Examples and guides would be most welcome!

Thanks,

Igor

[msaravan]

Hi lgor,

I am not sure about scripts or filer.  But we have a product called Operations Manager (part of DFM suite) that can do more than what you expect.

The below are the events Operations Manager can generate right today after monitoring your storage system's quota setup:

user-disk-space-quota-almost-full

user-disk-space-quota-full 

user-disk-space-quota-ok   

user-disk-space-soft-limit-exceeded 

user-disk-space-soft-limit-not-exceeded

user-files-quota-almost-full   

user-files-quota-full    

user-files-quota-utilization-normal  

user-files-soft-limit-exceeded       

user-files-soft-limit-not-exceeded     

volume-quota-almost-overcommitted

volume-quota-not-overcommitted        

volume-quota-overcommitted

Regards,

Saravanan

[igor]

Helo Saravanan,

Yes, I'm looking at DFM right now. It looks a bit complicated to navigate and configure. Under Setup menu, parameters regarding quota notifications can be found in Default Thresholds, Events And Alerts and Users. And you can also choose a specific qtree, and click on Edit Quota Settings and work there. This discussion here also mentioned having to create notification groups and it seems it's not very easy to grasp...

Other than this guide http://now.netapp.com/NOW/knowledge/docs/DFM_win/rel371/pdfs/opsmgr.pdf is there any other document describing step-by-step setup for user quota alerts?

P.S. Just a thought... since Operations Manager is a part of Base Pack software pack, this may work for users which have purchased NetApp recently. Old users may not have that option.

[txskibum2000]

Need to create an email account for the user in DFM.  domainname\username email address is blahblah@domain.com  This is in quota user settings.  I can send a screenshot if you like.

I would like to change the default email address from the sender of the email notification.  Right now, emails being sent to users come from "IT Administrator".

Dale

[igor]

A screenshot would be helpful, thanks!

Igor

[rmharwood]

I didn't read the linked articles but we're doing quota notifications with DFM. See the "dfm quota mailformat" command - it allows you to specify an email template file, and in that file you can specify your own mail headers. For example, at the top of the file I have:

     From: <abc@xyz.com>

     Subject: Your personal drive is nearing its quota limit

     email content.....

You place the desired sender's email address in the From: field.

[txskibum2000]

Thanks rmarwood..  I will try that.

igor, see attachment.

[msaravan]

Hi Igor,

I have given step-by-step procedure below which you can use to configure quota alerts for individual users.

1=> Set your email server @ Setup -> Options -> Events and Alerts

2=> Look at the default quota thresholds @ Setup -> Options -> Default Thresholds

3=> Set your e-mail domain-name @ Setup -> Options -> Users. You can leave this field empty if you specify complete e-mail address at User settings page @ step7.

4=> Check that you set the credentials for the storage systems (which have user quotas) being monitored in DFM using dfm host set <storage system> hostlogin=xxxx hostpassword=xxxx. This is needed for the discovery of User quotas. Once the credentials are set, re-discover the storage system using 'dfm host discover <storage_system>'. This is required to keep DFM to have up-to-date information with respect to storage system.

5=> Wait for few mins (till userquotas mon completes) and launch "User Quotas All" Report (Go to Control Center -> Home -> Group Status -> File SRM/Quotas -> Report -> User Quotas, All)

6=> "User Quotal, All" page will show you all the usernames for whom quotas are configured.

7=> Click on one of the Usernames which will launch "User Details" page. Click "Edit Settings" link available in the bottom panel "QuotaUser Tools". It will launch "Edit User Settings" page where you can specify the e-mail address for the user.

8=> You need to repeat the above step for all the users. If you are not comfortable with UI, you can do this using CLI.

dfm quota user list
dfm quota user set <username> userEmail=abc@xyz.com

A simple script might help you here.

Hope these steps help you to fix your problem.

Regards,

Saravanan

[igor]

Thanks for the reply Saranavan,

I'll give it a try today!

One thing though... while testing quotas, I noticed that FilerView immeditelly registers when some exceeds their quota. However, DFM doesn't... it's only after I manually issue "dfm host discover <storage_system>" command that it registeres the change, and even than it'll be after a few minutes.

Is there a way to have DFM keep up with the changes in real-time, automatically?

Thanks,

Igor

[msaravan]

Hi Igor,

Filerview is something like you are on the box directly. So, whatever changes you do on CLI (or via some operations), that will be reflected immediately in Filerview.

But, DFM is a Storage Management product. It is designed in a way that the userquotas are monitored only once in 24 hours (1 day). i.e 24 hrs is a default timer set for userquota mon.

Again, this option is configurable using "dfm options set".  If you reduce this value, it means that DFM will do more polling which in turn will consume few more CPU cycles of your storage system.

[root@10 log]# dfm options list | grep -i userQuotaMonInterval
userQuotaMonInterval                  1 day
[root@10 log]#

Regards,

Saravanan

[msaravan]

On your other question :

>> Is there a way to have DFM keep up with the changes in real-time, automatically?

Please file a RFE for this.  We can make this possible in DFM using SNMP Traps feature. The flow should be :

1. Pre-Requisite : On your storage system, you need to configure DFM server as snmp traphosts
(snmp traphost add <DFM_Server_IP>)

2. When DFM server receives traps (quotaExceeded, quotaNormal) from these storage systems, we can have DFM to trigger userquota monitor which will fetch you the latest data. DFM would send mails to the users when their  quotas are exceeded.

Regards,

Saravanan

[rmharwood]

I definitely agree with this approach.

[igor]

Hello Saravanan,

There's something a miss with e-mail parameters... Here's your checklist from before:

1=> Set your email server @ Setup -> Options -> Events and  Alerts  - DONE.

2=>  Look at the default quota thresholds @ Setup -> Options ->  Default Thresholds - DONE.

3=> Set your e-mail domain-name @ Setup -> Options  -> Users. You can leave this field empty if you specify complete  e-mail address at User settings page @ step7.  - DEFINED UNDER STEP 7.

4=> Check that you set the  credentials for the storage systems (which have user quotas) being  monitored in DFM using dfm host set <storage system>  hostlogin=xxxx hostpassword=xxxx. This is needed for the discovery of  User quotas. Once the credentials are set, re-discover the storage  system using 'dfm host discover <storage_system>'. This is  required to keep DFM to have up-to-date information with respect to  storage system. - DONE.

5=> Wait for few mins (till userquotas mon completes)  and launch "User Quotas All" Report (Go to Control Center -> Home  -> Group Status -> File SRM/Quotas -> Report -> User Quotas,  All)

6=>  "User Quotal, All" page will show you all the usernames for whom quotas  are configured. - IT'S ALL THERE.

7=> Click on one of the Usernames which will launch  "User Details" page. Click "Edit Settings" link available in the bottom  panel "QuotaUser Tools". It will launch "Edit User Settings" page where  you can specify the e-mail address for the user. - DONE.

8=> You  need to repeat the above step for all the users. If you are not  comfortable with UI, you can do this using CLI. - OK.

Also, under Setup -> Options -> Monitoring, I've set User Quota Monitoring Interval at 10 seconds to see how it goes. Having done that, I filled up test user's home folder to the limit. FilerView registered it, some 30 seconds later DFM also registered that event, but no e-mails were sent out.

I've decided to test the e-mailing ability by setting up a random alarm for some critical event @ Setup ->Alarms. When I clicked on Test, an e-mail was sent out to test user detailing that it's an Alarm Test. So, mail settings are properly configured.

I than added another alarm, this time specifically for "User Disk Space Quota Almost Full" event and added both myself and the test user as e-mail recipients. Surely enough, 20 seconds after I filled up the home folder for the second time, we received an email each! This means that the quota report can successfully trigger alarms and follow their e-mail settings, but "User Disk Space Quota Almost Full" is a global event... any user can trigger it... I need DFM to contact specific users in  regards to their specific quotas.

Is there something else I need to configure for User Settings to work?

Thanks,

Igor

[igor]

Since DFM uses SMTP for communicating with storage, could this be a SMTP issue?

[msaravan]

No, DFM is not using SMTP for storage system communication.

DFM uses SNMP and HTTP (NetApp proprietary version) protocols for 95% of its opeartions.

For UserQuotas, DFM is using HTTP (NetApp proprietary version) protocol.

Regards,

Saravanan

[igor]

Sorry Saravanan,

I meant to say SNMP. 🙂

So, if I wanted to have a more frequent DFM update for UserQuotas I'd have to set userQuotaMonInterval to a lower value (e.g 1 minute, 30 secs), and that would be updated through HTTP? Is that enough to trigger a quota alarm message to be sent to a user's e-mail address?

I ask this because I think we're getting close to what I need -- to have a local user (with a quota on his home folder) receive an automated e-mail when they exceed soft/almost full/hard limits. Doesn't have to be reflected immediatelly, like on the box, but if I can get it to work within 1 minute or even 30 secs it would be great!

>>Again, this option is configurable using "dfm options set".  If you  reduce this value, it means that DFM will do more polling which in turn  will consume few more CPU cycles of your storage system.

Since userQuotaMonInterval works through HTTP, I suppose this woldn't be a significant increase in CPU load?

Thanks,

Igor

[rmharwood]

Here is my recommendation. Do not use the built-in quota notification system in DFM to generate near real-time alerts of users getting near to or going over quota. It is designed to run on a periodic basis because it asks your storage controllers for a quota report. This generates a lot of work for your DFM server and your storage controller, depending upon the complexity and scale of your systems. You MAY get away with bringing your quota monitoring interval down to an hour or so but I would still be wary of what additional load this generates.

As others have said you really should look into SNMP traps. Read the section in the DFM admin guide about these. It describes the pros and cons of changing monitoring intervals and suggests that SNMP traps - event notifications from the storage controller to an SNMP trap listender regarding a specific event - are better for near or real time monitoring. It's the difference between having your kids ask you every minute "are we there yet" or you telling them when you arrive.

I have not looked into the details but I'm quite sure the storage controller will send an SNMP trap when a user quota limit is hit. You (or someone) will have to examine the Netapp SNMP MIB to see what the specific event is so that you can watch for it. What you can do then is to configure an alarm in DFM that triggers when the trap is received. You then have the alarm call a script that sends email to the user as required. This approach isn't simple: it requires knowledge of SNMP and scripting.

As someone else pointed out, it would be nice if DFM could natively handle SNMP quota messages and send alerts to users in the same way that quota monitoring already works. I haven't checked - do Netapp have a mechanism for submitting RFEs?

Cheers,

Richard

[igor]

Hello Richard,

The systems are two 2020A models set in different parts of the building, cross-replicating data for DR purposes. User documents, various types of mailboxes, archive, etc. Purely NAS device so far. As for quota reports, I wasn't aware they're so demanding in terms of resources. I thought it farily was a simple process/query?

Yes, I read about SNMP listener in DFM. Haven't tried it yet, but if all SNMP traps look like this, they're not saying much other than something happened... Am I mistaking? Do SNMP traps carry more information that this?

If my notion of SNMP traps is true - and following your paradigm - it would be like receiving a "there was arrival" message. Doesn't say who arrived or where. Perhaps a SNMP trap informing DFM that a quota event happened, follow by an fresh quota monitoring query from DFM would be most effective.

On the other hand, what I'm attempting to pull off is a pretty handy thing. It would be nice if this could be incorporated in FilerView, under Quotas, and save us all a lot of network traffic.

Cheers,

Igor

[rmharwood]

Some SNMP traps carry more information than others, I couldn't say one way or the other if the quota traps contain enough information. Before we had DFM we were intending on using syslog to trap quota messages, until we realized that the syslog message provides a Windows SID rather than a username and no qtree name.

You can always try to decrease the monitoring interval for quota checks and see what happens. Every environment is different so it may be that your systems can tolerate such a load without adverselt affecting anything. Note that DFM also maintains its own database of user quota information so that information has to be updated also each time a check is run.

I doubt very much that the storage controller itself will ever have the functionality to do what you want. The more that you give it to do that isn't directly related to serving NAS/SAN stuff the less effective it is going to be at those tasks.

Cheers,

Richard

[igor]

Well, I've decreased the monitoring interval but I still can't get DFM to send out an email to the test user.

- Under Volume Tools I opened the volume's Quota Settings, and set Enable User Quota Alerts to yes.

- I did the same under Qtree Tools: Enable User Quota Alerts... yes.

- The User Settings are already configured

- DFM gets fresh quota reports in regular intervals...

... but unless I set a general Alarm for an event, no e-mails are sent out!

DFM simply doesn't work! 😞

Cheers,

Igor

[msaravan]

Hi Igor,

I just tested this in my lab setup and I saw that it is working and I got e-mails too. My setup has both DFM Server and Storage system part of same NIS and DNS domains. But, I dont know whether that really matters. This is one of the legacy features of DFM Server and it should work. 

Since this doesnt work for you, can you file a burt on DFM server product. We need to understand your environment better and some one from Development team can look into your issue.

Regards,

Saravanan