Is there a writeup on how Provisioning/Protection Manager selects the "best" resource within a resource pool?

[hill]

I've been looking for a writeup that documents how Provisioning Manager / Protection Manager selects the "best" resource  (filer / aggregate) to use within a resource pool. 

Example:  The customer would like to provision new storage via Provisioning Manager.  They've set up their Provisioning Policies, and their Dataset.  They're selecting to Provision via the GUI, using the following Resource pool makeup:

• Primary_pool:
  • Filer1
  • Filer2

Controllers: Filer1 & Filer2 have the following makeup:

• Filer1
  • aggr1 - 50% full
  • aggr2 - 20% full
  • aggr3 - 70% full
• Filer2
  • aggr1 - 45% full
  • aggr2 - 80% full
  • aggr3 - 10% full

Assume that all the aggregates are the same configuration and the same overall size.

When the customer selects to provision another "chunk" of storage (amounts to 10%) aggr capacity.... which Aggregate within the resource pool will be selected and why?

[adaikkap]

Hi hill,

     Generally, the least used aggr is taken(which has more free space), also space balancing is done when repetitive provisioning is done.

Starting DFM 3.8 we do have manual resource selection for Primary provisioning.

There are also controls given to user to control/dictate how to select the best aggr within a respool by the following ways.

[root@lnx/]# dfpm reslimit set
Valid options are
  maxFlexVols               Maximum number of FlexVols per storage controller
  maxCPUUtilThreshold       Maximum CPU utilization threshold of storage controller
  maxDiskUtilThreshold      Maximum Disk utilization threshold of an aggregate
  maxDedupeSizeInGB         Maximum Deduplication size of a storage system model and ONTAP version (in GB)
[root@lnx /]#

Regards

adai

[sharaf]

Hi Hill

In addition to what Adai has mentioned, performance metrics are also considered.

For performance metrics to be considered Performance Advisor needs to be enabled on DataFabric Manager server and at least 2 weeks data should be collected for the storage system or aggregate. Provisioning Manager looks at CPU busy and aggregate disk IO when selecting a resource for provisioning from the resource pool.

Provisioning Manager has an internal algorithm to select the right storage system and aggregate in the resource pool when serving a provisioning request. But in case there is a requirement to provision all the volumes of a data set on specific storage systems or aggregates in a resource pool, then administrators can make use of a mechanism called "labels".

For example administrators can set a label called "low-cost" on all aggregates with SATA disks and specify the same in the provisioning policy. Now, when provisioning volumes using this provisioning policy in a data set, only those resources marked with label "low-cost" in the resource pool are considered.

You can also refer TR at the following link http://media.netapp.com/documents/tr-3729.pdf

Regards

Sharaf

[hill]

Sharaf & Adai,

Thank you for the information, however, this only partially answers my question.  Sharaf, pulling from your response you indicated: " Provisioning Manager has an internal algorithm to select the right storage system and aggregate in the resource pool when serving a provisioning request. "  Perfect.  I am asking if THAT is documented.  I do understand the examples both you and Adai provided, but that was not my concern.

I am asking if there is any detailedinformation on the internal algorithm Provisioning Manager selects the "right" storage system and aggregate.  There is a pretty good summary within the "Simplified SAN Provisioning and Improved Space Utilzation Using NetApp Provisioning Manager" document (also known as TR-3729) you identified.  The summary I found under a Storage Provisioning section in that document is as follows for anyone following this thread:

Following are some of the checks done by Provisioning Manager as a part of its resource selection algorithm during provisioning:

•

Current health of resource (whether storage system is down or it is heavily loaded, etc.). If a storage system is heavily loaded, then Provisioning Manager tries to do proper load balancing by looking for another suitable resource in resource pool. Provisioning Manager currently takes into account CPU and disk utilization of storage system.

•

Storage resiliency parameters like RAID-DP, RAID 4, SyncMirror, active/active pair

•

Data ONTAP versions

•

Space and aggregate overcommit thresholds

•

Required licenses

•

Protocol service status

The storage administrator need not be bothered about the resource selection once the admin has properly configured the resource pool. After all the available resources in the resource pool have been tried and no suitable resource exists, only then Provisioning Manager will fail the provisioning request with appropriate error messages and suggestions. Based on the suggestions, the storage administrator can take required remedial actions and retry the provisioning request.

While I'm happy to see something, I don't consider it suffient to have the only place this is document is within a TR for "Simplified SAN Provisioning".  The summary may be in other similar documents (say for NAS).  Where this can be a concern is when customers ask questions like:

"The internal algorithm will choose the appropriate resource pool based on current health of a resource.  Can you provide a few examples of how Provisioning Manager will make this decision? And which criteria takes precedence?"

According to some feedback I've received outside this thread it seems there is an on-going effort for writing up such a document. In lieu of that I believe I have enough information to use currently and look forward to when the internal algorithm for selection of the right resource within Provisioning Manager is documented.

Thank you,

Kevin

[adaikkap]

Hi Kevin,    

    The reason i havent disclosed the internal  algorithms is because most of these are  patentalbe or patented.So i dint want to discuss this.

Can you move this thread to other closed community of StorageManagement Software ?

Where we can take up this ?

Regards

adai

[hill]

Hi Adai,

     We can move this to a different community if there is concern that too much information might be shared.  That said, when the field facing folks and customers have questions on "how does Provisioning Manager select the best resource", it would be beneficial to have something to point to to give at least an overview.  That something being a document (TR or KB), or at least a section with in the Provisioning Manager and Protection Manager Administration Guide.  No proprietary or patented or soon to be patented information need be shared.

Thanks,

Kevin

[adaikkap]

Hi Kevin,

     That's true we will soon update you on what is the approach(TR/KB/Admin guide) we are taking to give an high level info on the same.

BTW can you create a similar thread in the other community ?

Regards

adai

[kramar]

What was decided on where to document this type of detail?  I still can't find it:-(

[adaikkap]

Sadly Its still not out......., and I missed the thread.......