How to understand performance

[bader]

Hi,

My storage admin duties are growing and I need to understand performance and how to trouble shoot performance issues.  I found  tr-3525 but it is a little dated, and does not cover ops manager 4.0.   I keep asking this question and people say it is an art and a science.  Does anyone have any good resoruces for how to learn more on understanding and troublshooting performance?

I do have operations manager 4.0 and with NetApp Management console I get all the info I need, I just need to learn how to use.

Thanks

Bob

[adaikkap]

Did you take the NetApp university courses? For DFM 4.0 and its suite of products?

Regards

adai

[bader]

Adai,

Have you seen a specific course?  I looked at university  and I found
Operations Manager 3.7, Protection Manager 3.7 and Provisioning Manager 3.7 AdMinistration  --- which is not for version 4 and also not being offered.

I also did the web based Getting Started with NetApp Software: Manage Efficient Storage Growth --- but it did not cover performance.

Bob

[rbalaji]

Hi Bader,

We have created a new TR-3836 (INTERNAL ONLY) on "Performance Diagnosis using Performance Advisor". This document provides best practices for using Performance Diagnosis  feature available through Operations Manager 4.0. It is intended for  Storage Administrators/NGS who want to troubleshoot the storage system  for performance related issues.

Currently this TR is available from fieldportal  (https://fieldportal.netapp.com/default.asp).

Regards

Balaji Katika

[amiller_1]

Would it be considered all right to pass along to sufficiently technical customers and/or customers under NDA?

I really like the ability in Performance Advisor now (given OM 4.0) to track aggregate disk busy-ness from a historical perspective.

[harish]

What counters do you use to track aggregate disk busy-ness ?

[lwei]

You can use statit, disk utilization.   -Wei

[amiller_1]

In the Performance Advisor section of NetApp Management Console 3.0 (come with Operations Manager 4.0), I specifically like to make a custom historical graph tacking aggregate:pa_avg_disk_busy, aggregate:pa_max_disk_busy, aggregate:pa_min_disk_busy.

I'm also experimenting with some alerts around aggregate:pa_mix_disk_busy if stays above 90% for more than 10 minutes (for example).

See attached screenshot....

[harish]

aggregate:pa_*_disk_busy counters were added in DFM 4.0 along with

system:pa_concurrency that gives the queue depth. Please let us know if

you face any issues accessing data for these counters in views,

thresholds, metrics, etc.

Regards

Harish

[anantha_dommeti]

pa_concurrency that gives the queue depth ? What does the queue depth over here represent ?

I see pa_concurrency around 1300, when we see latencies during writes.

[harish]

Queue depth of 1300 means there are 1300 operations being processed

inside the filer. Note that the number itself does not represent

anything (good/bad), but the variations in the number might be useful...

If the number increases, it means that lot more operations are now

inside the filer that might impact latency.

Queue depth = ops (ops/sec) * latency (milli or micro second) and hence

has no units. The formula used by performance advisor is as follows:

system:pa_concurrency = nfsv3_avg_op_latency * nfsv3_ops +

nfsv4_avg_latency *

nfsv4_ops +

cifs_latency * cifs_ops +

iscsi_read_latency *

iscsi_read_ops +

iscsi_write_latency *

iscsi_write_ops +

fcp_read_latency *

fcp_read_ops +

fcp_write_latency *

fcp_write_ops

Regards

Harish

[reide]

Harish,

Any idea why pa_concurrency is not enabled in OnCommand 5.0?  All of the counters used in its calculation are being collected for my controllers.  But this particular counter is always in italics and its description states, "...(Data collection currently disabled.)"

I've seen other users stating the same issue in other threads.

[jeff_spiegel]

Does system:pa_concurrency exist in 5.0?  My charts are blank now that we upgraded.  I was looking at queue depth just fine before we upgraded.  Thanks.

[rbalaji]

Do you mean this TR can be shared with external customers ?

Answer is yes. But any modifications to the diagnosis configuration (including adding new rules) are recommended to be done under the guidance/supervision of NGS.

[amiller_1]

Hmm....searching on "Performance Advisor" or "tr-3836" in Field Portal doesn't turn up anything. Is this available to partners?

[netappsanadmin]

if we can't get the performance advisor(for internal only document)

can we have other document which is related to performance advisor.

Thanks

.....