telder1958,
This really goes back to the original question and my initial response. "It depends" is a common response to this question. In reality, the it depends bit is defined by application tolerance and user tolerance. For example, Microsoft Exchange and the JetStress application expect response times to never exceed 20ms to be considered "good".
The "it depends" also has a factor of operation size embedded in it. If I were to say and alert that 5ms was "bad", does that really have any value without also understanding the overall operation size? If it's a 4K operation and it takes 5ms, maybe that's bad in your environment.. but what if the requested op was a 1meg op that took 6ms? Is that bad?
I'm not intending to answer in circles, it just isn't exactly cut and dry and greatly depends on your environment.
In lieu of some specific application requirement, I would say that expecting read times < 20ms for operations that aren't huge (for some definition of "huge") is a reasonable starting expectation. For writes to a NetApp system, I'd expect that the nominal response time is well less than that (5ms or less?). These starting figures would be for a healthy system that is not being overrun with more work than it is designed or sized to handle. These expectations might need to be adjusted up or down depending on what is normal for the environment (e.g., measured "normal" by running for a period of time and averaging out the daily/weekly spikes [shift on, shift off, lunch, etc.]).
Parting thought... given two applications, A and B: Application A may be completely content with a response time of 30 ms for an 8k operation as it's work is considered to be more batch than directly user-interactive. However, Application B might expect a response time of 10 ms worst case for an 8k operation because that IO response directly feeds back to the user in some form of UI.
-jbl
