Is PAM II working optimally?

casey_condo — Thu, 05 Jun 2025 06:20:34 GMT

We have a 512GB PAM II card

and when I run a stats show -p flexscale-pcs it shows a high % (92%) usage, a low Hit % (9%) but it doesnt show a high evict or insert rate.

But when I run a sysstat -u 1 I get a cache hit rate (94%) but my disk utilization is high (91%)

We run VMs on this server. My question: is my system cache maninly being used a very little making it to the PAM ? It shows a lot of information in PAM but the hit rate is very low.

Or is this to be expected because the VMs are large files etc?

Re: Is PAM II working optimally?

GARDINEC_EBRD — Tue, 21 Aug 2012 13:42:34 GMT

Hi Casey,

Here's some info from memory...

Firstly, the cache hit rate shown in sysstat output isn't showing the same thing as the flexscale stats. Ignore the sysstat one for now...

With a low hit rate you should look into what type of data you are missing, and what you are caching. If you use stats show ext_cache_obj this will give you more info. Better still run it over a few minutes using stats start -I cache ext_cache_obj ...wait for a minute or two during a busy period... stats stop -I cache.

This will give an output like this (notes added for some counters to check):

StatisticsID: cache

ext_cache_obj:ec0:type:IOMEM-FLASH

ext_cache_obj:ec0:blocks:67108864

ext_cache_obj:ec0:size:256 <------- size of cache in GB

ext_cache_obj:ec0:usage:92% <------- How full the cache is (should be >90% unless you've just booted)

ext_cache_obj:ec0:accesses:32047/s <----- number of times the cache was accessed per sec

ext_cache_obj:ec0:disk_reads_replaced:3768/s <---- how many disk reads were not replaced by the cache.

ext_cache_obj:ec0:hit:17898/s <---- ave number of hits per sec (add to the number of misses below to get total)

ext_cache_obj:ec0:hit_normal_lev0:16517/s <---- this is 'normal' data hits. here you can see most of the hits are 'normal' data

ext_cache_obj:ec0:hit_metadata_file:907/s <--- this and the next few are the types of hit

ext_cache_obj:ec0:hit_directory:42/s

ext_cache_obj:ec0:hit_indirect:435/s

ext_cache_obj:ec0:total_metadata_hits:1385/s

ext_cache_obj:ec0:total_metadata_misses:434/s <--- if this is high relative to the total operations, you should consider metadata cache mode

ext_cache_obj:ec0:miss:14149/s <---- number of misses per sec

ext_cache_obj:ec0:miss_metadata_file:85/s <---- this and the next few are the types of miss

ext_cache_obj:ec0:miss_directory:1/s

ext_cache_obj:ec0:miss_indirect:347/s

ext_cache_obj:ec0:hit_percent:55% <--- % hits (hits / miss)

ext_cache_obj:ec0:inserts:426/s

ext_cache_obj:ec0:inserts_normal_lev0:337/s

ext_cache_obj:ec0:inserts_metadata_file:49/s

ext_cache_obj:ec0:inserts_directory:6/s

ext_cache_obj:ec0:inserts_indirect:39/s

ext_cache_obj:ec0:evicts:11/s <---- this indicates the number of blocks evicted (ie not accesses frequently) If this is very high you *may* need more flash caches

ext_cache_obj:ec0:evicts_ref:6/s

ext_cache_obj:ec0:readio_solitary:1524/s

ext_cache_obj:ec0:readio_chains:3768/s

ext_cache_obj:ec0:readio_blocks:15788/s

ext_cache_obj:ec0:readio_max_in_flight:511

ext_cache_obj:ec0:readio_avg_chainlength:4.19

ext_cache_obj:ec0:readio_avg_latency:0.56ms

ext_cache_obj:ec0:writeio_solitary:0/s

ext_cache_obj:ec0:writeio_chains:6/s

ext_cache_obj:ec0:writeio_blocks:426/s

ext_cache_obj:ec0:writeio_max_in_flight:182

ext_cache_obj:ec0:writeio_avg_chainlength:64.00

ext_cache_obj:ec0:writeio_avg_latency:2.14ms

ext_cache_obj:ec0:invalidates:514/s <--- data in the cache that's been overwritten on disk (and thus 'invalidated')

Based on this you may need to adjust your cache settings (options flexscale) to cache different data.

You should also check you're VM alignment - misaligned blocks may affect caching efficiency. If that looks OK, dedupe may also help.

If you have Performance Advisor you can also collect these stats and review regularly if required.

Hope that helps,

Craig

Re: Is PAM II working optimally?

casey_condo — Tue, 21 Aug 2012 15:49:25 GMT

I feel a bit silly - it was because we had a degraded raid set and it was reb rebuilding...

topic Re: Is PAM II working optimally? in ONTAP Discussions

Is PAM II working optimally?

Re: Is PAM II working optimally?

Re: Is PAM II working optimally?