After a recent update from 8.3.2P10 to 9.1P10 on FAS 62xx systems, I noticed a major change in read caching patterns on existing Flash Pool aggregates. More specifically:
the percentage of reads from Flash Pool remained constant
the percentage of reads from HDD increased by about 25%
the percentage of reads from RAM decreased by the same amount.
As a result, read latencies increased significantly. The workload is the same as before the update and no configuration setting has been changed, so I believe this behaviour is a consequence of the ONTAP upgrade.
Having a look at ONTAP 9.1 release notes I could not find anything related, except that the caching retention policy on Flash Pool volumes can be configured. Currently this settings is at the default level "normal" for all volumes. Did anyone notice such behaviour? Was the default caching behaviour changed in ONTAP 9.1? What kind of mitigation could be put in place? Thanks.