FabricPool is only for SSD aggregates. FlashCache does not give benefits for SSD aggregates; therefore FlashCache is not working with FabricPool aggregates.
The same applies to FlashPool: if you have a hybrid aggregate consists of HDD + SSD (FlashPool), then FlashCache not going to work for that aggregate.
Do not mix up FabricPool (Data Tiering Technology to S3 object storage) with FlashPool (Hybrid Disk Caching Technology) for read/write caching, while FlashCache is almost the same as FlashPool, except for the fact it is a card (PCIe or M2 with NVMe memory on it), it works controller-wide, and only for read caching.
Well, I have an internal NetApp document infront of me called "How to configure flash cache cardsto work on a Flash pool aggregate.pdf" which states:
Starting with Ontap 9.1 onwards, Flash Cache could be configured to be used on a Flash pool aggregate together; this is controlled by a node shell option flexscale.flash_pool_caching, which is by default set to off.
To enable this:
clusthershell> node run <node_name> nodeshell> options flexscale.flash_pool_caching on
To find out how much Flash Chache helps in a Flash pool aggregate:
ext_cache_obj:ec0:disk_reads_replaced: Estimated number of disk reads per second replaced by cache ext_cache_obj:ec0:hya_reads_replaced: Estimated number of HyA reads per second replaced by cache
And I have several confirmations that this works from netapp engineers.
Hi @as - To help make your question clearer, I think you need to clarify - do you mean fabric pools or flash pools? I know the names are similar, but the technology is not and it seems you're using them interchangably in your question and haven't addressed this in your subsequent followups
Also.. internal documents are internal for good reasons. They may refer to functionality which is not available or supported publicly, and which may impact data integrity or availability. Some functionality requires undocumented boot flags to be set, or even license codes applied.
If you are having performance concerns, please open a ticket with our support team and they will work through it with you, including enabling other functionality if required.
And yes, I understand they are internal documents and I was thinking for a while until I posted it - maybe I should not have done it...
Fully undertand that it might have impact and requires undocumented options etc., hence my question if someone has expirience with it and has it running in production.
I tried to involve NetApp support via our SAM but no luck so far, only answer I got is that this is not a support question. I tried to involve pre-sale, also no luck and no answer. That's why I asked in the end the community...
That KB explains how to use caching policies in a FlashPool aggregate and not related to FlashCache in any way. Though FlashCache & FlashPool have very similar names, algorithms, and some similar read caching policies, they are different not compatible things. FlashCache, FlashPool & FabricPools are technologies which currently mutually exclude each other.
Maybe, just maybe at some point in the future, when NetApp will have Storage Class memories (SCM) in AFF systems, then will be a new generation of FlashCache cards based on SCM memory which will work with FabricPool or All Flash aggregates. It's not available yet though.
"Data from volumes that are provisioned in a FlashPool aggregate or an all-SSD aggregate is automatically excluded from being inserted into FlashCache cache. Data from volumes provisioned in a FlashPool aggregate use the FlashPool cache, and data from volumes provisioned on an SSD aggregate does not benefit from using FlashCache."
Though NVMe interface is more powerful than SAS, FlashCache has already been PCIe cards, so there should be no or small difference in performance between FlashCache based on PCIe vs. FlashCache based on NVMe interface. While media itself for FlashCache for old & new cards was and are nearly the same NAND technology, therefore no significant performance difference and consequently no significant caching difference there too. Only SCM have big performance difference which can give a boost for caching for all-SSD aggregates.
That is precisely the reason why All Flash systems do not have FlashCache cards on-board.
Hi @D_BEREZENKO thank you for this, and all your contributions on this forum, but I have triple checked (I double checked before I posted 🙂 and I stand by my original comments. The TR you linked to pre-dates ONTAP 9.1 where this was released, and so that piece of information is technically invalid.
It is not a major piece of functionality and so it is not broadly covered by marketing or other material. Flashcache and Flashpool SE Presentation v2.0, slide 28 if you have access and you'd like to read more on the subject.
In my understanding, there are few potential areas for benefit when using (current gen) FlashCache + FlashPool. Apparently in this case you'll get the bigger total amount of cache, which meant nothing by itself, but for certain data pattern it might be beneficial:
When you have more read operations & those read operations are small random blocks
Optimization for certain read caching operations
Some benefits for situations for short bursts of sequential operations and/or big blocks of data been read