How to identify Cold/Archive/Infrequent data from Netapp Volumes on FAS systems?

[av]

Hi,

I've quite a few FAS systems, mostly running HDD aggregates & some on SSD aggregates. These systems have been in use for ages & likely to have orphan or less used data on the volumes.

I'd like to find a way to scan volume data & identify files/directories that are infrequently used or cold? Is there a tool/script to do this on Netapp FAS systems?

I understand Fabric pool has a feature " Inactive data reporting" that can be leveraged however I DON'T use fabric pools. 

I do have Netapp OCI installed. Can it be used for above use case? Is yes, please advise how?

Additionally, can Netapp's XCP tool be used for a such a task? I am also open to explore 3rd party tools - preferably open source.

I'd like to hear your views on how to go about achieving this task?

Regards,

Ashwin

[donny_lang]

A script something like what is described in this blog post could potentially fit the bill - it'll just run recursively on a given share/directory and give you the list of files that fit your definition of "cold/archive/infrequent" and also the total size of all of the files that fit the criteria. 

As far as I know, inactive data reporting will only identify the total amount of data that is considered inactive, not necessarily the files themselves. 

[av]

Appreciate your response Donny. I'll try out web link.

I am sure others would have dealth with a similar situation.  Would like to hear views/idea to achieve it.

Maybe get some reference about an open source tool that I could use 🙂

Regards,

Ashwin

[Sergey_Osipov]

Inactive Data Reporting
First available in ONTAP 9.4, inactive data reporting (IDR) is an excellent tool for determining the amount of inactive (cold) data that can be tiered from an aggregate.
IDR uses a 31-day cooling period to determine what data is considered inactive. The amount of cold data that is tiered is dependent on the tiering policies set on the aggregate volumes. This amount might be different than the amount of cold data detected by IDR using a 31-day cooling period.
• IDR is enabled by default on all SSD aggregates in ONTAP 9.6.
• IDR is enabled by default on FabricPool aggregates in ONTAP 9.4 and ONTAP 9.5.
• IDR can be enabled on non-FabricPool aggregates using the ONTAP CLI. This includes HDD aggregates starting in ONTAP 9.6.

• IDR cannot be enabled for situations in which FabricPool cannot be enabled: for example, root, aggregates, MetroCluster, and so on.

Please have a look by this link

https://docs.netapp.com/ontap-9/topic/com.netapp.doc.dot-cm-psmg/GUID-78C09B0C-9508-4CEC-96FE-7ED73F7F5120.html?cp=2_8_5_5_0

[av]

Thank you Sergey.

I wasn't aware of "IDR can be enabled on non-FabricPool aggregates using the ONTAP CLI. This includes HDD aggregates starting in ONTAP 9.6.".  

Hopefully it reports detail info about File name, Size, Access & Moodified tim. I'll try it out.

Does XCP provide a similar funtion?

Regards,

Ashwin

[Sergey_Osipov · Accepted solution]

Hello,

"Hopefully it reports detail info about File name, Size, Access & Moodified tim. I'll try it out."

Nope, it is internal Ontap feature which only dipslays how much data is inactive on a Netapp volume.

Please bear in mind that  FabricPool operates at the block level from the ONTAP point of view, but the effective results enable you to tier entire files as well as blocks within a larger file.

Please have a look on Fabric Pool TRs 

https://www.netapp.com/us/media/tr-4598.pdf

https://www.netapp.com/us/media/tr-4695.pdf