topic Re: Shares, LUNs, and Qtrees - what is the best method for storing 6mil+ records? in ONTAP Discussions

Shares, LUNs, and Qtrees - what is the best method for storing 6mil+ records?

AKDOE — Wed, 04 Jun 2025 09:56:07 GMT

One of our applications stores and retrieves PDFs from a single directory. This directory exists as a share on on a FlexVol, and contains more than 6 million documents. The issue that I am having is with the memory allocated to that share index. For example, as new records are added, I have to increase the allocated memory in order to add the new files to that directory. As of now, I'm somewhere around 1GB of memory.

My question is: given the number of records the directory contains, is there a better way to store the documents? That is, should I use Qtrees or LUNs?

I am a novice when it comes to data storage as well as NetApp, so some guidance and recommendations are much appreciate. My apologies if I've not been clear in describing the problem, I can try to clarify as needed.

Re: Shares, LUNs, and Qtrees - what is the best method for storing 6mil+ records?

paul_stejskal — Fri, 28 Oct 2022 14:23:04 GMT

Can you put the files into separate folders? Having a very wide directory like that creates a lot of memory pressure and could make things extremely latent.

Re: Shares, LUNs, and Qtrees - what is the best method for storing 6mil+ records?

AKDOE — Fri, 28 Oct 2022 17:55:42 GMT

Hi @paul_stejskal, thanks for replying. Yeah, that would be the preferred method as we do experience latency issues. I was able to fine-tune the QOS policy group to set priority to that directory. However, the files are saved to the directory via an automated process in the application, and to add them to separate folders requires a code update in the application. Unfortunately, our application developer (vendor) is not very responsive, and we are not able to get this work done anytime soon. I was hoping there would be a way I could tackle the problem from the NetApp side.

Re: Shares, LUNs, and Qtrees - what is the best method for storing 6mil+ records?

paul_stejskal — Fri, 28 Oct 2022 19:41:27 GMT

Oof. No unfortunately there isn't a great way to improve this. Hopefully your application developer can change this or something.

Re: Shares, LUNs, and Qtrees - what is the best method for storing 6mil+ records?

AKDOE — Fri, 28 Oct 2022 19:57:07 GMT

Rats. That's what I was afraid of. Thank you for clarifying for me.

Re: Shares, LUNs, and Qtrees - what is the best method for storing 6mil+ records?

paul_stejskal — Fri, 28 Oct 2022 20:00:10 GMT

You're welcome. Sory I couldn't have better news.