FAS and V-Series Storage Systems Discussions

Performance Issue because of having high number of files in filer


Hi Team,

I am facing performance issue in one of our NetApp cluster.

I have checked and found the filer having large number of files/directories (millons of files) have slow response time as compare to the filer which have less number of files.

we have recently purchased FAS 3270 which again have millons of files and is working slowly.

I have few questions in my mind:

Is this issue comes because of total number of files stored in a filer or files used by memory?

Infact I have increased the read only memory by installing the 520 GB Flash Cache module which will not worked.

Is there any way through which we are able to see, out of total available files how may files are actually used.?

Did some one know root cause of this issue?

Please try to help me for resolving this issue which really create a big impact in our environment.

Best Regards

Pulkit Kaul



Do you see a lot of "other" IOPs?  Is the flashcache in default mode (data and metadata)?  It might help to set to metadata only if a lot of directory and metadata lookups.  A perfstat would be interesting to have analyzed to see performance.


HI have tried both the modes but still not get any success.

I have generated the perfstat output which clearly show the number of files in filer dlhfs15 is much higher as compare to filer bnrfs04.

Below is the analysis which NetApp has provided:

Hello Pulkit,

I checked both the perfstat and as suspected the high file
count is the main issue. Below is the detailed analysis.


volume:fs04ag1v1:avg_latency:18.53us – BNRFS04

volume:fs15ag5v2:avg_latency:311.90us–DLHFS15 – The latency on the volume is
way higher than BNRFS04

nfsv3: nfs:
64.66us -

nfsv3:nfs:nfsv3_op_latency.readdirplus:6041.57us - DLHFS15


Server nfs V3: (2824081 calls)

getattr    setattr  
lookup     access   
readlink   read

2 0%   
  2210077 78% 24892 1%   241781 9%  115522 4%  0
0%       24381 1%

create     mkdir    
symlink    mknod    
remove     rmdir

106985 4%  66943 2% 
6200 0%    298 0%     0
0%       18853 1%   412 0%

link       readdir   
readdir+   fsstat   
fsinfo     pathconf

2238 0%    0
0%       0 0%      5440 0%    55
0%      2 0%       0 0%


nfs V3: (1309331 calls)

getattr    setattr  
lookup     access   
readlink   read

0%      379717 29% 43240 3%   110999 8%
127401 10% 516 0%     114686 9%

create     mkdir    
symlink    mknod    
remove     rmdir

15% 46584 4%   5460 0%    454
0%     0 0%       105022
8%  27123 2%

link       readdir   
readdir+   fsstat   
fsinfo     pathconf

1%   1268 0%    252 0%    
138377 11% 881 0%     56
0%      5 0%



nfsv3:nfs:nfsv3_op_latency.readdir:0us - BNRFS04 directory scanning

nfsv3:nfs:nfsv3_op_latency.readdirplus:64.66us - BNRFS04 directory scanning

nfsv3:nfs:nfsv3_op_latency.readdir:1132.03us - DLHFS15 directory scanning

nfsv3:nfs:nfsv3_op_latency.readdirplus:6041.57us - DLHFS15 directory scanning – This
build does a lot of directory scanning (11% of the operation is directory
scanning) and the latency for the same on DLHFS15 is much higher than that of


iused      ifree  %iused  Mounted on

934210   30942479      3%

3611657   28265032     11%  /vol/fs04ag1v2/

4833894   27042795     15%  /vol/fs04ag2v1/

224695    7229982      3%

Total number of files 9604456


iused      ifree  %iused  Mounted on

18732711   41267274     31%  /vol/fs15ag6v1/

36341917   25534764     59%  /vol/fs15ag2v1/

18479935   13396754     58%  /vol/fs15ag5v1/

2270413   37729577      6%

5215281   26661408     16%  /vol/fs15ag6v2/

9713544   43286444     18%  /vol/fs15ag2v2/

21013190   10863499     66%  /vol/fs15ag5v2/

23117514   27013924     46%  /vol/fs15ag1v1/

13323666   18553023     42%  /vol/fs15ag4v1/

7319    3451513      0%

11803990   19325591     38%  /vol/fs15ag4v2/

12170159   19706530     38%  /vol/fs15ag1v3/

18837357   21162633     47%  /vol/fs15ag1v2/

Total number of files 191026996

The number of files in BNRFS04 is
9604456whereas in DLHFS15, it is 191026996. This is the root cause of it. The
solution for this problem is to have a system with more memory. We have already
tried this with different configuration of PAM on FAS6040, but its caching is
not helping with your workload. This is the configuration that we tried in your
environment for PAM

  1. flexscale.enable           
    on         (same value in local+partner
  2. flexscale.lopri_blocks     
    off        (same value in local+partner
  3. flexscale.normal_data_blocks
    off        (same value in local+partner
  4. flexscale.pcs_high_res     
    off        (same value in local+partner
  5. flexscale.pcs_size         
    0GB        (same value in local+partner

But just want to confirm why Flash Cache is not usefull in such type of environment.

Also I believe the number of file available in filer doesn't matters. In fact the thing which matter is the the number of files or indoes for which lookup requests is generated.

Did any one know how we find the the number of lookup request generated by the filer?

Please comment on that.


Hi Guys,

Still waiting for the reply.

Can someone please comment on this.

Best Regards

Pulkit Kaul.


This sounds like a support related question. Performance related issues are rather difficult to resolve via communities, I suggest you open a case with NetApp Technical Support.



NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner