topic Re: Quick file scan in Software Development Kit (SDK) and API Discussions

Quick file scan

crousseaux — Wed, 04 Jun 2025 22:02:40 GMT

Hello,

I am searching for the possibility with NetApp API and/or Powershell to do a quick scan on CIFS and NFS volume. My objective is to working with result offline?

The aim is to retrieve the list of files, created time, modified ... size
And then parse the results for statistical or identify the file to copy ...

In a Windows NTFS FS (NTFS Master File Table) or Linux EXTx (inode), it is possible to access and scan this table of pointers in order to quickly browse and recover these items without having to really scan the FS.
This is very useful for a FS with millions of files.

cordially,

Re: Quick file scan

mbeattie — Thu, 18 Feb 2016 01:35:06 GMT

Hi,

Sure It's possible to write a script to recursively enumerate a file system and log the results (assuming you run the script as a user who has administrative access throughout the entire file system) if such a user exists? Often NTFS permission inheritance is removed and users can remove administrative access in the event where they have been granted full control permissions, so the process you are suggesting it is prone to error and your script would need to implement a high degree of error checking to account for access denied errors. Depending on the number of files in your file system it could take a long time to run. I don't think there is any method available within WAFL to directly\quickly enumerate this information. You will need to peform a file system scan.

However, if your objective is to peform a migration of files to a destionation based on file age or date last access\modified then have you considered using robocopy with the following paramaters?

/CREATE :: CREATE directory tree and zero-length files only.

/MAXAGE:n :: MAXimum file AGE - exclude files older than n days/date.
/MINAGE:n :: MINimum file AGE - exclude files newer than n days/date.
/MAXLAD:n :: MAXimum Last Access Date - exclude files unused since n.
/MINLAD:n :: MINimum Last Access Date - exclude files used since n.

/matt

Re: Quick file scan

dstephens — Thu, 18 Feb 2016 05:52:54 GMT

Robocopy is a good way to do it but I'd recommend using the /l parameter as well.

Another option I have been investigating is Powershell e.g. get-childitem -path h:\usergarbage -recurse | select Name,FullName,Length,CreationTime,LastAccessTime,LastWriteTime | export-csv test.csv

❤️ Dan

Re: Quick file scan

anton_oks — Thu, 18 Feb 2016 12:20:32 GMT

If I get your question right, you want a NetApp API way to check the "file meta" informations (aka. inode's), right?

If so, your (7-Mode) API is "file-list-directory-iter-start".. and you need to implement a "recursive" way of searching, if desired

Example:

input:

<?xml version="1.0" encoding="UTF-8"?>
<netapp xmlns="http://www.netapp.com/filer/admin" version="1.19">
   <file-list-directory-iter-start>
     <path>/vol/anton_test</path>
   </file-list-directory-iter-start>
</netapp>

output:

      .
      .
      .
   <file-info>
    <name>anton</name>
    <file-type>directory</file-type>
    <creation-timestamp>1354800171</creation-timestamp>
    <modified-timestamp>1355758658</modified-timestamp>
    <changed-timestamp>1355758658</changed-timestamp>
    <accessed-timestamp>1428096544</accessed-timestamp>
    <perm>755</perm>
    <owner-id>0</owner-id>
    <group-id>0</group-id>
    <file-size>4096</file-size>
    <hard-links-count>5</hard-links-count>
    <inode-number>96</inode-number>
    <acl-type>no_acl</acl-type>
    <is-empty>false</is-empty>
    <is-vm-aligned>false</is-vm-aligned>
   </file-info>
      .
      .
      .

And if you have access to the filesystems with a NFS and/or SMB client, it's faster and smarter to use host OS tools, as other stated already...

regards

Anton Oks

Re: Quick file scan

crousseaux — Mon, 22 Feb 2016 14:11:52 GMT

thank for your answer Anton,

actually, costume use the unix cmd find in script
but for millions of file it is not quicly method

on filesystem ntfs, it is possible tu use tools scan like TreeSize.
this tools scan MFT (Master File Table). It is very quicly.
but this work only with local or san FS.

i not tested your solution, i going try it.

thank you,

Re: Quick file scan

crousseaux — Mon, 22 Feb 2016 14:19:56 GMT

Hello Dan,

I try it,

it make exactly wath i want,

but i am not sur that more quickly than robocopy

thank,

Re: Quick file scan

richard_payne — Mon, 22 Feb 2016 14:51:03 GMT

It still requires access from a client (vs API calls to the filer etc...) but you can also look at NetApp's XCP copy tool - https://xcp.netapp.com. It has some options to scan a file system and report on the metadata.

--rdp

Re: Quick file scan

mbeattie — Tue, 23 Feb 2016 00:48:05 GMT

Hi Anton,

Great information, your right, i didn't consider that. Looking at those ZAPI's you can also use the following PowerShell CmdLets. EG

Import-Module DataONTAP
Get-Help Read-NaDirectory -full
Get-Help Get-NaFile -full

These could be combined to list all directories and files recursively...However keep in mind you would be potentially invoking a lot of ZAPI calls depending on the size of the file system. I'm not sure if this would be any quicker than scaning the file system. Either way it might have a performance impact on the system so you'd want to ensure you monitor that whilst running the script. It would be interesting to guage the performance of the directory and file listing by using measure-object on the ZAPI method compared to other methods mentioned here:

http://thesurlyadmin.com/2014/08/04/getting-directory-information-fast/

/matt