Debug failed mount: No such file or directory

[PatrikOsgnach]

Hi all, I recently inherited an already configured AFF-A300. It already has some volumes, exported via NFS3 and they all work fine.

Today I tried to add a new volume via the Ontap system manager, but the mount fails:

# mount -v -t nfs vrc01n01b:/proj_rfip_hki /root/hki
mount.nfs: timeout set for Fri Dec 20 19:21:29 2024
mount.nfs: trying text-based options 'vers=4,addr=10.217.231.21,clientaddr=10.217.231.88'
mount.nfs: mount(2): Protocol not supported
mount.nfs: trying text-based options 'addr=10.217.231.21'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 10.217.231.21 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 10.217.231.21 prog 100005 vers 3 prot UDP port 635
mount.nfs: mount(2): No such file or directory
mount.nfs: mounting vrc01n01b:/proj_rfip_hki failed, reason given by server: No such file or directory

To check the most obvious reasons: 

> volume show -vserver vrcv01a -fields volume,node,junction-path,policy
vserver volume    policy                  junction-path node      
------- --------- ----------------------- ------------- ---------
vrcv01a proj_rfip_hki 
                  clients                 /proj_rfip_hki 
                                                        vrc01n01b

> vserver export-policy check-access -vserver vrcv01a -client-ip 10.217.231.88 -volume proj_rfip_hki -authentication-method sys -protocol nfs3 -access-type read
                                Policy    Policy     Rule             Security
Path                 Policy     Owner     Owner Type Index Access     Style
-------------------- ---------- --------- ---------- ---- ---------- ----------
/                    default    vrcv01a_rootvol 
                                          volume        2 read       unix
/proj_rfip_hki       clients    proj_rfip_hki 
                                          volume        2 read       unix

>  export-policy access-cache show -node vrc01n01b -vserver vrcv01a -policy clients -address 10.217.231.88

                                        Node: vrc01n01b
                                     Vserver: vrcv01a
                                 Policy Name: clients
                                  IP Address: 10.217.231.88
                    Access Cache Entry Flags: has-usable-data
                                 Result Code: 0
                 First Unresolved Rule Index: -
                      Unresolved Clientmatch: -
              Number of Matched Policy Rules: 1
         List of Matched Policy Rule Indexes: 2
                                Age of Entry: 741s
                 Access Cache Entry Polarity: positive
Time Elapsed since Last Use for Access Check: 407s
      Time Elapsed since Last Update Attempt: 741s
               Result of Last Update Attempt: 0
                List of Client Match Strings: 10.217.231.0/25

From the client:

# showmount -e vrc01n01b
Export list for vrc01n01b:
/proj_rfip_hki                          (everyone)

So, volume is there, it is mounted, it has an export policy and the client is allowed to mount.

What am I missing? Is there anything like "exportfs -r" that I have to run?

[TMACMD]

Not sure if I missed it…how about

 export-policy rule show -instance -vserver vserver

[PatrikOsgnach]

There it is

> export-policy rule show -instance -vserver vrcv01a 

                                    Vserver: vrcv01a
                                Policy Name: clients
                                 Rule Index: 2
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: 10.217.231.0/25
                             RO Access Rule: any
                             RW Access Rule: any
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: false
                  Allow Creation of Devices: true

                                    Vserver: vrcv01a
                                Policy Name: default
                                 Rule Index: 1
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: 0.0.0.0/0
                             RO Access Rule: any
                             RW Access Rule: any
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: false
                  Allow Creation of Devices: true

                                    Vserver: vrcv01a
                                Policy Name: policy_10390_1707921307
                                 Rule Index: 1
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: 10.217.231.0/25
                             RO Access Rule: any
                             RW Access Rule: none
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: false
                  Allow Creation of Devices: true

                                    Vserver: vrcv01a
                                Policy Name: policy_11210_1707920505
                                 Rule Index: 1
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: 10.217.231.0/25
                             RO Access Rule: any
                             RW Access Rule: any
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: true
                  Allow Creation of Devices: true

[PatrikOsgnach]

Part 2

 Vserver: vrcv01a
                                Policy Name: policy_11210_1707920505
                                 Rule Index: 3
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: @vr.urania.intel.com
                             RO Access Rule: any
                             RW Access Rule: any
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: true
                  Allow Creation of Devices: true

                                    Vserver: vrcv01a
                                Policy Name: policy_29374_1725895138
                                 Rule Index: 1
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: 10.217.231.0/25
                             RO Access Rule: any
                             RW Access Rule: any
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: true
                  Allow Creation of Devices: true

                                    Vserver: vrcv01a
                                Policy Name: policy_5716_1707922398
                                 Rule Index: 1
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: @adm_h_nasadm_fah_ivr
                             RO Access Rule: any
                             RW Access Rule: any
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: true
                  Allow Creation of Devices: true

                                    Vserver: vrcv01a
                                Policy Name: policy_7612_1707915997
                                 Rule Index: 1
                            Access Protocol: any
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: 10.217.231.0/25
                             RO Access Rule: any
                             RW Access Rule: none
User ID To Which Anonymous Users Are Mapped: 65534
                   Superuser Security Types: none
               Honor SetUID Bits in SETATTR: false
                  Allow Creation of Devices: true
8 entries were displayed.

[TMACMD]

Which export policy is applied to the svm root volume? For nfs clients, they must be able to access each junction path including / (svm root). I do not see that on the showmount command and I would expect it

 and how about this 

Vserver nfs show -fields v3,v4.0,v4.1

 plus the

 volume show -fields policy -volume svm-root

[PatrikOsgnach]

Ah, yes, sorry, concerning showmount I pasted only the line about my affected share (since there is a big amount of them

> showmount -e vrc01n01b
Export list for vrc01n01b:
/proj_rfip_hki                          (everyone)
/                                       (everyone)

> vserver nfs show -fields v3,v4.0,v4.1
vserver v3      v4.0     v4.1     
------- ------- -------- -------- 
vrcv01a enabled disabled disabled

> volume show -fields policy -volume vrcv01a_rootvol
vserver volume          policy  
------- --------------- ------- 
vrcv01a vrcv01a_rootvol default

Then I have also

> volume show -fields policy -nodes vrc01n01b        
vserver   volume policy 
--------- ------ ------ 
vrc01n01b vol0   -    
...

Now there are many volumes, of course, but I don't recognize what this vol0 is

[PatrikOsgnach]

> vserver show
                               Admin      Operational Root
Vserver     Type    Subtype    State      State       Volume     Aggregate
----------- ------- ---------- ---------- ----------- ---------- ----------
vrc01       admin   -          -          -           -          -
vrc01n01a   node    -          -          -           -          -
vrc01n01b   node    -          -          -           -          -
vrcv01a     data    default    running    running     vrcv01a_   aggr1_
                                                      rootvol    vrc01n01a_
                                                                 S
4 entries were displayed.

> export-policy rule show -policyname default
             Policy          Rule    Access   Client                RO
Vserver      Name            Index   Protocol Match                 Rule
------------ --------------- ------  -------- --------------------- ---------
vrcv01a      default         1       any      0.0.0.0/0             any

[TMACMD]

vol0 is an moot volume. No data should ever be there

 try this temporarily 

mount -v -o vers=3 vrc01n01b:/ /tmp_path that should work. Then try to access the path you are looking for on the temp location. You may get better diagnostic messages. 

 also, it appears that the “string” path you are supplying to mount is not matching the “string” path on the Netapp. Maybe there is a space somehow in the name on the Netapp? 

[PatrikOsgnach]

Good idea

# mount -v -o vers=3 vrc01n01b:/ /tmp/hki
mount.nfs: timeout set for Sat Dec 21 16:28:17 2024
mount.nfs: trying text-based options 'vers=3,addr=10.217.231.21'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 10.217.231.21 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 10.217.231.21 prog 100005 vers 3 prot UDP port 635
# ls -l /tmp/hki/proj_rfip_hki
ls: cannot access '/tmp/hki/proj_rfip_hki': No such file or directory

Seems there is no directory for the mountpoint

[TMACMD]

Just do an ls of the temp directory and see what’s there. Like I said maybe there is  a mismatch of what you think versus what actually is. 

still looks like the directory make you are trying to look for isn’t there. What about

 vol show

 don’t add anything. Just find your volume and then paste the output. Wondering if the volume is online, read/write

[PatrikOsgnach]

I did and I see directories for all the other volumes

[TMACMD]

Which is why you should see if the volume is online and read/write capable from the basic vol show output

[TMACMD]

What is the volume security on tries volume? There is a possibility if it is ntfs that may be why its not showing

[PatrikOsgnach]

It's UNIX

[PatrikOsgnach]

> vol show -volume proj_rfip_hki
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
vrcv01a   proj_rfip_hki 
                       aggr1_vrc01n01b_S 
                                    online     RW       1.05TB     1023GB    0%

[TMACMD]

Another thought. Network lif. 
net int show -vserver xx -lif lifname -fields service-policy

 then

 net int service-policy show -vserver xx -policy fromabove 

[TMACMD]

That will be a problem!

 Based on the ONTAP output you should be able to mount

NFSip:/proj_rfip_hki 

if you want another path in there you either need to create a small flex vol and junction to root or get on the root volume as super user and make your directory

 then unmount the volume you are trying to access from the name space and mount against to the junction path. 

the full path must exist

[PatrikOsgnach]

Accorting to the ONTAP System Manager, I should be able to mount the volume from either 

10.217.231.20:/proj_rfip_hki or 10.217.231.21:/proj_rfip_hki ... neither work.

Shouldn't the junction path be created automatically? The mount command seems to work (or at least it doesn't give any explicit error)

If there is no mkdir command in the CLI interface, shall I mount the root of the SVM on a linux host and create the mount point?

[TMACMD]

I’ll send some directions later when I’m at a computer

[PatrikOsgnach]

> net int show -vserver vrcv01a -lif vrc01n01b-1 -fields service-policy 
  (network interface show)
vserver lif         service-policy    
------- ----------- ----------------- 
vrcv01a vrc01n01b-1 custom-data-48468
> net int service-policy show -vserver vrcv01a -policy custom-data-48468
  (network interface service-policy show)

                   Vserver: vrcv01a
               Policy Name: custom-data-48468
         Included Services: data-core, data-nfs, data-fpolicy-client,
                            management-dns-client, management-ad-client,
                            management-ldap-client, management-nis-client,
                            data-dns-server, backup-ndmp-control
Service: Allowed Addresses: data-core: 0.0.0.0/0
                            data-nfs: 0.0.0.0/0
                            data-fpolicy-client: 0.0.0.0/0
                            management-dns-client: 0.0.0.0/0
                            management-ad-client: 0.0.0.0/0
                            management-ldap-client: 0.0.0.0/0
                            management-nis-client: 0.0.0.0/0
                            data-dns-server: 0.0.0.0/0
                            backup-ndmp-control: 0.0.0.0/0

Now... what happens if the path where the volume should be mounted does not exist? Can I manually create it?

I can mount all other shares, except this one