VMware and NetApp NFS volume issue

[Andrew7193]

I am not sure if this is a VMware question best posted on their forums or a NetApp question best posted here. However, since there are people here that use NetApp with VMware I figured some one would have seen this happen.

We recently attached a 2 TB data store to a VM farm. We started to migrate VM to the store and all went well until the last two were to migrate and they failed with an insufficient space error. We had only moved 500 GB of virtual machines to the store. The admin added 500 more GB to the store and the migrations failed again with the same error. The interesting thing is every machine we were moving were either Windows servers made from a template or a virtual appliance except for the two we are having issues with and they are Linux machines made from scratch.

I shut the VMs down and they vMotioned over correctly but when I attempted to bring them up VMware complained about not enough room and now I cannot vMotion them off of the data store.  

Environment:

vCenter 6.5

ESXi 6.5

ONTAP 9.4P2

Export Policy:

 Rule Index: 1
Access Protocol: nfs3
List of Client Match Hostnames, IP Addresses, Netgroups, or Domains: 0.0.0.0/0
RO Access Rule: sys
RW Access Rule: any
User ID To Which Anonymous Users Are Mapped: 65534
Superuser Security Types: any
Honor SetUID Bits in SETATTR: true
Allow Creation of Devices: true

[mbeattie]

Hi Andrew,

Have you checked that there is sufficent space within the aggregate that hosts the flexvol on the storage? Is your flexvol volume thin provisioned (IE "space-guarantee = none")? Also what's the snapshot policy applied to the flexvol? If you are taking freqent snapshots whilst you are migrating to the volume you are probably consuming space for snapshots.

Here are some example commands to check the storage configuration:

cluster1::*> aggr show -aggregate testc1n1_aggr1 -fields size, usedsize
aggregate      size    usedsize
-------------- ------- --------
testc1n1_aggr1 147.7GB 3.18GB

cluster1::*> vol show -vserver vserver2 -volume nfs_data_001 -fields size, used, snapshot-space-used, snapshot-count, snapshot-reserve-available, snapshot-policy, space-guarantee
vserver  volume       size used   space-guarantee snapshot-space-used snapshot-policy snapshot-count snapshot-reserve-available
-------- ------------ ---- ------ --------------- ------------------- --------------- -------------- --------------------------
vserver2 nfs_data_001 10GB 1.82MB none            1%                  default         11             507.2MB

cluster1::*> snapshot policy show -policy default
Vserver: cluster1
                         Number of Is
Policy Name              Schedules Enabled Comment
------------------------ --------- ------- ----------------------------------
default                          3 true    Default policy with hourly, daily & weekly schedules.
    Schedule               Count     Prefix                 SnapMirror Label
    ---------------------- -----     ---------------------- -------------------
    hourly                     6     hourly                 -
    daily                      2     daily                  daily
    weekly                     2     weekly                 weekly

Note: You might consider restricting your NFS clientmatch in your export policy rule to the subnet ofyour ESX hosts.

cluster1::*> export-policy rule show -vserver vserver2 -policyname default
             Policy          Rule    Access   Client                RO
Vserver      Name            Index   Protocol Match                 Rule
------------ --------------- ------  -------- --------------------- ---------
vserver2     default         1       any      0.0.0.0/0             any

cluster1::*> export-policy rule modify -vserver vserver2 -policyname default -ruleindex 1 -clientmatch 192.168.100.0/24

Hope that helps

/Matt

[Andrew7193]

Thank you for the reply Mike. 

It is not the aggregate:

NA1-FAS01::> aggr show -aggregate aggr01_DATA_na1_fas01a -fields size, usedsize
aggregate size usedsize
---------------------- ------- --------
aggr01_DATA_na1_fas01a 99.99TB 285.1GB

Our Snap shot policy is the default policy and that is barely filling the volume. I created two more NFS volumes and these same two machines failed to vMotion over to them as well. One of the new volumes had a restricted host list.

I created a new VM the same way these were created and it was able to move between all of the volumes. I believe this is moving more to the VM side of the house than the NetApp side.