VMware NFS Datastore lost from config during Snapshot

vMarkus_K · ‎2012-12-11

Hi,

I`m using SC 3.6.

I have 2 VMware DS Configured (iSCSI and NFS), during the Snapshot process the NFS store is removed from Konfig.

LOG Lines:

########## Parsing Environment Parameters ##########

[Tue Dec 11 12:38:58 2012] DEBUG: Parsing VOLUMES - controller: 10.20.70.40 volume: vol_rdm

[Tue Dec 11 12:38:58 2012] DEBUG: Parsing VOLUMES - controller: 10.20.70.40 volume: vol_vmstore_foobar

[Tue Dec 11 12:38:58 2012] DEBUG: Parsing NTAP_USERS - controller: 10.20.70.40 user: root

[Tue Dec 11 12:38:58 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: daily retention: 7

[Tue Dec 11 12:38:58 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: weekly retention: 2

[Tue Dec 11 12:38:58 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: monthly retention: 1

.

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Searching for Datastore hekate_vmstore00_nfs ...

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Requested Datastore (hekate_vmstore00_nfs) is available.

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] INFO: Collecting VM information on NFS Datastore hekate_vmstore00_nfs.

.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Searching for Datastore vol_rdm ...

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Requested Datastore (vol_rdm) is available.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Collecting LUN information ...

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Already collected LUN serial numbers for storage controller/vserver 10.20.70.40 ...

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Searching for matching disk name naa.60a9800032466651762b42504a346249 ...

[Tue Dec 11 12:39:31 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Checking host system frederike.foobar.lan ...

[Tue Dec 11 12:39:31 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Name = naa.60a9800032466651762b42504a346249, UUID = 32466651762b42504a346249.

[Tue Dec 11 12:39:31 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Found LUN with UUID 32466651762b42504a346249, host 10.20.70.40, path /vol/vol_rdm/rdm/rdm.lun.

[Tue Dec 11 12:39:31 2012] [10.20.70.6:9090(3.6.0.1)] INFO: Collecting VM information on VMFS Datastore vol_rdm.

.

[Tue Dec 11 12:39:38 2012] INFO: ENV Parameter VOLUMES set in plugin vibe

[Tue Dec 11 12:39:38 2012] DEBUG: Old value for VOLUMES is 10.20.70.40:vol_rdm,vol_vmstore_foobar

[Tue Dec 11 12:39:38 2012] DEBUG: New value for VOLUMES is 10.20.70.40:vol_rdm

[Tue Dec 11 12:39:38 2012] INFO: Config File C:/Program Files/NetApp/NetApp_Snap_Creator_Framework/scServer3.6.0/configs/VMware/vcenter.conf successfully updated.

.

########## Parsing Environment Parameters ##########

[Tue Dec 11 12:39:40 2012] DEBUG: Parsing VOLUMES - controller: 10.20.70.40 volume: vol_rdm

[Tue Dec 11 12:39:40 2012] DEBUG: Parsing NTAP_USERS - controller: 10.20.70.40 user: root

[Tue Dec 11 12:39:40 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: daily retention: 7

[Tue Dec 11 12:39:40 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: weekly retention: 2

[Tue Dec 11 12:39:40 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: monthly retention: 1

Config Lines:

VOLUMES=10.20.70.40:vol_rdm,vol_vmstore_foobar

.

VIBE_VSPHERE_NAMES=vcenter:hekate_vmstore00_nfs,vol_rdm:

After a Snapshot run line "VOLUMES changes in:

VOLUMES=10.20.70.40:vol_rdm

NFS mount looks like this:

hekate_vmstore00_nfs

172.16.15.40:/vmstore00

NFS

Nachricht wurde geändert durch: Markus Kraus

I tried the situation in a second Environment, it`s the same problem:

[Tue Dec 11 15:19:12 2012] [10.8.15.202:9090(3.6.0.1)] DEBUG: Setting VOLUMES entry for 10.8.15.230 to vol_vmware_02.

[Tue Dec 11 15:19:12 2012] [10.8.15.202:9090(3.6.0.1)] DEBUG: Expanding VOLUMES entry for 10.8.15.230 to vol_vmware_02,vol_vmware_01.

[Tue Dec 11 15:19:12 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Setting VOLUMES configuration variable for Snap Creator to "10.8.15.230:vol_vmware_02,vol_vmware_01".

[Tue Dec 11 15:19:12 2012] INFO: ENV Parameter VOLUMES set in plugin vibe

[Tue Dec 11 15:19:12 2012] DEBUG: Old value for VOLUMES is 10.8.15.230:vol_vmware_01,vol_vmware_02,vol_vmware_03

[Tue Dec 11 15:19:12 2012] DEBUG: New value for VOLUMES is 10.8.15.230:vol_vmware_02,vol_vmware_01

[Tue Dec 11 15:19:12 2012] INFO: Config File C:/Program Files/NetApp/NetApp_Snap_Creator_Framework/scServer3.6.0/configs/vCenter/root-lab_vcs.conf successfully updated

########## Parsing Environment Parameters ##########

[Tue Dec 11 15:19:12 2012] DEBUG: Parsing VOLUMES - controller: 10.8.15.230 volume: vol_vmware_02

[Tue Dec 11 15:19:12 2012] DEBUG: Parsing VOLUMES - controller: 10.8.15.230 volume: vol_vmware_01

[Tue Dec 11 15:19:12 2012] DEBUG: Parsing NTAP_USERS - controller: 10.8.15.230 user: root

[Tue Dec 11 15:19:12 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: daily retention: 7

is it maybe a problem with the export:

root-lab-fas2050> exportfs

/vmware_03 -actual=/vol/vol_vmware_03/vmware_03,sec=sys,rw=172.16.50.201,root=172.16.50.201

Arora_Kapil · ‎2012-12-11

Hi,

This might be a dumb question, but I have to ask.

Do you have any VMs inside hekate_vmstore00_nfs datastore?

Kapil

vMarkus_K · ‎2012-12-11

HI,

yes all VMs are running in this DS.

See Log:

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Searching for Datastore hekate_vmstore00_nfs ...

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Requested Datastore (hekate_vmstore00_nfs) is available.

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] INFO: Collecting VM information on NFS Datastore hekate_vmstore00_nfs.

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine list on Datastore hekate_vmstore00_nfs.

[Tue Dec 11 12:39:20 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for valign.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for sv-test-win2003.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for sv-test-nagios-sat.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for snare.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for nagios_monitor.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for hermes_neu.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for cacti.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for nos-collector.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for nos-monitor.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for helena.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for freya.

[Tue Dec 11 12:39:21 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for hermes.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for hera.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for hades.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for OMD.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for echidna.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for eris.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for vCenter.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for horkos.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for mooh.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for doh-dc-00.

[Tue Dec 11 12:39:22 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for doh-exc-00.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for nagios_test.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for sf-arc-01.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for PHDVBA.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for sf-adc-01.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for manfred.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for sf-exc-01.

[Tue Dec 11 12:39:23 2012] [10.20.70.6:9090(3.6.0.1)] DEBUG: Saving virtual machine information for PJ_Exchange_Test.

vMarkus_K · ‎2012-12-11

Update 1:

if I change export to this (remove qtree and use original path as export path):

root-lab-fas2050> exportfs

/vol/vol_vmware_03 -sec=sys,rw=172.16.50.201,root=172.16.50.201

then everything works fine.

and also when i use a qtree everything is fine:

root-lab-fas2050>

root-lab-fas2050> exportfs

/vol/vol_vmware_03/qtree_vmware_03 -sec=sys,rw=172.16.50.201,root=172.16.50.201

semms to be a problem with -actual option

root-lab-fas2050> exportfs

/vmware_03 -actual=/vol/vol_vmware_03/vmware_03,sec=sys,rw=172.16.50.201,root=172.16.50.201

ktenzer · ‎2012-12-11

I think this is a bug on our end, we do not expect the "-actual" so I would export without this option if you want to use SC.

SC has to figure out from the storage system what volumes are involved since we cant do it from ESX. For NFS this involves looking at exports.

Kapil can you bring this up to Matt and have engineering look at this and open a bug?

Regards,

Keith

ktenzer · ‎2012-12-11

Oh another option is, forgot

You can turn off SC discovery in vmware plugin...VIBE_DISCOVERY_something is the option. If you set this to N then the config file will never be updated and it should work with -actual. Only problem is then we arent detecting if you move a VM to a new datastore so you would have to update config manually and add new datastores / storage systems when they are added.

Regards,

Keith

vMarkus_K · ‎2012-12-11

OK,

Disabling SC discovery is an option. I will try it.

I think there is only a manual add of new Volumes and Datastore / Share in the config necessary, is that right?

Moving VMs between know resources shouldn’t be a problem?

But the best way is to fix this issue, a lot of people use this export-way...

rmatt · ‎2012-12-23

If you want to set all the volumes into "manual" state, set:

VIBE_DYNAMIC_VOLUMES_UPDATE=N

This will prevent the update of VOLUMES, SNAPMIRROR_VOLUMES, SNAPVAULT_VOLUMES and NTAP_DFM_DATA_SET. If you want to set those values manually at that point, you can. This will help you if you still intend to use RDMs as the vSphere/vCloud plug-in doesn't support RDMs directly. The risk element is that if you move the VM from one Datastore (backed by a volume) to another and that isn't in your VOLUMES parameter, it will be quiesced/unquiesced but not be backed up from a NetApp snapshot standpoint.

Thanks,

--Matt