XenServer Restore Issue

vMarkus_K · ‎2012-12-12

Hi,

I`am running a test XenServer 6.0.2 witt 2 VMs:

CentOS

W2K9R2

SC 3.6.0.1 is uses on Windows 2008 R2

Backup seems to be working Fine (without error). But no restore is possible.

For me it semms to be a metadata Problem.

NFS Tree, including snapshot:

[root@localhost mnt]# tree -a

.

+-- 8a530fff-0047-ddde-ebe0-d827c131705e

¦ +-- aea97fc2-1856-4b51-a47e-c12e35ac7c47.vhd

¦ +-- f2aa92af-6f72-4af9-baab-4b1cc10f11af.vhd

¦ +-- filelog.txt

+-- fd9e028f-980c-d33b-7247-c71c4709d893

¦ +-- 02009757-f4a3-4f2a-84f1-e45adbecd9df.vhd

¦ +-- 048e5375-50ae-478b-b1e2-e950df272ee4.vhd

¦ +-- d3da9d9a-c664-46cd-bc8b-25a265bf1bf8.vhd

¦ +-- e4fb5af6-a172-4700-beff-bc8da16dc2d0.vhd

¦ +-- filelog.txt

+-- .snapshot

+-- SC-daily_20121212121227

+-- 8a530fff-0047-ddde-ebe0-d827c131705e

¦ +-- aea97fc2-1856-4b51-a47e-c12e35ac7c47.vhd

¦ +-- f2aa92af-6f72-4af9-baab-4b1cc10f11af.vhd

¦ +-- filelog.txt

+-- fd9e028f-980c-d33b-7247-c71c4709d893

+-- 02009757-f4a3-4f2a-84f1-e45adbecd9df.vhd

+-- 048e5375-50ae-478b-b1e2-e950df272ee4.vhd

+-- 6cd5a5e4-f043-464f-9f28-e24f1f8bebe4.vhd

+-- 7399c394-faf8-4a8a-b8fa-9b47fc60b8fc.vhd

+-- 767720fe-9d7b-4178-8344-4c5bf6fbc895.vhd

+-- b2ca4229-d599-4acc-a61b-d9bb9beb87f9.vhd

+-- d3da9d9a-c664-46cd-bc8b-25a265bf1bf8.vhd

+-- e4fb5af6-a172-4700-beff-bc8da16dc2d0.vhd

+-- filelog.txt

6 directories, 20 files

W2K8 Metadata&colon;

02009757-f4a3-4f2a-84f1-e45adbecd9df;fd9e028f-980c-d33b-7247-c71c4709d893;nfs;172.16.50.230:/vol/vol_xenserver_00

b2ca4229-d599-4acc-a61b-d9bb9beb87f9;fd9e028f-980c-d33b-7247-c71c4709d893;nfs;172.16.50.230:/vol/vol_xenserver_00

7399c394-faf8-4a8a-b8fa-9b47fc60b8fc;fd9e028f-980c-d33b-7247-c71c4709d893;nfs;172.16.50.230:/vol/vol_xenserver_00

CentOS Metadata&colon;

e4fb5af6-a172-4700-beff-bc8da16dc2d0;fd9e028f-980c-d33b-7247-c71c4709d893;nfs;172.16.50.230:/vol/vol_xenserver_00

767720fe-9d7b-4178-8344-4c5bf6fbc895;fd9e028f-980c-d33b-7247-c71c4709d893;nfs;172.16.50.230:/vol/vol_xenserver_00

6cd5a5e4-f043-464f-9f28-e24f1f8bebe4;fd9e028f-980c-d33b-7247-c71c4709d893;nfs;172.16.50.230:/vol/vol_xenserver_00

Restore LOG:

[Wed Dec 12 11:56:41 2012] INFO: Logfile timestamp: 20121212115641

[Wed Dec 12 11:56:41 2012] DEBUG: GMT - Wed Dec 12 10:56:41 2012

[Wed Dec 12 11:56:41 2012] DEBUG: Version: NetApp Snap Creator Framework 3.6.0

[Wed Dec 12 11:56:41 2012] DEBUG: Profile: vCenter

[Wed Dec 12 11:56:41 2012] DEBUG: Config Type: STANDARD

[Wed Dec 12 11:56:41 2012] DEBUG: Action: restore

[Wed Dec 12 11:56:41 2012] DEBUG: Application Plugin: xen

[Wed Dec 12 11:56:41 2012] DEBUG: File System Plugin: null

[Wed Dec 12 11:56:41 2012] DEBUG: Policy: daily

[Wed Dec 12 11:56:41 2012] DEBUG: Snapshot Name: SC-daily_20121212110920

[Wed Dec 12 11:56:41 2012] INFO: Plugin validation for xen completed successfully

########## Parsing Environment Parameters ##########

[Wed Dec 12 11:56:41 2012] DEBUG: Parsing VOLUMES - controller: 10.8.15.230 volume: vol_xenserver_00

[Wed Dec 12 11:56:41 2012] DEBUG: Parsing NTAP_USERS - controller: 10.8.15.230 user: root

[Wed Dec 12 11:56:41 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: daily retention: 3

########## Restore handling for plugin xen ##########

[Wed Dec 12 11:56:41 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Set environment variables successfully in xen plug-in

[Wed Dec 12 11:56:41 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Performing restore operation in xen plug-in

[Wed Dec 12 11:56:41 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Creating executor for the filer 10.8.15.230

[Wed Dec 12 11:56:41 2012] [10.8.15.202:9090(3.6.0.1)] DEBUG: Parsing NTAP_USERS - filer: 10.8.15.230 user: root

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Creating XEN RPC session on host : 10.8.15.206

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Getting the host instance and record information from host 10.8.15.206

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Found host with address 10.8.15.206

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Collecting host record information 10.8.15.206

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Host discovery completed successfully. Host address 10.8.15.206 with label xenserver-ipmeuedr

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Validating metadata file D:\XenServer_meta\xenserver-ipmeuedr_W2K8R2_SC-daily_20121212110920.metadata

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Reading the metadataFile D:\XenServer_meta\xenserver-ipmeuedr_W2K8R2_SC-daily_20121212110920.metadata

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Validating snapshot

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Checking VDI 8a530fff-0047-ddde-ebe0-d827c131705e/aea97fc2-1856-4b51-a47e-c12e35ac7c47 in snapshot 172.16.50.230:/vol/vol_xenserver_00/SC-daily_20121212110920

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: [xen-00031] Not a valid snapshot. Not able to find aea97fc2-1856-4b51-a47e-c12e35ac7c47.vhd in snapshot SC-daily_20121212110920

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: [xen-00032] Metadata file and snaphost information is not matching. W2K8R2

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: Restoring the VMs failed :10.8.15.206

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: Restore operation failed for the host 10.8.15.206

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: [scf-00006] Restore handling of plugin xen failed with exit code 1, Exiting!

[Wed Dec 12 11:56:44 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Set environment variables successfully in xen plug-in

[Wed Dec 12 11:56:44 2012] DEBUG: Application specific restore_cleanup operations are not yet implemented for plugin xen

[Wed Dec 12 11:56:44 2012] DEBUG: Exiting with error code - 1

Nachricht wurde geändert durch: Markus Kraus Add debug log from snapshot

Nachricht wurde geändert durch: Markus Kraus Add metadata File

Nachricht wurde geändert durch: Markus Kraus add Restore Log

ktenzer · ‎2012-12-12

Are you using qtrees? In SC 3.6 XEN plugin wont work with qtrees, this will be fixed in next release of SC 4.0

Keith

charly · ‎2012-12-12

Yes agree with Keith. If SR is in qtree, restore won't work and same is fixed in SC4.0. Would be great if could send the xenserver-ipmeuedr_W2K8R2_SC-daily_20121212110920.metadata file.

Thanks

vMarkus_K · ‎2012-12-13

Hi,

I´am not using qtrees in this Test Enviroment.

charly · ‎2012-12-13

In that case, can you send us the metadata file. <name>.metadata ..?

vMarkus_K · ‎2012-12-13

metadata file is already attatched.

charly · ‎2012-12-13

sorry didn't notice that. Can you give the latest restore log also..? Please try to restore the snapshot SC-daily_20121212161024

vMarkus_K · ‎2012-12-13

Log is now attatched:

[Wed Dec 12 16:13:42 2012] INFO: Logfile timestamp: 20121212161342

[Wed Dec 12 16:13:42 2012] DEBUG: GMT - Wed Dec 12 15:13:42 2012

[Wed Dec 12 16:13:42 2012] DEBUG: Version: NetApp Snap Creator Framework 3.6.0

[Wed Dec 12 16:13:42 2012] DEBUG: Profile: vCenter

[Wed Dec 12 16:13:42 2012] DEBUG: Config Type: STANDARD

[Wed Dec 12 16:13:42 2012] DEBUG: Action: restore

[Wed Dec 12 16:13:42 2012] DEBUG: Application Plugin: xen

[Wed Dec 12 16:13:42 2012] DEBUG: File System Plugin: null

[Wed Dec 12 16:13:42 2012] DEBUG: Policy: daily

[Wed Dec 12 16:13:42 2012] DEBUG: Snapshot Name: SC-daily_20121212161024

[Wed Dec 12 16:13:42 2012] INFO: Plugin validation for xen completed successfully

########## Parsing Environment Parameters ##########

[Wed Dec 12 16:13:42 2012] DEBUG: Parsing VOLUMES - controller: 10.8.15.230 volume: vol_xenserver_00

[Wed Dec 12 16:13:42 2012] DEBUG: Parsing NTAP_USERS - controller: 10.8.15.230 user: root

[Wed Dec 12 16:13:42 2012] DEBUG: Parsing NTAP_SNAPSHOT_RETENTIONS - policy: daily retention: 3

########## Restore handling for plugin xen ##########

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Set environment variables successfully in xen plug-in

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Performing restore operation in xen plug-in

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Creating executor for the filer 10.8.15.230

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] DEBUG: Parsing NTAP_USERS - filer: 10.8.15.230 user: root

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Creating XEN RPC session on host : 10.8.15.206

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Getting the host instance and record information from host 10.8.15.206

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Found host with address 10.8.15.206

[Wed Dec 12 16:13:42 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Collecting host record information 10.8.15.206

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Host discovery completed successfully. Host address 10.8.15.206 with label xenserver-ipmeuedr

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Validating metadata file D:\XenServer_meta\xenserver-ipmeuedr_CentOS-neu_SC-daily_20121212161024.metadata

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Reading the metadataFile D:\XenServer_meta\xenserver-ipmeuedr_CentOS-neu_SC-daily_20121212161024.metadata

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Validating snapshot

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Checking VDI fd9e028f-980c-d33b-7247-c71c4709d893/e78f0a64-c723-4c56-908b-6a4492777db4 in snapshot 172.16.50.230:/vol/vol_xenserver_00/SC-daily_20121212161024

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: [xen-00031] Not a valid snapshot. Not able to find e78f0a64-c723-4c56-908b-6a4492777db4.vhd in snapshot SC-daily_20121212161024

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: [xen-00032] Metadata file and snaphost information is not matching. CentOS-neu

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: Restoring the VMs failed :10.8.15.206

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: Restore operation failed for the host 10.8.15.206

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] ERROR: [scf-00006] Restore handling of plugin xen failed with exit code 1, Exiting!

[Wed Dec 12 16:13:43 2012] [10.8.15.202:9090(3.6.0.1)] INFO: Set environment variables successfully in xen plug-in

[Wed Dec 12 16:13:43 2012] DEBUG: Application specific restore_cleanup operations are not yet implemented for plugin xen

[Wed Dec 12 16:13:43 2012] DEBUG: Exiting with error code - 1

charly · ‎2012-12-13

So the VM is having two disks right..?

vMarkus_K · ‎2012-12-13

No,

VM has only one disk.

But aufter Snapshot creation and removal I can see 2 vhd files per VM (2 VMs) in the original location.

[root@localhost /]# cd /mnt/fd9e028f-980c-d33b-7247-c71c4709d893/

[root@localhost fd9e028f-980c-d33b-7247-c71c4709d893]# ll

total 8731364

-rw-r--r--. 1 root root 981299712 Aug 19 2007 402b2cb9-6c71-439d-a3f6-2be21e9f6056.vhd

-rw-r--r--. 1 root root 1088638976 Aug 20 2007 8187ed66-8932-4112-afa7-3ffa23e7f6f9.vhd

-rw-r--r--. 1 root root 7480492544 Aug 19 2007 ae0e95a3-e62f-4596-a22d-0e458df4cabd.vhd

-rw-r--r--. 1 root root 59092992 Aug 20 2007 e78f0a64-c723-4c56-908b-6a4492777db4.vhd

-rw-r--r--. 1 root root 8856 Aug 19 2007 filelog.txt

[root@localhost SC-daily_20121212161024]# cd /mnt/.snapshot/SC-daily_20121212161024/fd9e028f-980c-d33b-7247-c71c4709d893/

[root@localhost fd9e028f-980c-d33b-7247-c71c4709d893]# ll

total 8207324

-rw-r--r--. 1 root root 981299712 Aug 19 2007 402b2cb9-6c71-439d-a3f6-2be21e9f6056.vhd

-rw-r--r--. 1 root root 20992 Aug 19 2007 6bd0e5bf-f53f-4843-8bc2-dc4ecc52c062.vhd

-rw-r--r--. 1 root root 4526080 Aug 19 2007 8187ed66-8932-4112-afa7-3ffa23e7f6f9.vhd

-rw-r--r--. 1 root root 7480492544 Aug 19 2007 ae0e95a3-e62f-4596-a22d-0e458df4cabd.vhd

-rw-r--r--. 1 root root 54784 Aug 19 2007 b5f78abb-061d-47e5-aa27-e1bf0ee6401c.vhd

-rw-r--r--. 1 root root 20992 Aug 19 2007 e78f0a64-c723-4c56-908b-6a4492777db4.vhd

-rw-r--r--. 1 root root 8856 Aug 19 2007 filelog.txt

[root@localhost fd9e028f-980c-d33b-7247-c71c4709d893]#

XenServer Output:

[root@xenserver ~]# xe vdi-list name-label="W2K8R2-neu 0"

uuid ( RO) : 8187ed66-8932-4112-afa7-3ffa23e7f6f9

name-label ( RW): W2K8R2-neu 0

name-description ( RW): Created by template provisioner

sr-uuid ( RO): fd9e028f-980c-d33b-7247-c71c4709d893

virtual-size ( RO): 25769803776

sharable ( RO): false

read-only ( RO): false

charly · ‎2012-12-19

Tried to reproduce this in our environment but not happening. The issue here is vdi-list API is giving two disks but actually only one disk present. This might be because some stale data. Is it possible to use your environment for some testing..?

You have two VMs here and both the VM restore behaves the same way..?