ONTAP Discussions

Backup problem with SMVI 2.1 after Single File Restore...: "The disk signature of disk 1 is equal to the disk signature of disk 0"

neves_jacques
3,525 Views

My customer is running and using:

  • vSphere 4.10 build 60247;
  • Backup & Recovery VSC 2.1 (SMVI);
  • FAS2040HA with Ontap 7.3.6RC1;
  • About 20 VMs running into 2x NFS Datastores;
  • SMVI Backup is done on the Datastore level.

Everything was running correctly with Backup & Recovery (SMVI) version 2.1.

Last week, he did a Single File Restore (SFR) on a VM (running W2K8 R2) but it didn't get the drive instantly; he had to put the drive online via the Disk Management Utility in order to see the drive and to restore his file.

Since then, the backup for this VM is not working anymore but it works for other VMs in the same NFS Datastore.

Here bellow is the errors from vSphere + Event Viewer + hostd.log:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Cannot create a quiesced snapshot because the create snapshot operation exceeded the time limit  for holding off I/O in the frozen virtual machine.

The disk signature of disk 1 is equal to the disk signature of disk 0

[2011-07-05 11:19:59.804 F5786B90 warning 'App'] Can't convert IP Address of type 0

...

[2011-07-05 11:27:02.848 F5405B90 info 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx'] Upgrade is required for virtual machine, version: 4

...

[2011-07-05 11:28:25.564 F5446B90 verbose 'App'] Looking up object with name = "haTask-48-vim.vm.Snapshot.remove-571570" failed.

[2011-07-05 11:28:25.564 F5446B90 verbose 'App'] Looking up object with name = "haTask-160-vim.vm.Snapshot.remove-571569" failed.

[2011-07-05 11:28:25.564 F5446B90 verbose 'App'] Looking up object with name = "haTask-240-vim.vm.Snapshot.remove-571613" failed.

[2011-07-05 11:28:25.564 F5446B90 verbose 'App'] Looking up object with name = "haTask-64-vim.vm.Snapshot.remove-571571" failed.

[2011-07-05 11:28:25.565 F5446B90 verbose 'App'] Looking up object with name = "haTask-64-vim.vm.Snapshot.remove-571571" failed.

[2011-07-05 11:28:25.565 F5446B90 verbose 'App'] Looking up object with name = "haTask-48-vim.vm.Snapshot.remove-571570" failed.

[2011-07-05 11:28:25.565 F5446B90 verbose 'App'] Looking up object with name = "haTask-240-vim.vm.Snapshot.remove-571613" failed.

[2011-07-05 11:28:25.565 F5446B90 verbose 'App'] Looking up object with name = "haTask-160-vim.vm.Snapshot.remove-571569" failed.

...

[2011-07-05 11:34:42.825 F5704B90 error 'App' opID=d6d4cda1-84] Failed to get data ring: vmodl.fault.InvalidArgument

...

[2011-07-05 11:35:21.603 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Tools version status: ok

[2011-07-05 11:35:21.603 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] VMware Tools are current in guest: true

[2011-07-05 11:35:21.603 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Quiesced snapshot backup agent event: (vim.vm.BackupEventInfo) {

dynamicType = <unset>,

eventType = "requestorError",

code = 3,

message = " Error when notifying the sync provider .",

}

[2011-07-05 11:35:21.603 F5B74B90 error 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Quiesced snapshot backup agent error: Error when notifying the sync provider. (3)

[2011-07-05 11:35:21.604 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Tools version status: ok

[2011-07-05 11:35:21.604 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] VMware Tools are current in guest: true

[2011-07-05 11:35:21.604 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Quiesced snapshot backup agent event: (vim.vm.BackupEventInfo) {

dynamicType = <unset>,

eventType = "providerAbort",

code = 3,

message = "Snapshot operation aborted",

}

[2011-07-05 11:35:21.604 F5B74B90 error 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Quiesced snapshot backup agent error: Snapshot operation aborted (3)

[2011-07-05 11:35:21.604 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Tools version status: ok

[2011-07-05 11:35:21.604 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] VMware Tools are current in guest: true

[2011-07-05 11:35:21.635 F57EBB90 info 'DictionaryOptionsProvider(Config.GlobalSettings.)' opID=d6366ed8-32] Set called with key 'Config.GlobalSettings.guest.commands.sharedPolicyRefCount' value '1'

[2011-07-05 11:35:21.674 F57EBB90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux23/cflux23.vmx' opID=d6366ed8-32] Guest login successful.

[2011-07-05 11:35:22.014 F6469900 verbose 'NetConfigProvider'] FetchFn: List of pnics opted out

[2011-07-05 11:35:22.035 F57EBB90 info 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux23/cflux23.vmx' opID=d6366ed8-32] File copied successfully.

[2011-07-05 11:35:22.131 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Quiesced snapshot backup agent event: (vim.vm.BackupEventInfo) {

dynamicType = <unset>,

eventType = "requestorError",

code = 5,

message = "Asynchronous operation failed: VssSyncStart",

}

[2011-07-05 11:35:22.131 F5B74B90 error 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Quiesced snapshot backup agent error: Asynchronous operation failed: VssSyncStart (5)

[2011-07-05 11:35:22.131 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] Tools version status: ok

[2011-07-05 11:35:22.131 F5B74B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux25/cflux25.vmx' opID=bf2bde51-9f] VMware Tools are current in guest: true

[2011-07-05 11:35:22.654 F5786B90 verbose 'vm:/vmfs/volumes/78998d28-bd4c205e/cflux26/cflux26.vmx' opID=cd4a8b58-89] Quiesced snapshot backup agent event: (vim.vm.BackupEventInfo) {

dynamicType = <unset>,

eventType = "keepAlive",

code = 0,

message = "",

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Any ideas?

Thanks in advance.

JNV

5 REPLIES 5

DANBROWNCHUFT
3,525 Views

Hi,

We have a very similar if not identical problem with a 2K8R2 VM that had an SFR session run on it, where the disks had to be brought online manually. And now it won't backup due to VM snapshot failure. with very similar errors to yours.

Did you ever get a resolution to it?

yannick_n
3,525 Views

Hi,

We have exactely the same issue after a SFR session and we have the same configuration (2k8r2 & VSC 2.1.1).

The only way to backup again the vm is to clone it before...

Have you solved the problem ?

Thank you in advance.

DANBROWNCHUFT
3,525 Views

We removed the VSS Driver from the VMWare tools install on the VM.

The VM now backs up ok with SMVI. Not ideal removing VSS, but it is a workaround for now.

The SFR session seems to corrupt the VSS Driver within the VM, confusing the Disk signatures...

yannick_n
3,525 Views

You can also place the disk.enable.uuid to false...(and the advanced settings from the virtual machine) to temporarly solve the problem and don't use the quiescing (but you will be file level consistant) (insteas of removing the vss driver from the vmware tools)

I have choose an other solution for the moment :

Keep the VSS driver from the vmware tools and if SFR needed, mount the disk on an other virtual machine.

So you are vss protected...

If you have the problem on some other virtual machine where you do an SFR session just clone the virtual machine to solve the problem and after this always use SFR on an other VM.

We are thinking about having a VM only for SFR sessions use.

rhideg_linear
3,525 Views

Hi,

I have found out another workaround (this applies for Windows 2008 R2):

0. Shut down the VM and make a backup (for example a VMware Snapshot without quescieing)


1. You have to change the uuid of the VMs disk(s), that was previously attached by SFR and SMVI:

C:\Windows\system32>diskpart     //start the diskpart utility

DISKPART> select disk 0     //select the problematic disk, that was previously attached by SFR agent

Disk 0 is now the selected disk.

DISKPART> detail disk     //view the actual disk uuid

VMware Virtual disk SCSI Disk Device

Disk ID: 77AC14B9

Type   : SAS

Status : Online

Path   : 0

Target : 0

LUN ID : 0

Location Path : PCIROOT(0)#PCI(1500)#PCI(0000)#SAS(P00T00L00)

Current Read-only State : No

Read-only  : No

Boot Disk  : Yes

Pagefile Disk  : Yes

Hibernation File Disk  : No

Crashdump Disk  : Yes

Clustered Disk  : No

  Volume ###  Ltr  Label        Fs     Type        Size     Status     Info

  ----------  ---  -----------  -----  ----------  -------  ---------  --------

  Volume 1         System Rese  NTFS   Partition    100 MB  Healthy    System

  Volume 2     C                NTFS   Partition     39 GB  Healthy    Boot

DISKPART> uniqueid disk id=77AC14B5     //modify the uuid of disk 0

2. Repair the Windows:

Restart your Windows -> Won't boot because of the uuid change -> You will need to boot with Windows 2008 R2 Installation Media -> Recovery Mode

Based on http://support.microsoft.com/kb/927392 you have to run the following commands in the command line:

Bootrec.exe /fixmbr

Bootrec.exe /fixboot

Bootrec.exe /scanos

Bootrec.exe /rebuildbcd

(Probably not all of the commands above are required.)

Public