SnapManager 5 SQL backup failing - Unexpected termination VDI Error

[tecaccount]

Hi, I upgraded from SnapManager 2.1.1 and Snapdrive 4.2.1 to SnapManager 5 and SnapDrive 6.0.1.

Does anyone have and idease what the cause may be, and how to fix.

Thanks, Peter

The first full backup error out with

SQL Server is prepared to freeze the database now [KDC-SQL1 : PowerDesigner]...
[08:03:09.060]  [KDC-SQL1 : Tertiary_Contact_Management_MSCRM] Unexpected termination: 0x80770004
[08:03:09.060]  [VDI Error]: An abort request is preventing anything except termination actions.

[08:03:09.060]  The preparation of one of more VDI database backups failed.
[08:03:09.060]  All database backups are aborted.
[08:03:09.060]  CLEANUP FAILED BACKUP

11:20:25.243]  VDI backup thread was informed to abort the backup of [KDC-SQL1 : WebWorkflow].

[11:20:25.258]  Error Code: 0xc00408d4
The VDI backup thread received a termination request as a result of one or more SQL database backups failing, or the SnapDrive failing to prepare for snapshot creation.

[11:20:25.258]  Error encountered
[11:20:25.258]  Msg 7694, SevLevel 16, State 1, SQLState 42000

[11:20:25.258]  [Microsoft][ODBC SQL Server Driver][SQL Server]Failed to pause full-text catalog for backup. Backup was aborted. Check to make sure Microsoft Full-text Service is running properly and SQL server service have access to full-text service.
[11:20:25.258]  Msg 3271, SevLevel 16, State 1, SQLState 42000occurred on file "NTAPDEV__KDC-SQL1_Tertiary_Contact_Management_MSCRM:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).
[11:20:25.258]  Msg 3013, SevLevel 16, State 1, SQLState 42000
==================================================

I then deinstallled and reinsatlled the previous versions, and still get an error,  the backups were working beforehand.

11:47:53.047]  [KDC-SQL1 : Tertiary_Contact_Management_MSCRM] Unexpected termination: 0x80770004
[11:47:53.047]  [VDI Error]: An abort request is preventing anything except termination actions.

[11:47:53.062]  The preparation of one of more VDI database backups failed.
[11:47:53.062]  All database backups are aborted.
[11:47:53.062]  CLEANUP FAILED BACKUP

[11:47:53.062]  Error encountered
[11:47:53.062]  Msg 7694, SevLevel 16, State 1, SQLState 42000

[11:47:53.062]  [Microsoft][ODBC SQL Server Driver][SQL Server]Failed to pause full-text catalog for backup. Backup was aborted. Check to make sure Microsoft Full-text Service is running properly and SQL server service have access to full-text service.
[11:47:53.062]  Msg 3271, SevLevel 16, State 1, SQLState 42000

[11:47:53.062]  [Microsoft][ODBC SQL Server Driver][SQL Server]A nonrecoverable I/O error occurred on file "NTAPDEV__KDC-SQL1_Tertiary_Contact_Management_MSCRM:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).
[11:47:53.078]  VDI backup thread was informed to abort the backup of [KDC-SQL1 : SmartBroker].
[11:47:53.078]  Msg 3013, SevLevel 16, State 1, SQLState 42000

[11:47:53.078]  [Microsoft][ODBC SQL Server Driver][SQL Server]BACKUP DATABASE is terminating abnormally.

[radek_kubka]

Hi,

Re SnapDrive update:

Is your host running Win 2003? Did you install all the required hot fixes? (here is the list: http://now.netapp.com/NOW/knowledge/docs/snapdrive/relsnap60/html/software/admin/provisioning/reference/r_sdw_prov_reqs-host.html#r_sdw_prov_reqs-host...

Regards,
Radek

[tecaccount]

Hi,

Yup - we're hotfixed to the max.

Turns out theres a max number of DB's per LUN. Nice.

Somewhere along the way we got to 150+ DB's (and possibly a limitation from one version of SM SQL to the latest) and the jobs just stopped running. As soon as we dropped the number of DB's backed up they started running again.

The Snaps still occur at the Volume level for all DB's but something fails due to this numerical limit (I'm not a DBA but it was either the DBCC check mounting the Snap for a verify or it couldn't handle the quiesce process).

By this stage the DBA's had got fed up by dealing with SM SQL and went back to trad transaction logging and db dumps - not very exciting but more reliable and less maintenance than dealing with limits and bugs in vendor software (and they were willing to live with the trade-off for 'instant' backup/restore).

Cheers,
Raj.

[dmauro]

not a vendor limitation:

http://support.microsoft.com/kb/943471

[dmauro]

An update to this old thread:

apart from the worker threads limit and the structural limitation of a SQL server to quiesce X number of Databases simultaneously (which does not depend on the backup client, but pretty much on the server's load),

here are the latest SMSQL limitations, some of which are not imposed by the NetApp software, but by the platform and application installed on the Windows OS:

Domenico.