Data Backup and Recovery

SnapCreator 3.6.0 & Lotus Notes 8.5.3

CENDRE_CEDRIC
6,272 Views

Hi all,

I try to use SnapCreator 3.6.0 with Lotus Domino 8.5.3FP3 on a Windows Server 2008R2. All software versions are 64bit.

My NetApp Storage is FAS3220 with DOT 8.1.2 7-Mode.

Everything works correctly until taking snapshot, I get an error on the "Application unquiesce."

Here is the error log from the server SnapCreator:

########## Application unquiesce ##########

76 [Thu Feb 21 09:32:46 2013] ERROR: 500 Status read failed: An existing connection was forcibly closed by the remote host. at /<C:\Program Files\NetApp\NetApp_Snap_Creator_Framework\scServer3.6.0\snapcreator.exe>SnapCreator/Agent/Remote.pm line 541

77

78 [Thu Feb 21 09:32:46 2013] [spmail004:9090(3.6.0.1)] ERROR: [scf-00054] Application unquiesce for plugin domino failed with exit code 1, Exiting!

79

80 ########## Application unquiesce ##########

 

Here is the error log ( event log) from the client SnapCreator:

Application Log:

A device or program has requested attention. Device or application: C:\Program Files\IBM\Lotus\Domino\nrunjava.EXE. Message title: ERROR.

A device or program has requested attention. Device or application: C:\Program Files\IBM\Lotus\Domino\nserver.exe. Message title: C:\Program Files\IBM\Lotus\Domino\nserver.exe.

System Log:

Application popup: Microsoft Visual C++ Debug Library : Debug Error!

Program: ...pp\NetApp_Snap_Creator_Framework\scAgent3.6.0\snapcreator.exe

Invalid allocation size: 18446744073709551610 bytes.

(Press Retry to debug the application)

Do you have any idea?

Thank you for your help.

Sincerely,

Cedric.

1 ACCEPTED SOLUTION

CENDRE_CEDRIC
6,272 Views

Hi everybody,

I solved this issue.


I just changed in my Lotus/Domino configuration, the Domino logical links ( link to folder ) by a simply Windows mount points.


And now, it works perfectly ( snapshot / unqiesce / ... ).


Thank you to all those who have helped me.

Best Regards,

Cedric.

View solution in original post

9 REPLIES 9

sivar
6,272 Views

could you please set this in your config file and try this again?

SC_AGENT_UNQUIESCE_TIMEOUT=1800

Looks like unquiescing takes long time.

Siva Ramanathan

Snap Creator Community Moderator

CENDRE_CEDRIC
6,272 Views

Hello Siva,

I just applied this parameter, and no change.

SnapCreator & Domino crash always after taking snapshot.

The Domino server is almost empty, there is only one mailbox.

62 ########## Creating snapshot(s) ##########

63

64 ########## SNAPSHOT CREATE COMMANDS ##########

65 [Mon Feb 25 09:18:58 2013] INFO: Running snapshot create command NTAP_SNAPSHOT_CREATE_CMD01 [sdcli snap create -s spmail003_lotus-daily_20130225091838 -D G: E: I: J: K: O: T:]

66 [Mon Feb 25 09:19:06 2013] INFO: Running snapshot create command [sdcli snap create -s spmail003_lotus-daily_20130225091838 -D G: E: I: J: K: O: T:] finished successfully

67

68 ########## SNAPSHOT CREATE COMMANDS FINISHED SUCCESSFULLY ##########

69

70 ########## PRE APPLICATION UNQUIESCE COMMANDS ##########

71 [Mon Feb 25 09:19:06 2013] INFO: No commands defined

72

73 ########## PRE APPLICATION UNQUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

74

75 ########## Application unquiesce ##########

76 [Mon Feb 25 09:19:44 2013] ERROR: 500 Server closed connection without sending any data back at /<C:\Program Files\NetApp\NetApp_Snap_Creator_Framework\scServer3.6.0\snapcreator.exe>SnapCreator/Agent/Remote.pm line 541

77

78 [Mon Feb 25 09:19:44 2013] [spmail003:9090(3.6.0.1)] ERROR: [scf-00054] Application unquiesce for plugin domino failed with exit code 1, Exiting!

79

80 ########## Application unquiesce ##########

And in the events logs :

Information 25/02/2013 09:19:47 Interactive Services detection 1000 None A device or program has requested attention. Device or application: C:\Program Files\IBM\Lotus\Domino\nserver.exe. Message title: C:\Program Files\IBM\Lotus\Domino\nserver.exe.
Information 25/02/2013 09:19:09 Interactive Services detection 1000 None A device or program has requested attention. Device or application: C:\Program Files\IBM\Lotus\Domino\nrunjava.EXE. Message title: ERROR.
Information 25/02/2013 09:19:06 Application Popup 26 None Application popup: Microsoft Visual C++ Debug Library : Debug Error!
Information 25/02/2013 09:19:05 SnapDrive 190 Snapshot Copy event Snapshot (spmail003_lotus-daily_20130225091838) of LUN(s) on storage system (spsda001) volume (LOTUS_SDA001_SPMAIL003_TRANSLOG) was successfully created.
Information 25/02/2013 09:19:05 SnapDrive 190 Snapshot Copy event Snapshot (spmail003_lotus-daily_20130225091838) of LUN(s) on storage system (spsda001) volume (LOTUS_SDA001_SPMAIL003_DAOS) was successfully created.
Information 25/02/2013 09:19:04 SnapDrive 190 Snapshot Copy event Snapshot (spmail003_lotus-daily_20130225091838) of LUN(s) on storage system (spsda001) volume (LOTUS_SDA001_SPMAIL003_MAIL3) was successfully created.
Information 25/02/2013 09:19:04 SnapDrive 190 Snapshot Copy event Snapshot (spmail003_lotus-daily_20130225091838) of LUN(s) on storage system (spsda001) volume (LOTUS_SDA001_SPMAIL003_MAIL2) was successfully created.
Information 25/02/2013 09:19:04 SnapDrive 190 Snapshot Copy event Snapshot (spmail003_lotus-daily_20130225091838) of LUN(s) on storage system (spsda001) volume (LOTUS_SDA001_SPMAIL003_MAIL1) was successfully created.
Information 25/02/2013 09:19:04 SnapDrive 190 Snapshot Copy event Snapshot (spmail003_lotus-daily_20130225091838) of LUN(s) on storage system (spsda001) volume (LOTUS_SDA001_SPMAIL003_CHANGEINFO) was successfully created.
Information 25/02/2013 09:19:03 SnapDrive 190 Snapshot Copy event Snapshot (spmail003_lotus-daily_20130225091838) of LUN(s) on storage system (spsda001) volume (LOTUS_SDA001_SPMAIL003_DATA) was successfully created.
Information 25/02/2013 09:19:00 SnapDrive 213 Snapshot Copy event SnapDrive is ready to create Snapshot copy (spmail003_lotus-daily_20130225091838) of LUN(s).

########## Creating snapshot(s) ##########

########## SNAPSHOT CREATE COMMANDS ##########
[Mon Feb 25 09:18:58 2013] INFO: Running snapshot create command NTAP_SNAPSHOT_CREATE_CMD01 [sdcli snap create -s spmail003_lotus-daily_20130225091838 -D G: E: I: J: K: O: T:]
[Mon Feb 25 09:18:58 2013] [spmail003:9090(3.6.0.1)] DEBUG: Executing command [sdcli snap create -s spmail003_lotus-daily_20130225091838 -D G: E: I: J: K: O: T:]
[Mon Feb 25 09:19:06 2013] [spmail003:9090(3.6.0.1)] TRACE: Command [sdcli snap create -s spmail003_lotus-daily_20130225091838 -D G: E: I: J: K: O: T:] finished with
exit code: [0]
stdout: [Initializing...SPMAIL003 : Checking policies
SPMAIL003 : Checking access control
SPMAIL003 : Preparing LUNs for Snapshot creation
SPMAIL003 : Ready to create snapshot copy
SPMAIL003 : Creating a Snapshot for the LUN
SPMAIL003 : Initiating snapmirror update for any source volumes.
The operation completed successfully.
]
stderr: []
[Mon Feb 25 09:19:06 2013] [spmail003:9090(3.6.0.1)] DEBUG: Command [sdcli snap create -s spmail003_lotus-daily_20130225091838 -D G: E: I: J: K: O: T:] finished successfully with message
[Initializing...SPMAIL003 : Checking policies
SPMAIL003 : Checking access control
SPMAIL003 : Preparing LUNs for Snapshot creation
SPMAIL003 : Ready to create snapshot copy
SPMAIL003 : Creating a Snapshot for the LUN
SPMAIL003 : Initiating snapmirror update for any source volumes.
The operation completed successfully.
]
[Mon Feb 25 09:19:06 2013] INFO: Running snapshot create command [sdcli snap create -s spmail003_lotus-daily_20130225091838 -D G: E: I: J: K: O: T:] finished successfully

########## SNAPSHOT CREATE COMMANDS FINISHED SUCCESSFULLY ##########

########## PRE APPLICATION UNQUIESCE COMMANDS ##########
[Mon Feb 25 09:19:06 2013] INFO: No commands defined

########## PRE APPLICATION UNQUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application unquiesce ##########
[Mon Feb 25 09:19:44 2013] ERROR: 500 Server closed connection without sending any data back at /<C:\Program Files\NetApp\NetApp_Snap_Creator_Framework\scServer3.6.0\snapcreator.exe>SnapCreator/Agent/Remote.pm line 541

[Mon Feb 25 09:19:44 2013] [spmail003:9090(3.6.0.1)] ERROR: [scf-00054] Application unquiesce for plugin domino failed with exit code 1, Exiting!

########## Application unquiesce ##########

Do you have another idea ?

Thanks for tour help,

Cedric.

sivar
6,272 Views

Please request a NetApp Global Support case for this issue.

This might need further troubleshooting from our engineers.

Also, is this a VMware implementation? Are there RDM LUNs involved?

CENDRE_CEDRIC
6,272 Views

Hello Siva,

There are 10 days ago, I opened a NetApp Case (P2 Level), and this time no return, from where my post on the community.

Absolutely, I use a type infrstructure Vmware ESXi 5.1 ( iSCSI datastores).

My Lotus-Domino Vms also use iSCSI luns via NetApp SnapDrive mode.

My Lotus-Domino is working in snapcreatoragent mode.

I have a master server to control the backup snapcreator for my 4 lotus-domino systems.

I followed the following Technicals Reports:

TR-3588  / TR-3756 / TR-3843 / TR-3883 / TR-3917 / TR-4009.

Best Regards,

Cedric.

sivar
6,272 Views

Thank you for the detail.

I am checking with my contacts.

1) Meanwhile, please supply me the Domino logs.

     C:\Program Files\IBM\Lotus\Domino\data\ibm_technical_support\*.log

2) What Antivirus solution is used in these servers? are there any antivirus logs that could shed some light on this issue?

Thanks,
Siva Ramanathan

sivar
6,272 Views

You can email me the logs here sivar at netapp.com

sivar
6,272 Views

Hello,

All my research points to the DAOS feature.

From the console.log, I found this information below.

07FC:0002-034C] 25/02/2013 15:32:00   Informational - The DAOS catalog is not synchronized. Deletions will be postponed. Please run 'tell daosmgr resync' at the next convenient opportunity to re-synchronize.

And also,

[0C8C:0009-0F6C] 25/02/2013 15:33:10   RM Evaluation: Started RMLogicalDisk

[0C8C:000A-0F88] 25/02/2013 15:33:11   Disk Spindle Information was not provided in the Directory

[0C8C:0009-0F6C] 25/02/2013 15:33:11   RM Evaluation: Started RMMemory

[0C8C:0009-0F6C] 25/02/2013 15:33:11   RM Evaluation: Started RMNetwork

[0C8C:0009-0F6C] 25/02/2013 15:33:11   RM Evaluation: Started RMProcessor

[0C8C:0009-0F6C] 25/02/2013 15:33:11   RM Evaluation: Initialization complete

[0C8C:0007-0CC8] 25/02/2013 15:35:31   ISpy: An error occurred getting ISpy's inbox view

[0C8C:0007-0CC8] 25/02/2013 15:35:31   ISpy: Mail Poller shutting down due to a fatal error.

I will reach out to my contacts on the "Disk Spindle information error", and keep you posted on the progress.

CENDRE_CEDRIC
6,273 Views

Hi everybody,

I solved this issue.


I just changed in my Lotus/Domino configuration, the Domino logical links ( link to folder ) by a simply Windows mount points.


And now, it works perfectly ( snapshot / unqiesce / ... ).


Thank you to all those who have helped me.

Best Regards,

Cedric.

PIYUSHBANSAL198722
5,576 Views

I am facing the same issue but using citrix xen plugin.

Could you please elaborate as to what exactly did you do to resolve the issue??

I am unable to understand the steps...

Public