Subscribe

Snap Creator causes domino to crash

[ Edited ]

Hi,

We installed Snap Creator with the domino plug-in at a customer. When we launch a backup with Snap Creator the domino crashes. The domino service crashes, the server stays online. After starting the service or server, everything is working again.

Yesterday the production domino server crashed.

The version of Snap Creator is 3.5 (both server and agent). The domino version is 8.5.1 and is running on Windows 2003 SP2.

Part of the domino log file:

<@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@>
Section: System Data -> Notes Process Summary (Time 11:10:57)
<@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@>

<@@ ------ Instances -> Instance1 (current) (Time 11:10:57) ------ @@>

                Ini File   : E:\notes\notes.ini
                Data Dir   : G:\Notes\Data
                Prog Dir   : e:\Notes
                Notes Build: Release 8.5.1 September 28, 2009            
            Proc List:
                  e:\Notes\nservice.exe (10c4)
                  e:\Notes\nSERVER.EXE (166c)
                  e:\Notes\nlogasio.EXE (c2c)
                  e:\Notes\nevent.EXE (1570)
                  e:\Notes\nReplica.EXE (950)
                  e:\Notes\nReplica.EXE (13d4)
                  e:\Notes\nRouter.EXE (f10)
                  e:\Notes\nUpdate.EXE (d64)
                  e:\Notes\nAMgr.EXE (16a0)
                  e:\Notes\nAdminp.EXE (11e0)
                  e:\Notes\nSched.EXE (1504)
                  e:\Notes\nCalConn.EXE (690)
                  e:\Notes\nRnRMgr.EXE (13e0)
                  e:\Notes\nSMTP.EXE (14b0)
                  e:\Notes\nDAOSMgr.EXE (12c4)
                  e:\Notes\nprocmon.EXE (e70)
                  e:\Notes\nAMgr.EXE (28c)
                  C:\Program Files\Netapp\NetApp_Snap_Creator_Framework\scAgent3.5.0\snapcreator.exe (19c)
                  e:\Notes\nrunjava.EXE (14bc)

<@@ ------ System Data -> Memory Usage (Time 11:10:57) ------ @@>

Total Physical Memory:  4.0G
Avail Physical Memory:  2.7G
Memory Usage        : 32%
Total Paging File   :  5.8G
Avail Paging File   :  4.3G
Total Virtual Memory:  2.0G
Avail Virtual Memory:  2.0G
Avail Extended Virtual Memory:  0.0K

<@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@>
Section: Notes Process Info (Time 11:10:57)
<@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@>

<@@ ------ Notes Process Info -> Load Modules for Process :: [snapcreator:  019c] (Time 11:10:58) ------ @@>

Base               End                      Size Module

0x00400000         0x009ce000            6086656 C:\Program Files\Netapp\NetApp_Snap_Creator_Framework\scAgent3.5.0\snapcreator.exe
0x7c800000         0x7c8c3000             798720 C:\WINDOWS\system32\ntdll.dll
0x77e40000         0x77f42000            1056768 C:\WINDOWS\system32\kernel32.dll
0x77420000         0x77523000            1060864 C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.3790.4770_x-ww_05FDF087\comctl32.dll
0x77ba0000         0x77bfa000             368640 C:\WINDOWS\system32\msvcrt.dll
0x7d1e0000         0x7d27c000             638976 C:\WINDOWS\system32\ADVAPI32.dll
0x77c50000         0x77cf0000             655360 C:\WINDOWS\system32\RPCRT4.dll
0x76f50000         0x76f63000              77824 C:\WINDOWS\system32\Secur32.dll
0x77c00000         0x77c49000             299008 C:\WINDOWS\system32\GDI32.dll
0x77380000         0x77411000             593920 C:\WINDOWS\system32\USER32.dll
0x7d180000         0x7d1d2000             335872 C:\WINDOWS\system32\SHLWAPI.dll
0x28000000         0x280e1000             921600 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\8c729c2aff65d775b132c7d49d29d544\perl512.dll
0x10000000         0x10006000              24576 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\ec28c347558976b3b00f890635a1ad1e\Fcntl.dll
0x01480000         0x01492000              73728 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\1071d1669e7950fc8a95b09a16ab1013\POSIX.dll
0x003f0000         0x003f6000              24576 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\2191a57c579f82b74fb0e7614766c1b8\MD5.dll
0x018a0000         0x018a6000              24576 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\e88ba92f4177d66ab9142140ed7f5511\DES.dll
0x018c0000         0x018c7000              28672 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\38a53dc3672f2020c54be8640078fee5\ReadKey.dll
0x018d0000         0x018d5000              20480 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\c2ad5d17486ebc62348c56d8d99311f6\Hostname.dll
0x018e0000         0x018e5000              20480 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\e6444bcbe2c35f31d64a6c64f26786c2\Cwd.dll
0x018f0000         0x018fa000              40960 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\57d07aa6fc2fc0bc6be0d1b728dbaa21\Win32.dll
0x77670000         0x777a9000            1282048 C:\WINDOWS\system32\ole32.dll
0x77b90000         0x77b98000              32768 C:\WINDOWS\system32\VERSION.dll
0x01b00000         0x01b07000              28672 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\798185d7ef2434824ba7f68e7ee8abe9\Socket.dll
0x71c00000         0x71c17000              94208 C:\WINDOWS\system32\WS2_32.dll
0x71bf0000         0x71bf8000              32768 C:\WINDOWS\system32\WS2HELP.dll
0x01b50000         0x01b56000              24576 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\c495302f73228e35ea8af4b919191ef6\IO.dll
0x01b60000         0x01b70000              65536 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\5a1200de280fd6df2bf32876db0b806c\Storable.dll
0x02370000         0x02393000             143360 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\3f45cdd2088a8bd8084aae5c9cf45e41\Expat.dll
0x023a0000         0x023d4000             212992 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\cacb5a3683438077bff51b75083c11a2\SSLeay.dll
0x023e0000         0x024ec000            1097728 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\cacb5a3683438077bff51b75083c11a2\LIBEAY32.dll
0x71bb0000         0x71bb9000              36864 C:\WINDOWS\system32\wsock32.dll
0x024f0000         0x02524000             212992 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\cacb5a3683438077bff51b75083c11a2\SSLEAY32.dll
0x02540000         0x02559000             102400 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\302d1d31da39bb77de0cafe04c5a1456\Zlib.dll
0x02560000         0x02567000              28672 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\cb61ec0630977efd0bd5b3dd859bda10\Util.dll
0x03570000         0x03578000              32768 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\105fc1dae8388b98e42d2696c8da0410\Dumper.dll
0x03580000         0x03586000              24576 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\42fcccac83c52664e937c09d0e1e7988\Glob.dll
0x03590000         0x03597000              28672 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\317b1c0392b4b1d384a83cb265c0d9d1\SHA1.dll
0x71b20000         0x71b61000             266240 C:\WINDOWS\System32\mswsock.dll
0x5f270000         0x5f2ca000             368640 C:\WINDOWS\system32\hnetcfg.dll
0x76ed0000         0x76efa000             172032 C:\WINDOWS\system32\DNSAPI.dll
0x76f70000         0x76f77000              28672 C:\WINDOWS\System32\winrnr.dll
0x76f10000         0x76f3e000             188416 C:\WINDOWS\system32\WLDAP32.dll
0x76f80000         0x76f85000              20480 C:\WINDOWS\system32\rasadhlp.dll
0x71ae0000         0x71ae8000              32768 C:\WINDOWS\System32\wshtcpip.dll
0x01a00000         0x01a06000              24576 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\314c3972b76d2f26a0d2aaf17e58c879\Base64.dll
0x01a10000         0x01a19000              36864 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\d7e3e5887f4c718c4f6eef61761c329a\Encode.dll
0x01a30000         0x01aae000             516096 C:\DOCUME~1\NETAPP~1\LOCALS~1\Temp\pdk-NETAPP001P\6a4a331537ab7f59816c625d833c556d\dominocore.dll
0x60000000         0x611c4000           18628608 e:\Notes\nnotes.dll
0x761b0000         0x76243000             602112 C:\WINDOWS\system32\CRYPT32.dll
0x76190000         0x761a2000              73728 C:\WINDOWS\system32\MSASN1.dll
0x62000000         0x62315000            3231744 e:\Notes\nxmlproc.dll
0x7c3a0000         0x7c41b000             503808 e:\Notes\MSVCP71.dll
0x7c340000         0x7c396000             352256 e:\Notes\MSVCR71.dll
0x618d0000         0x61929000             364544 e:\Notes\js32.dll
0x623d0000         0x625a5000            1921024 e:\Notes\NLSCCSTR.DLL
0x7d0e0000         0x7d16b000             569344 C:\WINDOWS\system32\OLEAUT32.dll
0x76920000         0x769e2000             794624 C:\WINDOWS\system32\USERENV.dll
0x61f40000         0x61f46000              24576 e:\Notes\ndgts.dll
0x619d0000         0x61aee000            1171456 e:\Notes\NSTRINGS.DLL
0x61b00000         0x61b06000              24576 e:\Notes\namhook.DLL
0x645f0000         0x64a55000            4608000 E:\notes\nsdhelp.dll
0x71bd0000         0x71be1000              69632 C:\WINDOWS\system32\MPR.dll

<@@ ------ Notes Process Info -> Call Stack for Process :: [snapcreator:  019c] (Time 11:10:58) ------ @@>

############################################################
### thread 1/4: [snapcreator:  019c:  0d90] FATAL THREAD (Panic)
### FP=0x0012b6dc, PC=0x7c82847c, SP=0x0012b66c
### stkbase=0x00130000, total stksize=69632, used stksize=18836
### EAX=0x00164dc8, EBX=0x0012c2ac, ECX=0x13800002, EDX=0x13800003
### ESI=0x00000490, EDI=0x00000000, CS=0x0000001b, SS=0x00000023
### DS=0x00000023, ES=0x00000023, FS=0x0000003b, GS=0x00000000 Flags=0x00200297
############################################################
[ 1] 0x7c82847c ntdll.KiFastSystemCallRet+0 (490,493e0,0,12be8c)
[ 2] 0x77e61c8d kernel32.WaitForSingleObject+18 (490,493e0,12c620,12c2bd)
@[ 3] 0x601b4ec5 nnotes.FRSendCommandToService+789 (12beac,12c2ac,12c2bd,0)
@[ 4] 0x601b5aff nnotes.OSRunExternalScript@8+1055 (12c,1)
@[ 5] 0x601b607f nnotes.FRTerminateWindowsResources+975 (1,1010,1,0)
@[ 6] 0x601b64a8 nnotes.OSFaultCleanupExt@24+984 (5786a68,1010,0,0,0,12c948)
@[ 7] 0x601b652a nnotes.OSFaultCleanup@12+26 (0,1010,0)
@[ 8] 0x601c1a24 nnotes.OSNTUnhandledExceptionFilter@4+276 (12d980)
@[ 9] 0x6018477d nnotes.Panic@4+589 (60d1b298)
@[10] 0x60002e42 nnotes.LockMemHandle@12+306 (f023de6c,12d9b8,12d9c4)
@[11] 0x6008fc96 nnotes.OSMemoryGetType@4+22 (6109d320)
@[12] 0x601954ad nnotes.OSMemoryLockWithType@8+13 (f023de6c,93bc)
@[13] 0x60005598 nnotes.OSLockVARRAY@4+88 (19c)
@[14] 0x60089b9b nnotes.EHOpenCtx@20+683 (5d7657c,14ceb678,61069090,3,0)
@[15] 0x600953fe nnotes.UNIDIndexOpen@4+622 (2ffc05)
@[16] 0x609c3daa nnotes.OpenUNIDIndex@8+634 (12dac6,0)
@[17] 0x609c4453 nnotes.DbLoadLite@4+179 (4240f9a4)
@[18] 0x608d1100 nnotes.NSFDbOpenExtended4@48+19664 (15aeba28,0,0,0,0,12fa3c,f10f10,ffffffff,0,0,0,0)
@[19] 0x600271b1 nnotes.NSFDbOpenExtended3@48+65 (15aeba28,0,0,0,0,12fb1c,f10f10,ffffffff,0,0,0,0)
@[20] 0x60074246 nnotes.NSFDbOpenExtended2@44+54 (15aeba28,0,0,0,0,12fb54,f10f10,ffffffff,0,0,0)
@[21] 0x60092bb6 nnotes.NSFDbOpenExtended@28+54 (15aeba28,0,0,0,0,12fb88,f10f10)
@[22] 0x600abb4a nnotes.NSFDbOpen@8+26 (15aeba28,12fbc8)
[23] 0x01a7f621 dominocore.boot_dominocore+321037 (459daf4,459df2c,3c57b2c,0)
[24] 0x01a3361f dominocore.boot_dominocore+9739 (334084,317731c,1,334084)
[25] 0x2804a23b perl512.Perl_find_runcv+39618 (1334084,334084,2802a5e2,334084)
[26] 0x28072266 perl512.Perl_runops_standard+12 (334084,4,335e80,332554)
[27] 0x004058e7 snapcreator+22759 (12fe18,0,0,4)
[28] 0x00405b42 snapcreator+23362 (0,0,4,335e80)
[29] 0x004010f6 snapcreator+4342 (3,3,332af0,40e000)
[30] 0x0040942d snapcreator+37933 (0,0,7ffd4000,0)
[31] 0x77e6f23b kernel32.ProcessIdToSessionId+521 (40934a,0,78746341,20)

############################################################
### PASS 2 : FATAL THREAD (Panic) with STACK FRAMES [snapcreator:  019c:  0d90]
############################################################
# ---------- Dump of stack from SP to FP ----------
            0012b66c 997B827C 1E1DE677 90040000 00000000 | .{.| ...w .... .... |
            0012b67c B0B61200 00000000 BDC21200 ACC21200 | .... .... .... .... |
            0012b68c 24000000 01000000 00000000 00000000 | $... .... .... .... |

It happens after the quiesce of the databases. The weird part is that the backup finishes succesfully... Is it a known issue?

We installed Snap Creator 3.5 for the archiving of the logs...

Anyone an idea?

Kind regards,

Gert

Re: Snap Creator causes domino to crash

Dear Gert,

I got the same problem as you. Can you fix the problem yet? If yes, please kindly provide an advise for me.

Thank you.

Regards,

Re: Snap Creator causes domino to crash

Hi,

The system that crashed was running on a Windows 2003  32-bit version. Now we have upgraded the OS to Windows 2008 R2, fully 64-bit and now the backups are running fine.

I’ll hope this helped.

Regards,

Gert