snapshot failling domino 3.6 p1

[RAKESHMOLE2]

Hi,

I am trying to create domino snapshots using  server /agent model type (3.6 p1)

Server is windows 2008r2 and agent is linux red hat (centos)

However it fails with the following.

[Mon Nov 26 15:08:58 2012] INFO: Logfile timestamp: 20121126150858
[Mon Nov 26 15:08:58 2012] INFO: Plugin validation for domino completed successfully

########## Parsing Environment Parameters ##########

########## PRE APPLICATION QUIESCE COMMANDS ##########
[Mon Nov 26 15:08:58 2012] INFO: No commands defined

########## PRE APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application quiesce ##########
[Mon Nov 26 15:09:36 2012] [10.94.18.15:9090(3.6.0.1)] ERROR: [agt-00013] ERROR: [ltd-00009] Domino plugin couldn't be loaded. Reasons could be 1) Domino plugin is not supported on this platform.

  2)Prerequisites for domino plugin to run are not satisfied. Please check documentation(domino plugin setup).
Trace:Can't load 'auto/dominocore/dominocore.so' for module dominocore: auto/dominocore/dominocore.so: cannot open shared object file: No such file or directory at /</usr/local/scAgent3.6p1/snapcreator>DynaLoader.pm line 219.
at /</usr/local/scAgent3.6p1/snapcreator>dominocore.pm line 11

[Mon Nov 26 15:08:59 2012] ERROR: ERROR: [ltd-00009] Domino plugin couldn't be loaded. Reasons could be 1) Domino plugin is not supported on this platform.

  2)Prerequisites for domino plugin to run are not satisfied. Please check documentation(domino plugin setup).
Trace:Can't load 'auto/dominocore/dominocore.so' for module dominocore: auto/dominocore/dominocore.so: cannot open shared object file: No such file or directory at /</usr/local/scAgent3.6p1/snapcreator>DynaLoader.pm line 219.
at /</usr/local/scAgent3.6p1/snapcreator>dominocore.pm line 11

[Mon Nov 26 15:08:59 2012] [10.94.18.15:9090(3.6.0.1)] ERROR: [scf-00053] Application quiesce for plugin domino failed with exit code 1, Exiting!

########## Application unquiesce ##########
[Mon Nov 26 15:09:37 2012] [10.94.18.15:9090(3.6.0.1)] ERROR: [agt-00013] ERROR: [ltd-00009] Domino plugin couldn't be loaded. Reasons could be 1) Domino plugin is not supported on this platform.

  2)Prerequisites for domino plugin to run are not satisfied. Please check documentation(domino plugin setup).
Trace:Can't load 'auto/dominocore/dominocore.so' for module dominocore: auto/dominocore/dominocore.so: cannot open shared object file: No such file or directory at /</usr/local/scAgent3.6p1/snapcreator>DynaLoader.pm line 219.
at /</usr/local/scAgent3.6p1/snapcreator>dominocore.pm line 11

[Mon Nov 26 15:08:59 2012] ERROR: ERROR: [ltd-00009] Domino plugin couldn't be loaded. Reasons could be 1) Domino plugin is not supported on this platform.

  2)Prerequisites for domino plugin to run are not satisfied. Please check documentation(domino plugin setup).
Trace:Can't load 'auto/dominocore/dominocore.so' for module dominocore: auto/dominocore/dominocore.so: cannot open shared object file: No such file or directory at /</usr/local/scAgent3.6p1/snapcreator>DynaLoader.pm line 219.
at /</usr/local/scAgent3.6p1/snapcreator>dominocore.pm line 11

[Mon Nov 26 15:08:59 2012] ERROR: [scf-00054] Application unquiesce for plugin domino failed with exit code 1, Exiting!

########## PRE EXIT COMMANDS ##########
[Mon Nov 26 15:08:59 2012] INFO: No commands defined

########## PRE EXIT COMMANDS FINISHED SUCCESSFULLY ##########

[spinks · Accepted solution]

Rakesh,

According to the output the prerequisites for the Domino plug-in have not been met.

This is likely one of a three things:

I'm assuming that the Red Hat agent is your Domino server.

1) You need to be sure that you are using the 32-bit version of Snap Creator.  Domino is only available on Red Hat as a 32-bit application, so you must use 32-bit Snap Creator.

2) There are three sim links that need to be created in order for the Domino plug-in to work - these are documented in the install guide.

Assuming that you are using default installation path, you can copy/paste - otherwise you will need to edit the following:

ln -s /opt/ibm/lotus/notes/latest/linux/libxmlproc.so /usr/lib/libxmlproc.so

ln -s /opt/ibm/lotus/notes/latest/linux/libndgts.so /usr/lib/libndgts.so

ln -s /opt/ibm/lotus/notes/latest/linux/libnotes.so /usr/lib/libnotes.so

3) The config file is not correct - for example if the paths are wrong or the notes.ini file is not fully qualified.  You'd have to post the config for me to comment on this.

The issue is almost certainly in one of the three items above.

Please check the above and let us know the result.

Thanks!

John

[RAKESHMOLE2]

Jo John,

I did install the 64 bit Snapcreator . Let me install the 32bit and i will let you know , how it goes.

Thanks again

[spinks]

Thanks for letting us know.

For the Domino plug-in we have to match the bit level of Snap Creator with the bit-level of Domino - since Domino is only available on Linux as a 32-bit application then we must use the 32-bit Domino API calls, so you have to use the 32-bit version of Snap Creator.

Hope that helps to clarify why this is needed.

You should be able to copy over your configs and get going pretty quickly.

One final thing - be sure that you are NOT starting the Snap Creator agent as root - ideally you will start Snap Creator as the Domino account (notes, domadmin, or similar user) and this user will own the Snap Creator directories.

Domino cannot be run as root, so we cannot call the Domino APIs as root, so Snap Creator (which calls the Domino APIs) cannot be run as root.

Hope this helps!

John

[RAKESHMOLE2]

Hi John,

Thank you it works.

The issue was that it needed 32 bit install.

I just have to try a restore now.

Since its a iSCSI map , i assume i have to do it through the shell GUI wont work isnt it?

[spinks]

Rakesh,

I assume you are asking about a single database restore operation.

With FC or iSCSI this does need to be done through the CLI as you will first need to go through the steps or mounting a snapshot and copying it to the restore path indicated in your config file.

There is a video that walks through an example of this process and it should be pretty well documented in the admin guide.

The video is here on the communities site - here is a direct link as well: http://www.youtube.com/watch?v=_1F7MHpaLTY&feature=plcp

Hope this helps,

John

[RAKESHMOLE2]

Hi John

I was trying to do a snaprestore to test the restore out using snapdrive to mount the snapshot and i get this error.

0001-067 Command error: snapshot lotus-daily-daily_20121126161539 doesn't exist in the storage system path NETAPP-FE-B:/vol/XEN_Mail_02_VOL.

I checked the snapshot name and path everything seems to be OK

[ktenzer]

snapshot name is wrong should be lotus-daily_20121126161539

The snapshot name must be exact else things dont work.

%SNAME-%SNAPT_TYPE_%SNAP_TIME

SNAME should not have -daily in it or any -

Keith

[spinks]

Please note that there is a minor typo in the above string... Should be:

%SNAME-%SNAP_TYPE_%SNAP_TIME

Please let us know the result!

Thanks,

John

[RAKESHMOLE2]

Hi John,

1) I did as you guys suggested.

Results are as follows.

1) error

[root@domamericas2 bin]# snapdrive snap connect -lun /mnt/restore /test -snapname NETAPP-FE-B:/vol/XEN_Mail_02_VOL:lotus-daily_20121127093932 non-snapdrive snapshot

Auto detection of file_spec(s) in progress ...

9000-305 Command error: Could not detect type of the entity non-snapdrive. Provide a specific option (-lun, -dg, -fs or -lvol) if you know the type of the entity.

question 1) why do i have to include "non-snapdrive snapshot" in the snapshot name , otherwise it comes back with the same error snapshot no found.

question2) what is the snapdrive snap connect command i should use as this is a iscsi lun which i have presented to a Xen server as a   (SR)shared storage) and then from that lun i have carved out a a storgae as ext4 FS for the linux FS.

[spinks]

Rakesh,

Are you using SnapDrive to take the Snapshot copy?

If you are not then that may be why you have to provide non-snapdrive.

I don't know the answer to the Xen question - I'll have to search that one.

Thanks,

John

[RAKESHMOLE2]

Hi John,

Nope i am not using snap drive, i am using snapcreator.

also i get this error after the snapshot is taken when it is trying to quiesce the DB.

[Tue Nov 27 09:39:32 2012] INFO: Logfile timestamp: 20121127093932
[Tue Nov 27 09:39:32 2012] INFO: Plugin validation for domino completed successfully

########## Parsing Environment Parameters ##########

########## PRE APPLICATION QUIESCE COMMANDS ##########
[Tue Nov 27 09:39:32 2012] INFO: No commands defined

########## PRE APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application quiesce ##########
[Tue Nov 27 09:40:11 2012] [10.94.18.15:9090(3.6.0.1)] INFO: Quiescing databases
[Tue Nov 27 09:41:19 2012] [10.94.18.15:9090(3.6.0.1)] INFO: No Errors encountered while opening DBs.
[Tue Nov 27 09:41:19 2012] [10.94.18.15:9090(3.6.0.1)] INFO: Quiescing databases finished successfully

########## POST APPLICATION QUIESCE COMMANDS ##########
[Tue Nov 27 09:40:41 2012] INFO: No commands defined

########## POST APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## PRE COMMANDS ##########
[Tue Nov 27 09:40:41 2012] INFO: No commands defined

########## PRE COMMANDS FINISHED SUCCESSFULLY ##########

########## Detecting Data OnTap mode for 10.94.18.238 ##########
[Tue Nov 27 09:40:44 2012] INFO: Data OnTap 7 mode detected

########## Generating Info ASUP on 10.94.18.238 ##########
[Tue Nov 27 09:40:44 2012] INFO: ASUP create on 10.94.18.238 finished successfully

########## Gathering Information for 10.94.18.238:XEN_Mail_02_VOL ##########
[Tue Nov 27 09:40:44 2012] INFO: Performing Snapshot Inventory for XEN_Mail_02_VOL on 10.94.18.238
[Tue Nov 27 09:40:44 2012] INFO: Snapshot Inventory of XEN_Mail_02_VOL on 10.94.18.238 completed Successfully

########## Running Snapshot Rename on Primary 10.94.18.238 ##########

########## Creating snapshot(s) ##########
[Tue Nov 27 09:40:44 2012] INFO: NetApp Snap Creator Framework 3.6p1 detected that SnapDrive is not being used. File system consistency cannot be guaranteed for SAN/iSAN environments

########## Taking Snapshot on Primary 10.94.18.238:XEN_Mail_02_VOL ##########
[Tue Nov 27 09:40:44 2012] INFO: Creating Snapshot for XEN_Mail_02_VOL on 10.94.18.238
[Tue Nov 27 09:40:47 2012] INFO: Snapshot Create of lotus-daily_20121127093932 on 10.94.18.238:XEN_Mail_02_VOL Completed Successfully

########## PRE APPLICATION UNQUIESCE COMMANDS ##########
[Tue Nov 27 09:40:47 2012] INFO: No commands defined

########## PRE APPLICATION UNQUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application unquiesce ##########
[Tue Nov 27 09:41:26 2012] [10.94.18.15:9090(3.6.0.1)] INFO: UnQuiescing databases
[Tue Nov 27 09:43:07 2012] [10.94.18.15:9090(3.6.0.1)] ERROR: [ltd-00003] Unquiescing databases finished with errors
[Tue Nov 27 09:42:29 2012] [10.94.18.15:9090(3.6.0.1)] ERROR: [scf-00054] Application unquiesce for plugin domino failed with exit code 1, Exiting!

########## Application unquiesce ##########
[Tue Nov 27 09:43:08 2012] [10.94.18.15:9090(3.6.0.1)] ERROR: [agt-00008] Either no quiesce operation performed or forced unquiesce operation already triggered from watchdog, backup might not be useable
[Tue Nov 27 09:42:30 2012] ERROR: Forced unquiesced possibly triggered from watchdog
[Tue Nov 27 09:42:30 2012] ERROR: [scf-00054] Application unquiesce for plugin domino failed with exit code 100, Exiting!

########## PRE EXIT COMMANDS ##########
[Tue Nov 27 09:42:30 2012] INFO: No commands defined

########## PRE EXIT COMMANDS FINISHED SUCCESSFULLY ##########

John if i want to mount the snapshot using the snapdrive..what would be the command that i would be using?

snapdrive snap connect -fs /mnt/restore -destfv /vol/volname-snapname netapp:/vol/volname:lotus-daily_20121127093932 non-snapdrive snapshot -igroup igroup1 -autorename

[spinks]

Rakesh,

In the previous error message you are attempting to use SnapDrive to execute a snap connect command.

SnapDrive expects that you will use it to mount a Snapshot that you took with SnapDrive - I suspect this is why you are getting the error.

You can tell Snap Creator to use Snap Drive by adding a Snap Create command - This is documented for Snap Drive for Windows in the installation guide.

Using the GUI you can load a config and scroll down to the bottom of  the page.  Hit the "+" to add a command and type in the command for SnapDrive to execute.

For SnapDrive for Windows this would be: 

“C:\Program Files\NetApp\SnapDrive\SDCLI.exe” snap create –s %SNAME-%SNAP_TYPE_%SNAP_TIME –D F G H

I do not have a SnapDrive for UNIX command available, but it should be similar to the above.

If you do this you will need to also edit the agent.conf file on the agent.

Add the SnapDrive command to the agent.conf file in order to allow the agent to execute the command.

You do not need to fully qualify the command - just put in snapdrive or sdcli.exe or similar command.

In regards to the error - I'd need to see the debug log to get a better idea of what is happening.

It looks like the quiesce is working fine, but unquiesce is not working properly.

Without seeing the log I would guess that the Snap Creator user does not have access to write the changeinfo files to the changeinfo directory specified in the config file.  It is just a guess at this point - the debug log should show the actual error message.

You may also want to check the debug log on the agent itself - it sometimes shows information that doesn't reach the server.

Hope this helps - John