snapmirror/ sqlplus issue

[boonstra]

Greetings all !

I'm running into the following two issues  on CentOS6x/ RAC 11Gr2  >

1) snapmirror won't update at all nor do i see any errors in the logs

2) I'm seeing a bunch of this below, i fi remove the /nolog, the regular backup doesn't work at all  >>>

[Mon Jul  9 23:22:43 2012] [192.168.244.7:8181(3.5.0.1)] DEBUG: Command [/bin/su - oracle -c "ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1/;export ORACLE_HOME;ORACLE_SID=orcldb;export ORACLE_SID;/u01/app/11.2.0/grid/bin/sqlplus /nolog /nolog @/tmp/8MVWgMe5Qq.sc"] finished with

exit code: [0]

stdout: [

SQL*Plus: Release 11.2.0.3.0 Production

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Use SQL*Plus to execute SQL, PL/SQL and SQL*Plus statements.

Usage 1: sqlplus -H | -V

    -H             Displays the SQL*Plus version and the

                   usage help.

    -V             Displays the SQL*Plus version.

scdump is attached, i'd appreciate if anyone could lend a hand ...

[ktenzer]

Hi There,

1) SNAPMIRROR_VOLUMES is wrong, you have secondary system listed, it is always primary. Sometimes you might want to only snapmirror or snapvault certain volumes so it just lets you filter but it is primary.

Change:

SNAPMIRROR_VOLUMES=rstnva90-c9a-3210-b-sm:cvp_wcar_dbms_prod_archvol,cvp_wcar_dbms_prod_datavol,cvp_wcar_dbms_prod_redovol

to

SNAPMIRROR_VOLUMES=lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol,cvp_wcar_dbms_prod_datavol,cvp_wcar_dbms_prod_redovol

If primary controller isnt specified SC doesnt find anything to snapmirror so you that is why you dont see anything.

2) The issue is you dont have correct command in config

change:

SQLPLUS_CMD=/u01/app/11.2.0/grid/bin/sqlplus /nolog

to:

SQLPLUS_CMD=/u01/app/11.2.0/grid/bin/sqlplus

SC automatically uses nolog options so you have it there double which is causing problem

Change

ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1/

to:

ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1

Basically remove trailing slash, not needed, probably not causing issues but still.

Let me know if this helps?

Keith

[boonstra]

partially  

the SQLplus errors are gone, trailing slash/ remove /nolog is what fixed that

snapmirror still not happy even with SNAPMIRROR_VOLUMES=lenxkssig-9a-3210-b:cvp_wcar_dbms_prod_archvol,cvp_wcar_dbms_prod_datavol,cvp_wcar_dbms_prod_redovol

snapmirror manual works:

rstnva90-c9a-3210-b> snapmirror update -S lenxksig-c9a-3210-b-sm:cvp_wcar_dbms_prod_datavol  rstnva90-c9a-3210-b:cvp_wcar_dbms_prod_datavol

Transfer started.

Monitor progress with 'snapmirror status' or the snapmirror log.

rstnva90-c9a-3210-b>

[root@lenxksig-c9a-om-A scServer3.5.0]# ./snapcreator --profile rac-repl-test --action snap --policy hourly --verbose

[Wed Jul 11 09:31:19 2012] INFO: Logfile timestamp: 20120711093119

[Wed Jul 11 09:31:19 2012] INFO: Removing log rac-repl-test.out.20120709191606.log

[Wed Jul 11 09:31:19 2012] INFO: Removing log rac-repl-test.debug.20120709191606.log

[Wed Jul 11 09:31:19 2012] INFO: Removing log rac-repl-test.stderr.20120709191606.log

########## Parsing Environment Parameters ##########

########## PRE APPLICATION QUIESCE COMMANDS ##########

[Wed Jul 11 09:31:19 2012] INFO: No commands defined

########## PRE APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application quiesce ##########

[Wed Jul 11 13:31:20 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Quiescing databases

[Wed Jul 11 13:31:20 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Quiescing database orcldb1

[Wed Jul 11 13:31:20 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Quiescing database orcldb1 finished successfully

[Wed Jul 11 13:31:20 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Quiescing databases finished successfully

########## POST APPLICATION QUIESCE COMMANDS ##########

[Wed Jul 11 09:31:21 2012] INFO: No commands defined

########## POST APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## PRE NETAPP COMMANDS ##########

[Wed Jul 11 09:31:21 2012] INFO: No commands defined

########## PRE NETAPP COMMANDS FINISHED SUCCESSFULLY ##########

########## Detecting Data OnTap mode for lenxksig-c9a-3210-b ##########

[Wed Jul 11 09:31:21 2012] INFO: Data OnTap 7 mode detected

########## Generating Info ASUP on lenxksig-c9a-3210-b ##########

[Wed Jul 11 09:31:21 2012] INFO: NetApp ASUP create on lenxksig-c9a-3210-b finished successfully

########## Gathering Information for lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol ##########

[Wed Jul 11 09:31:21 2012] INFO: Performing NetApp Snapshot Inventory for cvp_wcar_dbms_prod_archvol on lenxksig-c9a-3210-b

[Wed Jul 11 09:31:21 2012] INFO: NetApp Snapshot Inventory of cvp_wcar_dbms_prod_archvol on lenxksig-c9a-3210-b completed Successfully

########## Gathering Information for lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_datavol ##########

[Wed Jul 11 09:31:21 2012] INFO: Performing NetApp Snapshot Inventory for cvp_wcar_dbms_prod_datavol on lenxksig-c9a-3210-b

[Wed Jul 11 09:31:21 2012] INFO: NetApp Snapshot Inventory of cvp_wcar_dbms_prod_datavol on lenxksig-c9a-3210-b completed Successfully

########## Gathering Information for lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_redovol ##########

[Wed Jul 11 09:31:21 2012] INFO: Performing NetApp Snapshot Inventory for cvp_wcar_dbms_prod_redovol on lenxksig-c9a-3210-b

[Wed Jul 11 09:31:21 2012] INFO: NetApp Snapshot Inventory of cvp_wcar_dbms_prod_redovol on lenxksig-c9a-3210-b completed Successfully

########## Running NetApp Snapshot Rename on Primary lenxksig-c9a-3210-b ##########

########## Creating snapshot(s) ##########

[Wed Jul 11 09:31:21 2012] INFO: NetApp Snap Creator Framework 3.5.0 detected that SnapDrive is not being used. File system consistency cannot be guaranteed for SAN/iSAN environments

########## Taking Snapshot on Primary lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol ##########

[Wed Jul 11 09:31:21 2012] INFO: Creating NetApp Snapshot for cvp_wcar_dbms_prod_archvol on lenxksig-c9a-3210-b

[Wed Jul 11 09:31:22 2012] INFO: NetApp Snapshot Create of repl-test-hourly_20120711093119 on lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol Completed Successfully

########## Taking Snapshot on Primary lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_datavol ##########

[Wed Jul 11 09:31:22 2012] INFO: Creating NetApp Snapshot for cvp_wcar_dbms_prod_datavol on lenxksig-c9a-3210-b

[Wed Jul 11 09:31:22 2012] INFO: NetApp Snapshot Create of repl-test-hourly_20120711093119 on lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_datavol Completed Successfully

########## Taking Snapshot on Primary lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_redovol ##########

[Wed Jul 11 09:31:22 2012] INFO: Creating NetApp Snapshot for cvp_wcar_dbms_prod_redovol on lenxksig-c9a-3210-b

[Wed Jul 11 09:31:22 2012] INFO: NetApp Snapshot Create of repl-test-hourly_20120711093119 on lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_redovol Completed Successfully

########## PRE APPLICATION UNQUIESCE COMMANDS ##########

[Wed Jul 11 09:31:22 2012] INFO: No commands defined

########## PRE APPLICATION UNQUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application unquiesce ##########

[Wed Jul 11 13:31:22 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Unquiescing databases

[Wed Jul 11 13:31:22 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Unquiescing database orcldb1

[Wed Jul 11 13:31:33 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Unquiescing database orcldb1 finished successfully

[Wed Jul 11 13:31:33 2012] [192.168.240.14:8181(3.5.0.1)] INFO: Unquiescing databases finished successfully

########## POST APPLICATION UNQUIESCE COMMANDS ##########

[Wed Jul 11 09:31:33 2012] INFO: No commands defined

########## POST APPLICATION UNQUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Generating Info ASUP on lenxksig-c9a-3210-b ##########

[Wed Jul 11 09:31:33 2012] INFO: NetApp ASUP create on lenxksig-c9a-3210-b finished successfully

########## POST NETAPP DATA TRANSFER COMMANDS ##########

[Wed Jul 11 09:31:33 2012] INFO: No commands defined

########## POST NETAPP DATA TRANSFER COMMANDS FINISHED SUCCESSFULLY ##########

########## Running NetApp Snapshot Delete on Primary lenxksig-c9a-3210-b ##########

[Wed Jul 11 09:31:48 2012] WARN: WARN: More than 4 snapshots exist (17) but they could not be deleted because they are not older than the retention age (4 days) for lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol

[Wed Jul 11 09:31:48 2012] WARN: WARN: More than 4 snapshots exist (17) but they could not be deleted because they are not older than the retention age (4 days) for lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_datavol

[Wed Jul 11 09:31:48 2012] WARN: WARN: More than 4 snapshots exist (17) but they could not be deleted because they are not older than the retention age (4 days) for lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_redovol

########## POST NETAPP COMMANDS ##########

[Wed Jul 11 09:31:48 2012] INFO: No commands defined

########## POST NETAPP COMMANDS FINISHED SUCCESSFULLY ##########

########## ARCHIVE COMMANDS ##########

[Wed Jul 11 09:31:48 2012] INFO: No commands defined

########## ARCHIVE COMMANDS FINISHED SUCCESSFULLY ##########

########## NetApp Snap Creator Framework 3.5.0 finished successfully ##########

[Wed Jul 11 09:31:48 2012] INFO: INFO: Snap Creator finished successfully ( Action: snap )

[ktenzer · Accepted solution]

This is also wrong

SECONDARY_INTERFACES=lenxksig-c9a-3210-b-sm,rstnva90-c9a-3210-b-sm

Should be something like

SECONDARY_INTERFACES=lenxksig-c9a-3210-b:lenxksig-c9a-3210-b-sm,rstnva90-c9a-3210-b-sm/rstnva90-c9a-3210-b-sm

Also not sure but since you are also using MANAGEMENT_INTERFACES you may need to have this for SNAPMIRROR_VOLUMES:

SNAPMIRROR_VOLUMES=lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol,cvp_wcar_dbms_prod_datavol,cvp_wcar_dbms_prod_redovol

Basically could you try to configure things more complicated

Keith

[boonstra]

made the changes, still no go 

so in terms of your last statement  .. what would be easiest .. i put it on since we need to replicate over a separate network/interfaces

i removed all references to the secondary interfaces, and all the xxx-sm filernames to make it easier  .... but also no workey.

am i missing something here ? if i understand it correctly, if you put just the mgmt interfaces inthere, it should grab the actual relation from snapmirror.conf ?

is there any log that will show me why it won't run snapmirrors ?

VOLUMES=lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol,cvp_wcar_dbms_prod_datavol,cvp_wcar_dbms_prod_redovo

MANAGEMENT_INTERFACES=lenxksig-c9a-3210-b,rstnva90-c9a-3210-b

SECONDARY_INTERFACE=
SNAPMIRROR_VOLUMES=lenxkssig-9a-3210-b:cvp_wcar_dbms_prod_archvol,cvp_wcar_dbms_prod_datavol,cvp_wcar_dbms_prod_redovol

[boonstra]

erm .. i just recreated a complete new profile ... and all is happy ...

########## Detecting Data OnTap mode for rstnva90-c9a-3210-b ##########

[Wed Jul 11 11:16:34 2012] INFO: Data OnTap 7 mode detected

[Wed Jul 11 11:16:34 2012] INFO: NetApp ASUP create on rstnva90-c9a-3210-b finished successfully

[Wed Jul 11 11:16:34 2012] INFO: Running NetApp Snapmirror Update on destination rstnva90-c9a-3210-b:cvp_wcar_dbms_prod_archvol source lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_archvol

[Wed Jul 11 11:16:38 2012] INFO: NetApp Snapmirror Update on destination rstnva90-c9a-3210-b:cvp_wcar_dbms_prod_archvol Started Successfully

########## Running NetApp Snapmirror Update using source lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_datavol ##########

[Wed Jul 11 11:16:40 2012] INFO: Getting NetApp Snapmirror Status for lenxksig-c9a-3210-b

[Wed Jul 11 11:16:40 2012] INFO: NetApp Snapmirror Status for lenxksig-c9a-3210-b finished successfully

########## Detecting Data OnTap mode for rstnva90-c9a-3210-b ##########

[Wed Jul 11 11:16:42 2012] INFO: Data OnTap 7 mode detected

[Wed Jul 11 11:16:43 2012] INFO: NetApp ASUP create on rstnva90-c9a-3210-b finished successfully

[Wed Jul 11 11:16:43 2012] INFO: Running NetApp Snapmirror Update on destination rstnva90-c9a-3210-b:cvp_wcar_dbms_prod_datavol source lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_datavol

[Wed Jul 11 11:16:46 2012] INFO: NetApp Snapmirror Update on destination rstnva90-c9a-3210-b:cvp_wcar_dbms_prod_datavol Started Successfully

########## Running NetApp Snapmirror Update using source lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_redovol ##########

[Wed Jul 11 11:16:48 2012] INFO: Getting NetApp Snapmirror Status for lenxksig-c9a-3210-b

[Wed Jul 11 11:16:48 2012] INFO: NetApp Snapmirror Status for lenxksig-c9a-3210-b finished successfully

########## Detecting Data OnTap mode for rstnva90-c9a-3210-b ##########

[Wed Jul 11 11:16:50 2012] INFO: Data OnTap 7 mode detected

[Wed Jul 11 11:16:51 2012] INFO: NetApp ASUP create on rstnva90-c9a-3210-b finished successfully

[Wed Jul 11 11:16:51 2012] INFO: Running NetApp Snapmirror Update on destination rstnva90-c9a-3210-b:cvp_wcar_dbms_prod_redovol source lenxksig-c9a-3210-b:cvp_wcar_dbms_prod_redovol

[Wed Jul 11 11:16:54 2012] INFO: NetApp Snapmirror Update on destination rstnva90-c9a-3210-b:cvp_wcar_dbms_prod_redovol Started Successfully

[ktenzer]

Awesome glad it works

Yeah SECONDARY_INTERFACES could be easier, it is a bit complicated, hopefully we can improve this in future

Keith

[boonstra]

thanks again for your help, also a shout-out for neto .. he assisted with some oracle funny business