Snap Creator and Oracle RAC support

[ Edited ]


I've been trying to clone our 11.2 Oracle RAC databases on AIX 6.1 using Snap Creator 4.0p1d2 without success. The database uses ASM for database storage with each disk group using a separate volume. Example diskgroup layout:

DG_DATA - datafiles

DG_ORL - online redo

DG_FLASH - controlfiles

DG-ARCH - archived redo logs

A config was created for the data disk group which in turn calls another config for the archivelog disk group. Once snapshot copy is complete and volumes have been Snap mirrored to a remote server I then mount the ASM diskgroups and recover the database. However, the recovery process always fails. Here's an output of recovery operation after issuing the command RECOVER DATABASE USING BACKUP CONTROLFILE UNTIL CANCEL:

SQL> ORA-00279: change 7725379529097 generated at 12/18/2013 16:47:10 needed for
thread 1
ORA-00289: suggestion :
ORA-00280: change 7725379529097 for thread 1 is in sequence #21913

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
ORA-00279: change 7725379529097 generated at 12/18/2013 07:04:46 needed for
thread 2
ORA-00289: suggestion :
ORA-00280: change 7725379529097 for thread 2 is in sequence #20670

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
ORA-00328: archived log ends at change 7725368217050, need later change
ORA-00334: archived log:

ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01195: online backup of file 1 needs more recovery to be consistent

I've successfully used the same process for a single instance database. Is Oracle RAC supported for use with Snap Creator with the Oracle plug-in?



Re: Snap Creator and Oracle RAC support

Hello Leighton,

I see only 2 volumes in the config file. (asmdata_prod from filer14a and filer14b)

Where are your other volumes?

Also, once you add all the volumes, you may add this option.




Siva Ramanathan

Re: Snap Creator and Oracle RAC support

Hi Siva,

I'm cloning the database using different disk group names (because same disk groups already exist on the target server) so I don't need the diskgroups with the control files and online redo logs since they're being recreated.

This is the procedure I use:

1. Put database in hot backup mode
2.      Take snapshot copy of datafiles
3.      Take database out of hot backup mode
4.      Spool controlfile to trace.
5.      Switch archivelogs
6.      Take snapshot of archivelogs
7.      Mirror all snapshot volumes to remote server
8.      Mount snapshot copies on remote server
9.      Rename and mount ASM diskgroup
10.  Mount and recreate controlfile from trace
11.  Recover database using backup controlfile until cancel
12.  Open database resetlogs
13.  Shutdown and mount database
14.  Rename with NID
15.  Open database resetlogs

The NetApp Snap Creator software has a plug-in for Oracle that takes care of steps 1-8. The remaining steps are done using scripts.
All steps up to 10 complete successfully. Step 11 is where it’s failing during the recovery.

Also I'm going to try to clone all disk groups but keep the names the same (cloning to new server). Using this method should I create a separate config for all the diskgroups and call them from the data config?

What does the META_DATA_VOLUME parameter do? I've searched the compatibility matrix but didn't see any RAC mentioned.



Re: Snap Creator and Oracle RAC support

As of 4.0p1d2 you can designate one volume in META_DATA_VOLUME=filer:volume

This tells Snap Creator NOT to take backup of the volume during normal snapshot operation.

The workflow then becomes like this.

quiesce database

snapshot data volumes

unquiesce database  (this makes Oracle to flush logs to archive logs)

take snapshot of META_DATA_VOLUME (in your case archive logs)

You may also please refer this thread by

Re: Snap Creator and Oracle RAC support

Hi Siva,

Thanks for your response.

Isn't this the equivalent of calling the archivelog config from the data config POST_CLONE_CREATE_CMD?

POST_CLONE_CREATE_CMD03=SERVER:/opt/NetApp/snapcreator --profile prod1 --config prod1_archive --policy daily --action cloneVol

The archivelog config has ARCHIVE_LOG_ONLY=Y



Re: Snap Creator and Oracle RAC support

As long as prod1/prod1_archive has only the archive volume, I don't see a problem there.

Please refer to this thread, where has a script written.

I am looking for another article that Neto has written as well, and will soon include here.

Re: Snap Creator and Oracle RAC support

Here is another to refer.

Please let me know if you have further questions.

I will forward this to Oracle experts.

Re: Snap Creator and Oracle RAC support

Actually Neto, Mike and myself were working in parallel to create a script for this purpose. My script is not using ASMLib however since we're on AIX.

In any case, the ASM cloning piece works fine - it's just the recovery and more specifically RAC recovery that doesn't seem to work.

Re: Snap Creator and Oracle RAC support

I have just forwarded this thread to Neto and Jeff Steiner.

Since most of the folks are on holiday/vacation please expect delays.

Re: Snap Creator and Oracle RAC support

I am not using ASM, but clustered filesystem and have successfully cloned RAC databases using snapshots created by snapcreator, so it does work.

The procedure you outlined is very similar to the one I use. META_DATA_VOLUME is your friend to only have 1 config file instead of managing arclogs in a separate config. I also create the clone on a single instance and then modify init.ora to be clustered after recovery is complete.

The query listed in will tell you the arc logs needed for recovery. I would suggest running the query on the source database to discover which arclogs are needed, and then verifying that those logs are present in the mounted clone. (Isn't there some way to look at the content of the ASM diskgroup?)

I have also ran into the same error when I apply the log files in the incorrect order.