Help restoring Oracle 11G database from a SC backup

[michaeldparker]

Hi,

So, last week we fought through our ignorance and figured out how to configure SC to backup our Oracle DB.  Now my DBA is attempting to open an SC backup in Oracle to test the backup.  When he does his recovery steps, he's getting this output:

SQL> recover database using backup controlfile until cancel;

ORA-00279: change 237775255263 generated at 12/17/2012 09:30:02 needed for thread 1

ORA-00289: suggestion : +AGILE_ARCH1

ORA-00280: change 237775255263 for thread 1 is in sequence #197029

Specify log: {<RET>=suggested | filename | AUTO | CANCEL}

ORA-00308: cannot open archived log '+AGILE_ARCH1'

ORA-17503: ksfdopn:2 Failed to open file +AGILE_ARCH1

ORA-15045: ASM file name '+AGILE_ARCH1' is not in reference form

ORA-10879: error signaled in parallel recovery slave

Any suggestion on what is going on?  We have 3 ASM volumes: ctrl, data, and arch.  Using SC, we are snapping those 3 volumes just like we do for our SMO backup.  I think he is attempting to use the ctrl file in the ctrl volume.

Thanks

Michael

[steiner]

I can help. Can you tell me the sequence of the snapshots that were created the sequence of restore operations used? The backup procedure should be using the plugin to place the database in hot backup mode, then take the snapshots of the datafiles, remove the database from hot backup, and then run the "alter system archive log current", and finally snap the archive log destination. Do you actually have 3 ASM diskgroups?

[michaeldparker]

Thanks, for the offer to help. Yes, we have 3 ASM groups: Arch, Data, and ctrl. We are using the native SC Oracle plugin. If I look in the log, I see that it says it quiesced the DB, after it is quiesced, my snapdrive commands run which snap all 3 groups at the same time. Lastly it then unquiesces the DB and contacts PM to protect the backup. I was going to attach a copy of my last log and my config file, but I cannot connect to the communities site currently for some reason. I’ll try again shortly and post the files.

Thanks

Michael

[steiner]

It shouldn’t be doing all 3 volumes at the same time. The archive log volume needs to be last. There are two things you need for a valid hot backup – an image of the datafiles while in hot backup mode and every single archive log generated while in hot backup mode. It will make your life easier if you also protect the control files, but strictly speaking it’s not required. The archive log volume should be defined as a metadata volume. That will make it get snapshotted last.

If you need to restore, don’t forget to shut down the database and the ASM diskgroup itself before doing any snaprestore operations.

[michaeldparker]

Ok. That makes sense based on what I’ve seen SMO do. I thought if I choose the Oracle plugin and put in the lines for the snaps that the plugin would handle doing everything I needed in the correct sequence. I barely know how to spell Oracle, so do you by chance have some sample lines that I suppose I need to be putting in my post section of the config file to handle it correctly then?

Thanks

[michaeldparker]

Here is my Config file and last log file output.  Thanks

Michael

[steiner]

These are items in the config file that need to be set…

-The Archive Log Volume must be defined as one of the Flexvols:

VOLUMES=etc2a:hps16_11g_nfs_oractl,hps16_11g_nfs_oradata,hps16_11g_nfs_oraarch

-The Meta Data Volume parameter must be set for the Flexvol which needs to be snapshot'd after the unquiesce.

META_DATA_VOLUME=etc2a:hps16_11g_nfs_oraarch

The Oracle Plug-in continues to be used and configured as usual..

Should see something like this in the logs…

########## Taking Snapshot on Primary etc2a:hps16_11g_nfs_oradata ##########

[Tue Dec 18 16:35:33 2012] INFO: Creating Snapshot for hps16_11g_nfs_oradata on etc2a

[Tue Dec 18 16:35:33 2012] INFO: Snapshot Create of nfs11g-daily_recent on etc2a:hps16_11g_nfs_oradata Completed Successfully

[Tue Dec 18 16:35:33 2012] WARN: A meta data volume hps16_11g_nfs_oraarch was detected, skipping snapshot, it wil be taken after unquiesce

########## PRE APPLICATION UNQUIESCE COMMANDS ##########

[Tue Dec 18 16:35:33 2012] INFO: No commands defined

########## PRE APPLICATION UNQUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application unquiesce ##########

[Tue Dec 18 16:35:33 2012] INFO: Unquiescing databases

[Tue Dec 18 16:35:33 2012] INFO: Unquiescing database nfs11g

[Tue Dec 18 16:35:35 2012] INFO: Unquiescing database nfs11g finished successfully

[Tue Dec 18 16:35:35 2012] INFO: Unquiescing databases finished successfully

########## Creating meta data snapshot for etc2a:hps16_11g_nfs_oraarch ##########

[Tue Dec 18 16:35:35 2012] INFO: Creating Snapshot for hps16_11g_nfs_oraarch on etc2a

[Tue Dec 18 16:35:35 2012] INFO: Snapshot Create of nfs11g-daily_recent on etc2a:hps16_11g_nfs_oraarch Completed Successfully

[michaeldparker]

Thanks for the configuration information. Unfortunately it is not working for me. This is how I am specifying the volumes now:

VOLUMES=CDCFAS01:agilearch1;CDCFAS02:agilectrl1,agiledata1

META_DATA_VOLUME=CDCFAS01:agilearch1

I see this in my log file:

Wed Dec 19 08:12:09 2012] INFO: Logfile timestamp: 20121219081208

INFO: Plugin validation skipped for oracle, no plugin parameters file found or no parameters set

1. Parsing Environment Parameters ##########

ERROR: The META_DATA_VOLUME parameter is defined but cannot be the only volume specified in VOLUMES parameter, the meta data volume must be a separate volume

1. Application unquiesce ##########

ERROR: Either no quiesce operation performed or forced unquiesce operation already triggered from watchdog, backup might not be useable

ERROR: Forced unquiesced possibly triggered from watchdog

ERROR: Application unquiesce for plugin oracle failed with exit code 100, Exiting!

1. PRE EXIT COMMANDS ##########

INFO: No commands defined

1. PRE EXIT COMMANDS FINISHED SUCCESSFULLY ##########

INFO: Creating OM Event (script:critical-event) on dfmsrvr01.acuitylightinggroup.com

INFO: OM Event (script:critical-event) on dfmsrvr01.acuitylightinggroup.com created successfully

Also, my oracle db is mounted on fcp luns. How do I break out the snapshot commands for the luns? Do I put the snapdrive command for it in the post section?

Thanks

Michael

[steiner]

SC doesn't seem to like the syntax of the volumes section.

[steiner]

The architectural overloads of SC tell me this is a bug. SC doesn't like the fact that the archive log volume is the only volume on that particular filer in the VOLUMES section. What version of SC are you using?

[michaeldparker]

I'm using 3.6

Sent from my iPhone

[brauntvr2swiss]

Hi Michael

I'm not a DBA admin, but I try to explain you 🙂

We do it following Way (without archivelogs):

1. Restore snapshot with snapcreator

2. All Controlfiles have to be the one, which include in the snapshot

3. Now copy (and rename) the redolog to the location, where Oracle want the archive log (fe cp  /data/redo1/DB/redo-1a.dbf /data/arch/DB/+AGILE_ARCH1)

4. recover database using backup controlfile until cancel;

        5. When not working taking the redo from the other Group  (fe cp  /data/redo1/DB/redo-1b.dbf /data/arch/DB/+AGILE_ARCH1)

        6. With one of the redologs from the snapshot, you schould be able to get following Output:

                Log applied.

       Media recovery complete.

7.     SQL> alter database open resetlogs;

Hope this help

Thomas