Re: Configuring Snapcreator to work with Oracle (10.2) on Linux

michaeldparker · ‎2012-12-10

Hi,

Is there any documentation specific to snapcreator and Oracle on Linux or can someone who has a better understanding of how to configure snapcreator help me through this please? I am trying to get snapcreator to take snapshots of my oracle DB and vault it to another array. I went through the gui wizard to configure a basic file and it's not working. Below is the output of my failed attempt:

[Mon Dec 10 15:20:43 2012] INFO: Logfile timestamp: 20121210152043

[Mon Dec 10 15:20:43 2012] INFO: Plugin validation skipped for oracle, no plugin parameters file found or no parameters set

########## Parsing Environment Parameters ##########

########## PRE APPLICATION QUIESCE COMMANDS ##########

[Mon Dec 10 15:20:43 2012] INFO: No commands defined

########## PRE APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application quiesce ##########

[Mon Dec 10 15:20:49 2012] [orcyrac1:9090(3.6.0.1)] INFO: Quiescing databases

[Mon Dec 10 15:20:49 2012] [orcyrac1:9090(3.6.0.1)] INFO: Quiescing database orcy

[Mon Dec 10 15:20:49 2012] [orcyrac1:9090(3.6.0.1)] ERROR: [ora-00020] Oracle SQL*Plus command [/bin/su - oracle -c "ORACLE_HOME=/opt/oracle/product/10.2/;export ORACLE_HOME;ORACLE_SID=orcy;export ORACLE_SID;/opt/oracle/product/10.2/bin/sqlplus /nolog @/tmp/UWnSRTKKFZ.sc"] failed with return code [0] and message [The Oracle base for ORACLE_HOME=/opt/oracle/product/10.2 is /opt/oracle/product/10.2

SQL*Plus: Release 10.2.0.5.0 - Production on Mon Dec 10 15:20:49 2012

Connected to an idle instance.

select * from v$version

*

ERROR at line 1:

ORA-01034: ORACLE not available

Disconnected

]

[Mon Dec 10 15:20:49 2012] [orcyrac1:9090(3.6.0.1)] ERROR: [ora-00001] Verifying correct version of database orcy failed

[Mon Dec 10 15:20:47 2012] [orcyrac1:9090(3.6.0.1)] ERROR: [scf-00053] Application quiesce for plugin oracle failed with exit code 1, Exiting!

########## Application unquiesce ##########

[Mon Dec 10 15:20:53 2012] [orcyrac1:9090(3.6.0.1)] INFO: Unquiescing databases

[Mon Dec 10 15:20:53 2012] [orcyrac1:9090(3.6.0.1)] INFO: Unquiescing database orcy

[Mon Dec 10 15:20:53 2012] [orcyrac1:9090(3.6.0.1)] ERROR: [ora-00010] Ending hot backup mode for database orcy failed

[Mon Dec 10 15:20:49 2012] ERROR: [scf-00054] Application unquiesce for plugin oracle failed with exit code 1, Exiting!

########## PRE EXIT COMMANDS ##########

[Mon Dec 10 15:20:49 2012] INFO: No commands defined

########## PRE EXIT COMMANDS FINISHED SUCCESSFULLY ##########

I am attaching my configuration file as well for reference. As you can see, it's not even connecting to the oracle database, orcy, but once I get past that point, how does it take the snapshots? Do I need to specify additional information for snapdrive to take the snapshot since the server uses lun? Any help someone can provide is greatly appreciated.

Thanks

Michael

michaeldparker · ‎2012-12-10

Ok, i got a little further. For the home directory, I had a "/" at the end of the path. I took that out and re-ran. Now I'm getting further along. As you can see from the out log, it says that the db was quiesced, but it cannot take it out. My Oracle DB says that the DB is still not in hot backup so either it didn't put it in hotbackup or something took it out. Thought? Also, since it is luns, do I have to configure it to use snapdrive?

Thanks

[Mon Dec 10 16:53:18 2012] INFO: Logfile timestamp: 20121210165318

[Mon Dec 10 16:53:18 2012] INFO: Plugin validation skipped for oracle, no plugin parameters file found or no parameters set

########## Parsing Environment Parameters ##########

########## PRE APPLICATION QUIESCE COMMANDS ##########

[Mon Dec 10 16:53:18 2012] INFO: No commands defined

########## PRE APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application quiesce ##########

[Mon Dec 10 16:53:23 2012] [orcyrac1:9090(3.6.0.1)] INFO: Quiescing databases

[Mon Dec 10 16:53:23 2012] [orcyrac1:9090(3.6.0.1)] INFO: Quiescing database orcy

[Mon Dec 10 16:53:28 2012] [orcyrac1:9090(3.6.0.1)] INFO: Quiescing database orcy finished successfully

[Mon Dec 10 16:53:28 2012] [orcyrac1:9090(3.6.0.1)] INFO: Quiescing databases finished successfully

########## POST APPLICATION QUIESCE COMMANDS ##########

[Mon Dec 10 16:53:23 2012] INFO: No commands defined

########## POST APPLICATION QUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## PRE COMMANDS ##########

[Mon Dec 10 16:53:24 2012] INFO: No commands defined

########## PRE COMMANDS FINISHED SUCCESSFULLY ##########

########## Detecting Data OnTap mode for cdcfas02 ##########

[Mon Dec 10 16:53:28 2012] INFO: Data OnTap 7 mode detected

########## Generating Info ASUP on cdcfas02 ##########

[Mon Dec 10 16:53:29 2012] INFO: ASUP create on cdcfas02 finished successfully

########## Gathering Information for cdcfas02:orcyarch1 ##########

[Mon Dec 10 16:53:29 2012] INFO: Performing Snapshot Inventory for orcyarch1 on cdcfas02

[Mon Dec 10 16:53:29 2012] INFO: Snapshot Inventory of orcyarch1 on cdcfas02 completed Successfully

########## Running Snapshot Rename on Primary cdcfas02 ##########

[Mon Dec 10 16:53:30 2012] INFO: orcy-daily_20121210165318 is the first snapshot taken for cdcfas02:orcyarch1, Skipping!

########## Creating snapshot(s) ##########

[Mon Dec 10 16:53:30 2012] INFO: NetApp Snap Creator Framework 3.6.0 detected that SnapDrive is not being used. File system consistency cannot be guaranteed for SAN/iSAN environments

########## Taking Snapshot on Primary cdcfas02:orcyarch1 ##########

[Mon Dec 10 16:53:30 2012] INFO: Creating Snapshot for orcyarch1 on cdcfas02

[Mon Dec 10 16:53:48 2012] INFO: Snapshot Create of orcy-daily_20121210165318 on cdcfas02:orcyarch1 Completed Successfully

########## Detecting Data OnTap mode for cdcfas01 ##########

[Mon Dec 10 16:53:48 2012] INFO: Data OnTap 7 mode detected

########## Generating Info ASUP on cdcfas01 ##########

[Mon Dec 10 16:53:48 2012] INFO: ASUP create on cdcfas01 finished successfully

########## Gathering Information for cdcfas01:orcyctrl1 ##########

[Mon Dec 10 16:53:48 2012] INFO: Performing Snapshot Inventory for orcyctrl1 on cdcfas01

[Mon Dec 10 16:53:48 2012] INFO: Snapshot Inventory of orcyctrl1 on cdcfas01 completed Successfully

########## Gathering Information for cdcfas01:orcydata1 ##########

[Mon Dec 10 16:53:48 2012] INFO: Performing Snapshot Inventory for orcydata1 on cdcfas01

[Mon Dec 10 16:53:50 2012] INFO: Snapshot Inventory of orcydata1 on cdcfas01 completed Successfully

########## Running Snapshot Rename on Primary cdcfas01 ##########

[Mon Dec 10 16:53:50 2012] INFO: orcy-daily_20121210165318 is the first snapshot taken for cdcfas01:orcyctrl1, Skipping!

[Mon Dec 10 16:53:50 2012] INFO: orcy-daily_20121210165318 is the first snapshot taken for cdcfas01:orcydata1, Skipping!

########## Creating snapshot(s) ##########

[Mon Dec 10 16:53:50 2012] INFO: NetApp Snap Creator Framework 3.6.0 detected that SnapDrive is not being used. File system consistency cannot be guaranteed for SAN/iSAN environments

########## Taking Snapshot on Primary cdcfas01:orcyctrl1 ##########

[Mon Dec 10 16:53:52 2012] INFO: Creating Snapshot for orcyctrl1 on cdcfas01

[Mon Dec 10 16:54:02 2012] INFO: Snapshot Create of orcy-daily_20121210165318 on cdcfas01:orcyctrl1 Completed Successfully

########## Taking Snapshot on Primary cdcfas01:orcydata1 ##########

[Mon Dec 10 16:54:02 2012] INFO: Creating Snapshot for orcydata1 on cdcfas01

[Mon Dec 10 16:54:04 2012] INFO: Snapshot Create of orcy-daily_20121210165318 on cdcfas01:orcydata1 Completed Successfully

########## PRE APPLICATION UNQUIESCE COMMANDS ##########

[Mon Dec 10 16:54:04 2012] INFO: No commands defined

########## PRE APPLICATION UNQUIESCE COMMANDS FINISHED SUCCESSFULLY ##########

########## Application unquiesce ##########

[Mon Dec 10 16:55:04 2012] ERROR: 500 read timeout at /<C:\Program Files\NetApp\NetApp_Snap_Creator_Framework\scServer3.6.0\snapcreator.exe>SnapCreator/Agent/Remote.pm line 541

[Mon Dec 10 16:55:04 2012] [orcyrac1:9090(3.6.0.1)] ERROR: [scf-00054] Application unquiesce for plugin oracle failed with exit code 1, Exiting!

########## Application unquiesce ##########

[Mon Dec 10 16:55:17 2012] [orcyrac1:9090(3.6.0.1)] ERROR: [agt-00008] Either no quiesce operation performed or forced unquiesce operation already triggered from watchdog, backup might not be useable

[Mon Dec 10 16:55:13 2012] ERROR: Forced unquiesced possibly triggered from watchdog

[Mon Dec 10 16:55:13 2012] ERROR: [scf-00054] Application unquiesce for plugin oracle failed with exit code 100, Exiting!

########## PRE EXIT COMMANDS ##########

[Mon Dec 10 16:55:13 2012] INFO: No commands defined

########## PRE EXIT COMMANDS FINISHED SUCCESSFULLY ##########

ktenzer · ‎2012-12-10

Check SC_AGENT_TIMEOUT

The unquiesce is taking longer than the timeout. It looks like you have it set to 60 seconds so try 300.

The reason why the oracle databases are not in hotbackup mode after unquiesce is because SC did its job. They should never be left in hotbackup mode. SC server send requests to agent to unquiesce then waits for duration of timeout. If agent / plugin are still not done then you get a read timeout as above. The agent however will continue to carry out its last instruction which was unquiescce regardless of if server throws timeout or not.

Hope this helps

Keith

michaeldparker · ‎2012-12-11

Keith,

It definitely helped! Thanks for the reply. This is some great software and I appreciate the hard work you have provided on it and all of the support that I know I've received from you multiple times.

So, I'm a little further along now. I have some additional questions unless you can point me to a document that will fix in the blanks.

I noticed during in the logs it said filesystem consistency could not be guaranteed for SAN if Snapdrive is not used. Since, I am using SAN, I'll muddle through getting those commands correct, but I'm confused if I need to place a check in the "Consistency Group" box, enable snapdrive discovery, and what other options I need there.

Also, concerning the "Backup Control File Dir". I have seen other posts on this, but I'm still uncertain what this path should be set to. My Orcy DB is using ASM. Someone had mentioned to make the path on your data files path so that when you snap, the file is captured. Since I'm using ASM, it doesn't seem like I can do that. Since I'm already snapping my original ctrl location, archlogs, and db, how important is this field and what options do I have?

Thanks for the help.

Michael

ktenzer · ‎2012-12-11

SnapDrive is not required for SC, this is a big difference compared to SnapManager. The reason is flexibility and choice.

SC will not ensure file system is consistent, if you care about this then you need to use SnapDrive to trigger snapshots using SNAPDRIVE=Y, NTAP_SNAPSHOT_CREATE_CMD01=snapdrive bla bla bla -snap %SNAME-%SNAP_TYPE_recent, and TIMESTAMP_ONLY=N.

Do you need SnapDrive for unix? For NFS, never. For SAN also no not in my opinion unless you have UFS or some non-journaled file system. If you have a journaled file system then all SnapDrive buys you is maybe not having to do fsck when restoring. On Windows? yes snapdrive always.

The backup control file dir is simply where we place backup control files. We make a copy of control file before and after snapshot. Do you need these files? No not really only if you dont have original control files. I would just put them on some file system which is on netapp. Doesnt matter but then they will be captured on snapshot somewhere, otherwise local file system if you in fact dont plan on needing these.

Happy SnapCreating!

Keith

michaeldparker · ‎2012-12-12

Thanks Keith.

As a precaution, I decided to use snapdrive to create the snapshots and to the best of my ability, everything is configured properly. The creator job appears to works except it is not registering the backups with PM and updating the vault relationships. I've looked over the config multiple times and don't see why it's not working. Looking in the output, it's not even attempting to register the backup and protect it. What am I doing wrong as I'm at a loss? I did create the dataset and imported 14 relationships. Below is the command I used to create the dataset from my managment server. I've attached my config and my job run output. While you are at it, are my snapdrive snap commands correct?

Thanks!

Michael

Command to create dataset:

snapcreator.exe --profile ORCYRAC2 --action pmsetup

brauntvr2swiss · ‎2012-12-12

Hi Michael

Do you see the newly created Dataset in PM?

regards

Thomas

michaeldparker · ‎2012-12-12

Yes, I do. I was just discsussing this with a co-worker and we’re validating everything. I see the dataset, it says it was created by snapcreator, version3.6, etc. Looking back at the config via the gui, I see a Yes in the backup via PM options, etc. We’re both lost …. ☹

brauntvr2swiss · ‎2012-12-12

We have had Problems with case sensitiv things. How do you see the Filers in PM. Capital Letter or Lower case?

michaeldparker · ‎2012-12-12

I assume you are referring to the portion that says Physical resources. When I expand that, I see the controllers in all capital letters.

brauntvr2swiss · ‎2012-12-12

Can you please change following:

VOLUMES=cdcfas01:orcyctrl1,orcydata1;cdcfas02:orcyarch1

NTAP_SNAPSHOT_RETENTIONS=daily:0

NTAP_USERS=cdcfas01:snapcreator/53616c7465645f5f317b7b3ff848d28f2e64837365c912e2870f0c457a0e1c6d;cdcfas02:snapcreator/53616c7465645f5f4e2855b329e19bb8df96dad6d8ff8cde27df54750a74fd65

TO

VOLUMES=CDCFAS01:orcyctrl1,orcydata1;CDCFAS02:orcyarch1

NTAP_SNAPSHOT_RETENTIONS=daily:0

NTAP_USERS=CDCFAS01:snapcreator/53616c7465645f5f317b7b3ff848d28f2e64837365c912e2870f0c457a0e1c6d;CDCFAS02:snapcreator/53616c7465645f5f4e2855b329e19bb8df96dad6d8ff8cde27df54750a74fd65

And Try it again

michaeldparker · ‎2012-12-12

Good catch. I don’t think I would have thought about that. I’m getting further in that it attempts to protect it via PM now, but I’m getting this error:

<event-message>SNAPCREATOR ERROR: There is no object named 'snapcreator_ORCY'. (code = 22255) (Config: ORCYRAC2 Name: orcy Policy: daily)</event-message>

When I look in PM, I assume by object, it is referring to the dataset name. The dataset name that snapcreator had created was snapcreator_ORCYRAC2, not 'snapcreator_ORCY' as it appears to be looking for now.

Thanks.

michaeldparker · ‎2012-12-12

Ah, I just checked my config and I saw an entry for snapcreator_ORCY so I changed it to snapcreator_ORCYRAC2. It's running now. I'll take it a good while to update the relationships. I'll let you guys know if something doesn't complete properly. So far so good though....

Thanks very much for the help!

michaeldparker · ‎2012-12-12

Ok, it is running now and it’ll probably take a good while to update the relationships. All looks good except I’m see that it is not updating my orcyarch1 volume snapvault relationships on CDCFAS02. It’s only updating relationships on CDCFAS01. I don’t see any reasons for that problem in the output log or in the configuration file. Looking at the output section where it says “Creating Protection Manager backup version …..”, I see it add all members from CDCFAS01, but doesn’t mention CDCFAS02. Any thoughts on that?

Thanks

brauntvr2swiss · ‎2012-12-12

Do have the new configfile and a new log for me?

regards

Thomas

michaeldparker · ‎2012-12-12

Here you go. As you'll see from the out log that the job is still running waiting on the vaults to update. Nowhere in there do I see it mention anything about adding backup versions for orcyarch on CDCFAS02.

Thanks

brauntvr2swiss · ‎2012-12-12

Can you please run following Command on the dfm server:

dfpm dataset list -m snapcreator_ORCYRAC2

michaeldparker · ‎2012-12-12

Here you go:

Id Node Name Dataset Id Dataset Name Member Type Name

brauntvr2swiss · ‎2012-12-12

Its not complete. I dont see anything. Only the heading

michaeldparker · ‎2012-12-12

Sorry, I had replied via email and it chopped it out. See if this is any better. Thanks

Id Node Name Dataset Id Dataset Name Member Type Name

---------- -------------------- ---------- -------------------- -------------------------------------------------- -------------------------------------------------------

418 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q1

419 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q2

420 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q3

421 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q4

422 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q5

423 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q6

424 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q7

425 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q8

426 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q9

427 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q10

428 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcydata1/q11

430 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS01:/orcyctrl1/q1

469 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS02:/orcyarch1/q1

48410 Primary data 50892 snapcreator_ORCYRAC2 qtree CDCFAS02:/orcyarch1/q2

47840 Backup 50892 snapcreator_ORCYRAC2 volume CDCFAS10:/smo_orcyctrl1

47843 Backup 50892 snapcreator_ORCYRAC2 volume CDCFAS10:/smo_orcydata1

48209 Backup 50892 snapcreator_ORCYRAC2 volume CDCFAS10:/smo_orcyarch1

michaeldparker · ‎2012-12-12

By the way, we are using creator for backing up our DB2 databases and they are working great with PM. Something's obviously wrong, but I've been comparing my config against those that work and I'm not seeing anything obvious. As you ca see from the output of the log, it's not even trying to protect via PM so I don't have any errors to chase. It's like it's just not even reading that section of the config file.