SNapmanager for Oracle 3.2 Direct NFS RMAN operation hangs

[jpylkkodba]

Hi,

I have problem taking online backups with SMO3.2 even using RMAN catalog or backup via controlfiles.

When SMO has mounted the snapshotclones and will start to do catalog datafile copies it just hangs.

See attached file of process tree.

On RMAN session trace I can see that there is some waiting ongoing.

DISK FILE OPERATIONS I/O WAITING (rman@TARGETDATABASE)

There is not much logs available.

The system setup:

Oracle Virtual Machine

Oracle unbreakable Linux.

Oracle Enterprice database 10.2.0.3

SNapmanager for ORacle 3.2

Snapdrive 5P1

Direct NFS is enabled

Ontap 8

[jessick]

UEK is not yet supported by SDU/SMO/SMSAP team.  Oracle 10.2.0.3 may also be an issue?

I cannot tell the other details of your config however there is also qualification work being done on some newer Oracle VM versions as well.

If this is a business critical system(s), you need to work with your account team to gather exact version information for these components.  They can then get time/date estimates for when this will be supported.

Perhaps the best way to get this started in to send them the output from a smo operation or smo profile dump (this will automatically gather a snapdrive.dc).  The version information can be extracted from those logs.

ref:  http://support.netapp.com/matrix

-Mark

[jpylkkodba]

Some kind of solution found and tested few times. It's seems to related to DNFS and how Oracle works.
I do not want to "hack" SMO but the solution in this matter seems to be when there is to interfaces involved. One is connected to the another filer and second interface is connected to the another filer. Everything works when SDU makes changes on fstab file and the same changed are transporterted dynamically to Direct NFS. Like documentation says that after cloning I need to do that manually, but it seems that it's too late on this matter.

Best Regards,

Jani

[jpylkkodba]

How this works on latest Direct NFS Oracle 11.2.0.3. I will request also SR to Oracle.

Working solution:

SDU will add the mount infromation to /etc/fstab. So this is monitored and when event happens those changes will be added also into oranfstab then this will work.
Not working situation & tested:

If this is not done those snapshots are cloned and mountet, but RMAN can not handle this situation, because it's target DB operation.
I tested this when RMAN hangs that I killed the rman process on the system (after that hanging session, SMO reposrts that backup is ok, even catalog never finished. Then the filesystem on that snapshot was mounted by SMO and then try the same command to RMAN "catalog datafile copy..." RMAN hangs again waiting for disk io. When these mounts are added dynamically filer-path, local-path and export into oranfstab file the RMAN catalog operation works just fine.

[DBA_STL_MARK]

I would be curious to know if you got any response from Oracle and/or Netapp on this issue.

In my case I am using Oracle 11.2.0.3

I am running into the same problem while trying to set up test databases on our brand new Netapp.

I must say I'm disappointed with Netapp that they don't stay more current.

Although I have not verified whether or not I have the same exact wait condition, it does happen

regardless of whether or not  DNFS is enabled.

I haven't been able to get an rman  'restore controlfile from '<path>'' to work -- it just hangs.  

The source disk is not  Netapp but the target is an nfs volume.

I did not have an issue restoring to a netapp Fiber Channel volume.

[jpylkkodba]

Hi,

Sorry   II'm also using 11.2.0.3. That wa typing error from beginning. Yesterday I have really interesting chat with Oracle. Let see what will haoppen.

[DBA_STL_MARK]

In my case, the issue turned out to be a misconfigured /etc/oranfstab.

[jpylkkodba · Accepted solution]

Hi,

The issue is solved. After kernel update Oracle components like RMAN can catalog the datafile volumes which are out of the oranfstab scope.

Now I can remove the script which checks the fstab and adds the new lines directly on the oranfstab in real-time...

Everything seems to be ok also on OEL 5.8 and OEL 6.3 with NFS. No issues so far.

Done backups and several restores and cloning into different hosts.

Best Regards,

J. Pylkkö