SMO and ASM under Solaris

[frank_berger2]

Hello Everyone,

I am trying to setup an test environment for Snapshot Manager for Oracle with Oracle ASM under Solaris.

I am using the following Software Versions:

1. NetApp OnTap Simulator NetApp Release 7.3.1

2. Solaris 10 for Intel (SunOS sol10-t01 5.10 Generic_139556-08 i86pc i386 i86pc)

3. Storage Access via iSCSI Software Initiator

4. SAN Tool Utility for NetApp filers 3.4

5. Snapdrive 4.1.1

6. SnapManager for Oracle 3.0.2

7. Oracle Version 10.2.0.4

When I try to create an Profile for my Database (which uses ASM to store the datafiles), I get the following Error Messages:

--[ERROR] FLOW-11019: Failure in Discover: SD-00018: Error discovering storage for /devices: SD-10016: Error executing snapdrive command "/usr/sbin/snapdrive storage show -fs /devices": 0002-107 Command error: Filesystem -- /devices type is not as configured
.
--[ERROR] FLOW-11008: Operation failed: SD-00018: Error discovering storage for /devices: SD-10016: Error executing snapdrive command "/usr/sbin/snapdrive storage show -fs /devices": 0002-107 Command error: Filesystem -- /devices type is not as configured
.
--[ERROR] SMO-05075: Profile create failed: SMO-13527: Error performing fast restore check: FLOW-11019: Failure in Discover: SD-00018: Error discovering storage for /devices: SD-10016: Error executing snapdrive command "/usr/sbin/snapdrive storage show -fs /devices": 0002-107 Command error: Filesystem -- /devices type is not as configured
..

The same happens if I issue that command in the shell:

# /usr/sbin/snapdrive storage show -fs /devices
0002-107 Command error: Filesystem -- /devices type is not as configured

I am attaching the sd-trace.log file which has some more interessting output like:

17:12:40 11/30/09 [1d]v,2,10,FileSystem::init: /devices - type: devfs mount options: dev=4780000
17:12:40 11/30/09 [1d]E,2,10,FileSystem::isTypeSupported: /devices - No matching file system assistant (type: 'devfs')
17:12:40 11/30/09 [1d]i,2,10,FileSystem::getPersistentMountStatusFromMountEntry: PersistentlyMounted = yes

Just for your information, we have seen the same error message on Solaris SPRAC as well.

Bye and thanks for your help...

Frank

[jessick]

Can you send a copy of your snapdrive.conf file?  What is fstype set to?

Also, try downloading and running the ConfigChecker and reply with the output:

http://now.netapp.com/NOW/download/tools/snapdrive_config_checker_unix/

-Mark

[frank_berger2]

Hello Mark,

jessick wrote:
Can you send a copy of your snapdrive.conf file?  What is fstype set to?

I only set the following options in snapdrive.conf, fstype is not set:

enable-implicit-host-preparation=on
default-transport="iscsi"
use-https-to-filer=off

Also, try downloading and running the ConfigChecker and reply with the output:

# /opt/NTAPsnapdrive/bin/sdconfcheck check           
Detected Intel/AMD X86  Architecture
Detected Solaris OS
Detected Software iSCSI on Solaris
Detected Sun UFS File System

Did not find any supported Volume managers.

Did not find any supported Multipath solutions.

Did not find any supported cluster solutions.
Detected iSCSI Solaris Support Kit 3.0.1

Did not find any supported configurations by SDU Version 4.1.1

Looks ok from my side. Because I want to use Oracle ASM I just have LUNs from at the level of Snapdrive / Solaris.

Here is how I have created the LUNs:

# snapdrive storage create -lun netapp01:/vol/vol1/lun0 -lunsize 1.5g -noreserve -igroup oraasm

# snapdrive storage create -lun netapp01:/vol/vol2/lun0 -lunsize 1.5g -noreserve -igroup oraasm

# snapdrive storage show -devices

Connected LUNs and devices:

device filename  adapter path size proto state  clone lun path                backing snapshot

----------------  ------- ---- ---- ----- -----  ----- --------                ----------------

/dev/dsk/c2t1d0s2 -       P    1.5g iscsi online No    netapp01:/vol/vol1/lun0 -

/dev/dsk/c2t2d0s2 -       P    1.5g iscsi online No    netapp01:/vol/vol2/lun0 -

General question:

Is it possible to use SMO/Snapdrive in combination with Oracle ASM under Solaris if the LUNs are connected via iSCSI or FC to the host system?

For instance on Linux the NetApp documentation mentions that ASMLIB has to be used. But for Solaris there is no ASMLIB...

Bye and thanks for your help

Frank

[frank_berger2]

Hello Everyone,

to help verifying my environment I am also attaching the output from /opt/NTAP/SANToolkit/bin/solaris_info and  /opt/NTAP/SANToolkit/bin/filer_info.

Best Regards

Frank

[sindhe]

Hi,

   You may be hitting BURT: 310545

You may be storing some of the oracle files on local storage, which is not supported (snapshot capable storage).

Please send me output of data_files, v$log, control_file, archive_log paths.

Regards,

Rajarao

[frank_berger2]

wHello Rajarao,

sindhe schrieb:
   You may be hitting BURT: 310545
You may be storing some of the oracle files on local storage, which is not supported (snapshot capable storage).
Please send me output of data_files, v$log, control_file, archive_log paths.

well, my setup is a quite clean test install...

Tablespace Name / File Class  Filename
----------------------------- ----------------------------------------------------------------

SYSAUX                        +DG05/b23/datafile/sysaux.260.704544601                             
SYSTEM                        +DG05/b23/datafile/system.261.704544595                             
TEMP                          +DG05/b23/tempfile/temp.264.704545869                                
UNDOTBS1                      +DG05/b23/datafile/undotbs1.259.704544603                           

USERS                         +DG05/b23/datafile/users.258.704544605                                
[ CONTROL FILE    ]           +DG05/b23/controlfile/current.257.704545477
[ CONTROL FILE    ]           +DG06/b23/controlfile/current.256.704545485
[ ONLINE REDO LOG ]           +DG05/b23/onlinelog/group_1.256.704545495                            
[ ONLINE REDO LOG ]           +DG05/b23/onlinelog/group_2.262.704545607                            
[ ONLINE REDO LOG ]           +DG05/b23/onlinelog/group_3.263.704545711                           

[ ONLINE REDO LOG ]           +DG06/b23/onlinelog/group_1.257.704545553                            
[ ONLINE REDO LOG ]           +DG06/b23/onlinelog/group_2.258.704545659                            
[ ONLINE REDO LOG ]           +DG06/b23/onlinelog/group_3.259.704545761

SQL> show parameter log_archive_dest
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest                     string
log_archive_dest_1                   string      LOCATION=+DG05/

SQL> show parameter spfile

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------
spfile                               string      +DG05/b23/spfileb23.ora

The database is a fresh installation and has written no archivelogs sofar...

In my opinion the problem is more related to Snapdrive as to SMO. The Error does occur very early during the profile creation.

SMO Logfile:

--[ INFO] SMO-13046: Operation GUID 4028d1ee254ef53101254ef53c720009 starting on Profile B23
--[ INFO] SMO-13505: SnapDrive environment verification passed.
--[ WARN] SMO-13521: Minimum supported version for instance type "asm" not found.
--[ WARN] SMO-13521: Minimum supported version for instance type "rdbms" not found.
--[ INFO] SMO-13507: JDBC verification for "sys@localhost:1521/B23" passed.
--[ WARN] SMO-13521: Minimum supported version for instance type "rdbms" not found.
--[ INFO] SMO-13506: SQLPlus verification for database SID "B23" passed.  Environment: [ORACLE_HOME=/u01/app/oracle/product/10.2.0/dbhome_1]
--[ INFO] SMO-07431: Saving starting state of the database: B23(OPEN).
--[ INFO] SMO-07431: Saving starting state of the database: B23(OPEN).
--[ INFO] SD-00016: Discovering storage resources for /devices.
--[ERROR] FLOW-11019: Failure in Discover: SD-00018: Error discovering storage for /devices: SD-10016: Error executing snapdrive command "/usr/sbin/snapdrive storage show -fs /devices": 0002-107 Command error: Filesystem -- /devices type is not as configured
.
--[ERROR] FLOW-11008: Operation failed: SD-00018: Error discovering storage for /devices: SD-10016: Error executing snapdrive command "/usr/sbin/snapdrive storage show -fs /devices": 0002-107 Command error: Filesystem -- /devices type is not as configured
.
--[ERROR] SMO-05075: Profile create failed: SMO-13527: Error performing fast restore check: FLOW-11019: Failure in Discover: SD-00018: Error discovering storage for /devices: SD-10016: Error executing snapdrive command "/usr/sbin/snapdrive storage show -fs /devices": 0002-107 Command error: Filesystem -- /devices type is not as configured
..

Best Regards

Frank

[sindhe · Accepted solution]

Hi,

    Can you paste output of below command.

SQL>show parameter asm

--> Just to confirm, oracle is discovering disk from /dev/rdsk

Regards,
Rajarao

[frank_berger2]

Hello,

ok, thanks for the hint!

Well Oracle ASM discovers the the disks just fine! But we made a new directory /oraasm and put some symblic links to the /dev/rdsk/ disks in.

It looks like this:

-bash-3.00$ cd /oraasm/
-bash-3.00$ ls -l
total 4
lrwxrwxrwx   1 root     root          18 Nov 30 14:53 asm_disk01 -> /dev/rdsk/c2t1d0s6
lrwxrwxrwx   1 root     root          18 Nov 30 14:54 asm_disk02 -> /dev/rdsk/c2t2d0s6

In Oracle ASM you would see:

SQL> select HEADER_STATUS, MODE_STATUS, STATE, path from v$asm_disk;

HEADER_STATU MODE_ST STATE PATH
--------------------------------------------------------------------------------
MEMBER       ONLINE  NORMAL /oraasm/asm_disk01

MEMBER       ONLINE  NORMAL /oraasm/asm_disk02

SQL> show parameter asm

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups                       string      DG05, DG06
asm_diskstring                       string      /oraasm/*
asm_power_limit                      integer     1

We used that to increase manageabiltiy of LUNs.

But your question gave me the right hint to solve the problem.

I changed the ASM configuration to use the LUNs directly like this:

SQL> alter system set asm_diskstring = '/dev/rdsk/c2t1d0s6','/dev/rdsk/c2t2d0s6' sid='+ASM';
SQL> select HEADER_STATUS, MODE_STATUS, STATE, path from v$asm_disk;

HEADER_STATU MODE_ST STATE PATH
----------------------------------------------
MEMBER       ONLINE  NORMAL /dev/rdsk/c2t1d0s6
MEMBER       ONLINE  NORMAL /dev/rdsk/c2t2d0s6

SQL> show parameter asm
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups                       string      DG05, DG06
asm_diskstring                       string      /dev/rdsk/c2t1d0s6, /dev/rdsk/
                                                 c2t2d0s6
asm_power_limit                      integer     1

After that I could create a profile in SMO just fine.

Thank you for your help!

Bye

Frank

[jessick]

Possibly unrelated, but you appear to be running Solaris 10U7 and do not have the iSCSI Host Utilities 5.0 installed.

Just so you know this configuration is still being qualified and is not yet on our support matrix. These are the checks that likely caused ConfigChecker to report no supported configs found on your system

-Mark

[stephenb]

So are symbolic links to ASM disks supported in LUN environments?  I see in the best practice guide they are not with NFS, but no mention for LUN's anywhere.

Best Practices Guide page 13... “NetApp recommends not using symbolic links to the location of the database datafiles if linked from local storage to NFS. SnapManager does support symbolic links but only at the mountpoint level.”