ONTAP Discussions

OMF and netapp plugin

gasparuben
9,712 Views

Hi,

 

It turns out we are using OMF for both PDB$SEED and our PDB's. While trying to use the Netapp plug-in I get:

 

--using different paths

sys@DODCDB2:SQL> create pluggable database RUBEN04 from RUBEN03 file_name_convert=('/ORA/dbs03/RUBEN03','/ORA/dbs03/RUBEN04') snapshot copy;

create pluggable database RUBEN04 from RUBEN03 file_name_convert=('/ORA/dbs03/RUBEN03','/ORA/dbs03/RUBEN04') snapshot copy

*

ERROR at line 1:

ORA-01276: Cannot add file /ORA/dbs03/RUBEN04/DODCDB2/E9957ADE4BDF78E3E0438C08100A182D/datafile/o1_mf_system__179231698634_.dbf.  File has an Oracle Managed Files file name.

 

--looking for file

sys@DODCDB2:SQL> create pluggable database RUBEN04 from RUBEN03 file_name_convert=('__17923','totoclone') snapshot copy;

create pluggable database RUBEN04 from RUBEN03 file_name_convert=('__17923','totoclone') snapshot copy

*

ERROR at line 1:

ORA-01276: Cannot add file /ORA/dbs03/RUBEN03/DODCDB2/E9957ADE4BDF78E3E0438C08100A182D/datafile/o1_mf_systemtotoclone1698634_.dbf.  File has an Oracle Managed Files file name.

 

It looks like more a restriction on the "create pluggable database" command than in the plug-in, it also fails for an standard PDB creation from seed:

 

alter session set db_create_file_dest='';

create pluggable database RUBEN04 ADMIN USER ruben_admin IDENTIFIED BY ChangeMe13 ROLES=(DBA)

    DEFAULT TABLESPACE RUBEN DATAFILE '/ORA/dbs03/RUBEN05/datafile/myfile.dbf'  SIZE 250M AUTOEXTEND ON

          path_prefix='/ORA/dbs03/RUBEN05'

          file_name_convert=('/ORA/dbs03/DODCDB2/','/ORA/dbs03/RUBEN05/')

          TEMPFILE REUSE;

 

ERROR at line 1:

ORA-01276: Cannot add file /ORA/dbs03/RUBEN05/E97092FD38474BE4E0438C08100AAFFB/datafile/o1_mf_system__20748329356_.dbf.  File has an Oracle Managed Files file name.

 

sys@DODCDB2:SQL> @pdbs_data_files.sql

 

 

PDB_ID PDB_NAME        FILE_ID TABLESPACE_NAME                FILE_NAME

------ --------------- ------- ------------------------------ ------------------------------------------------------------------------------------------------------------------------

     2 PDB$SEED              6 USERTBS                        /ORA/dbs03/DODCDB2/E97092FD38474BE4E0438C08100AAFFB/datafile/o1_mf_usertbs__20802017065_.dbf

     2 PDB$SEED              4 SYSAUX                         /ORA/dbs03/DODCDB2/E97092FD38474BE4E0438C08100AAFFB/datafile/o1_mf_sysaux__20776046082_.dbf

     2 PDB$SEED              2 SYSTEM                         /ORA/dbs03/DODCDB2/E97092FD38474BE4E0438C08100AAFFB/datafile/o1_mf_system__20748329356_.dbf

 

alter session set db_create_file_dest='';

  Session altered.

  create pluggable database RUBEN04 ADMIN USER ruben_admin IDENTIFIED BY ChangeMe13 ROLES=(DBA)

    DEFAULT TABLESPACE RUBEN DATAFILE '/ORA/dbs03/RUBEN05/datafile/myfile.dbf'  SIZE 250M AUTOEXTEND ON

  path_prefix='/ORA/dbs03/RUBEN05'

  file_name_convert=('/ORA/dbs03/DODCDB2/E97092FD38474BE4E0438C08100AAFFB/datafile/','/ORA/dbs03/RUBEN05/datafile/')

  5    TEMPFILE REUSE;

create pluggable database RUBEN04 ADMIN USER ruben_admin IDENTIFIED BY ChangeMe13 ROLES=(DBA)

*

ERROR at line 1:

ORA-01276: Cannot add file /ORA/dbs03/RUBEN05/datafile/o1_mf_system__20748329356_.dbf.  File has an Oracle Managed Files file name.

 

 

Please let me know if there is something I am doing wrong.

Thanks for your time,

Ruben

1 ACCEPTED SOLUTION

gasparuben
9,711 Views

Hi Karthikeyan,

Sorry for my delay to get back to you.

db_create_file_dest='/ORA/dbs03/RUBEN'

sys@DODCDB1:SQL> create pluggable database ruben03_clone from ruben03 snapshot copy path_prefix = '/ORA/dbs03/RUBEN03_CLONE';

create pluggable database ruben03_clone from ruben03 snapshot copy path_prefix = '/ORA/dbs03/RUBEN03_CLONE'

*

ERROR at line 1:

ORA-00604: error occurred at recursive SQL level

ORA-17517: Database cloning using storage snapshot failed on file :

While looking at the trace files I see:

*** 2014-02-25 17:22:46.511

*** SESSION ID:(287.18491) 2014-02-25 17:22:46.511

*** SERVICE NAME:(SYS$USERS) 2014-02-25 17:22:46.511

*** MODULE NAME:(sqlplus@XXXXX.cern.ch (TNS V1-V3)) 2014-02-25 17:22:46.511

*** ACTION NAME:() 2014-02-25 17:22:46.511

*** CONTAINER ID:(1) 2014-02-25 17:22:46.511

Storage clones not supported on /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_iops__4498075797710_.dbf storage_type=0

ORA-17517: Database cloning using storage snapshot failed on file 1:/ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_iops__4498075797710_.dbf

The storage has a valid clone and snaprestore license.

Looking at the documentation I see that this error my be linked to: https://communities.netapp.com/servlet/JiveServlet/previewBody/30115-102-1-50847/NetApp%20Cloning%20Plug-in%201%200%20for%20Oracle%20Database%20Instal... page 25:

1) Verify that the alert_db.log contains “Found NAS server NetApp Release”.

2) If the alert_db.log does not contain “Found NAS server NetApp Release‖, perform the following

steps:

a) Install net-snmp-* rpms.

b) Run the snmpwalk command.

I certainly dont see this "Found NAS..." but the net-snmp rpms are installed. I hesitate installation of those will be reflected in the alert log. About the snmpwalk there is another reference on page 6 of the same document. But I have following remarks. I am working on C-mode cluster, please could you tell me where I should look for:

snmp.access all

snmp.enable on

options nfs.export.auto-update

My mount points look like this:

vserver-dbnasb402:/ORA/dbs03/RUBEN03 on /ORA/dbs03/RUBEN03 type nfs (rw,bg,hard,nointr,tcp,nfsvers=3,actimeo=0,timeo=600,rsize=65536,wsize=65536,addr=XXXXX)

sys@DODCDB1:SQL> select client from v$client_secrets;

CLIENT

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

vserver-dbnasb402

This is the lif I used to mount, and the credentials are defined using vsadmin user:

sys@DODCDB0:SQL> ADMINISTER KEY MANAGEMENT ADD SECRET 'xxxxx' FOR CLIENT 'vserver-dbnasb402' USING TAG 'vsadmin' IDENTIFIED BY xxx WITH BACKUP;

keystore altered.

Also the firewall of that lif is modified so snmp traffic goes through:

c02::> net int show -vserver dbvs -lif dbvs_lif3 -instance

  (network interface show)

                    Vserver Name: dbvs

          Logical Interface Name: dbvs_lif3

                            Role: data

                   Data Protocol: nfs

                       Home Node: dbnasb402

                       Home Port: a0c

                    Current Node: dbnasb402

                    Current Port: a0c

              Operational Status: up

                 Extended Status: -

                         Is Home: true

                 Network Address: XXXXXXX

                         Netmask: 255.255.255.192

             Bits in the Netmask: 26

                 IPv4 Link Local: -

              Routing Group Name: XXXXXXX/26

           Administrative Status: up

                 Failover Policy: nextavail

                 Firewall Policy: datandmgmt

                     Auto Revert: false

   Fully Qualified DNS Zone Name: vserver-dbnasb402.cern.ch

         DNS Query Listen Enable: true

             Failover Group Name: dbvs_10g_IP26

                        FCP WWPN: -

                  Address family: ipv4

                         Comment: -

c02::> firewall policy show -policy datandmgmt

  (system services firewall policy show)

Policy           Service    Action IP-List

---------------- ---------- ------ --------------------

datandmgmt

                 dns        allow  0.0.0.0/0

                 http       allow  0.0.0.0/0

                 https      allow  0.0.0.0/0

                 ndmp       allow  0.0.0.0/0

                 ntp        allow  0.0.0.0/0

                 snmp       allow  0.0.0.0/0

                 ssh        allow  0.0.0.0/0

                 telnet     deny   0.0.0.0/0

8 entries were displayed.

itrac>-RAC>-DODCDB11:/ORA/dbs01/oracle/product/rdbms/dbs$ snmpget -c public -v 2c vserver-dbnasb402 1.3.6.1.4.1.789.1.1.2.0

Timeout: No Response from vserver-dbnasb402.

Using the cluster management IP:

trac>-RAC>-DODCDB11:/ORA/dbs01/oracle/product/rdbms/dbs$ snmpget -c public -v 2c IPCLUSTERMANAGEMENT 1.3.6.1.4.1.789.1.1.2.0

SNMPv2-SMI::enterprises.789.1.1.2.0 = STRING: "NetApp Release 8.2P3 Cluster-Mode: Thu Sep 12 01:13:07 PDT 2013"

So please could you clarify how to enable snmp and check auto update exports on the vserver to which my data lifs belong to? How should I run snmpwalk?

Thank you,

Ruben

View solution in original post

12 REPLIES 12

nkarthik
9,679 Views

Good question, Let me check and get back to you.

Regards,

Karthikeyan.N

nkarthik
9,679 Views

Hello gasparuben,

Sorry for the delay, I got the answer for you. OMF with NetApp Plugin (snapshot copy ) is supported. You need to change the 'db_create_file_dest' parameter to NetApp storage mountpoint and no need to pass 'file_name_convert' with create pluggable database.

 

Please check the below link with an example ( Might be too detailed ) to show the OMF based PDB create and delete (Netapp flexclone PDB) using NetApp cloning Plug-In:

https://communities.netapp.com/docs/DOC-31695

Please let me know, if you need any more details.

Regards,

Karthikeyan.N

nkarthik
9,677 Views

Hello gasparuben,

Any update for your query. Does it answer your query? Please check the document and let me know, if you need any changes.

Regards,

Karthikeyan.N

gasparuben
9,712 Views

Hi Karthikeyan,

Sorry for my delay to get back to you.

db_create_file_dest='/ORA/dbs03/RUBEN'

sys@DODCDB1:SQL> create pluggable database ruben03_clone from ruben03 snapshot copy path_prefix = '/ORA/dbs03/RUBEN03_CLONE';

create pluggable database ruben03_clone from ruben03 snapshot copy path_prefix = '/ORA/dbs03/RUBEN03_CLONE'

*

ERROR at line 1:

ORA-00604: error occurred at recursive SQL level

ORA-17517: Database cloning using storage snapshot failed on file :

While looking at the trace files I see:

*** 2014-02-25 17:22:46.511

*** SESSION ID:(287.18491) 2014-02-25 17:22:46.511

*** SERVICE NAME:(SYS$USERS) 2014-02-25 17:22:46.511

*** MODULE NAME:(sqlplus@XXXXX.cern.ch (TNS V1-V3)) 2014-02-25 17:22:46.511

*** ACTION NAME:() 2014-02-25 17:22:46.511

*** CONTAINER ID:(1) 2014-02-25 17:22:46.511

Storage clones not supported on /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_iops__4498075797710_.dbf storage_type=0

ORA-17517: Database cloning using storage snapshot failed on file 1:/ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_iops__4498075797710_.dbf

The storage has a valid clone and snaprestore license.

Looking at the documentation I see that this error my be linked to: https://communities.netapp.com/servlet/JiveServlet/previewBody/30115-102-1-50847/NetApp%20Cloning%20Plug-in%201%200%20for%20Oracle%20Database%20Instal... page 25:

1) Verify that the alert_db.log contains “Found NAS server NetApp Release”.

2) If the alert_db.log does not contain “Found NAS server NetApp Release‖, perform the following

steps:

a) Install net-snmp-* rpms.

b) Run the snmpwalk command.

I certainly dont see this "Found NAS..." but the net-snmp rpms are installed. I hesitate installation of those will be reflected in the alert log. About the snmpwalk there is another reference on page 6 of the same document. But I have following remarks. I am working on C-mode cluster, please could you tell me where I should look for:

snmp.access all

snmp.enable on

options nfs.export.auto-update

My mount points look like this:

vserver-dbnasb402:/ORA/dbs03/RUBEN03 on /ORA/dbs03/RUBEN03 type nfs (rw,bg,hard,nointr,tcp,nfsvers=3,actimeo=0,timeo=600,rsize=65536,wsize=65536,addr=XXXXX)

sys@DODCDB1:SQL> select client from v$client_secrets;

CLIENT

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

vserver-dbnasb402

This is the lif I used to mount, and the credentials are defined using vsadmin user:

sys@DODCDB0:SQL> ADMINISTER KEY MANAGEMENT ADD SECRET 'xxxxx' FOR CLIENT 'vserver-dbnasb402' USING TAG 'vsadmin' IDENTIFIED BY xxx WITH BACKUP;

keystore altered.

Also the firewall of that lif is modified so snmp traffic goes through:

c02::> net int show -vserver dbvs -lif dbvs_lif3 -instance

  (network interface show)

                    Vserver Name: dbvs

          Logical Interface Name: dbvs_lif3

                            Role: data

                   Data Protocol: nfs

                       Home Node: dbnasb402

                       Home Port: a0c

                    Current Node: dbnasb402

                    Current Port: a0c

              Operational Status: up

                 Extended Status: -

                         Is Home: true

                 Network Address: XXXXXXX

                         Netmask: 255.255.255.192

             Bits in the Netmask: 26

                 IPv4 Link Local: -

              Routing Group Name: XXXXXXX/26

           Administrative Status: up

                 Failover Policy: nextavail

                 Firewall Policy: datandmgmt

                     Auto Revert: false

   Fully Qualified DNS Zone Name: vserver-dbnasb402.cern.ch

         DNS Query Listen Enable: true

             Failover Group Name: dbvs_10g_IP26

                        FCP WWPN: -

                  Address family: ipv4

                         Comment: -

c02::> firewall policy show -policy datandmgmt

  (system services firewall policy show)

Policy           Service    Action IP-List

---------------- ---------- ------ --------------------

datandmgmt

                 dns        allow  0.0.0.0/0

                 http       allow  0.0.0.0/0

                 https      allow  0.0.0.0/0

                 ndmp       allow  0.0.0.0/0

                 ntp        allow  0.0.0.0/0

                 snmp       allow  0.0.0.0/0

                 ssh        allow  0.0.0.0/0

                 telnet     deny   0.0.0.0/0

8 entries were displayed.

itrac>-RAC>-DODCDB11:/ORA/dbs01/oracle/product/rdbms/dbs$ snmpget -c public -v 2c vserver-dbnasb402 1.3.6.1.4.1.789.1.1.2.0

Timeout: No Response from vserver-dbnasb402.

Using the cluster management IP:

trac>-RAC>-DODCDB11:/ORA/dbs01/oracle/product/rdbms/dbs$ snmpget -c public -v 2c IPCLUSTERMANAGEMENT 1.3.6.1.4.1.789.1.1.2.0

SNMPv2-SMI::enterprises.789.1.1.2.0 = STRING: "NetApp Release 8.2P3 Cluster-Mode: Thu Sep 12 01:13:07 PDT 2013"

So please could you clarify how to enable snmp and check auto update exports on the vserver to which my data lifs belong to? How should I run snmpwalk?

Thank you,

Ruben

nkarthik
9,679 Views

Hello Ruben,

In your case, Please do the following:

1. create a LIF in vserver for public access using below

cluster1::> network interface create –vserver vs1_dnfs_rac –lif vs1_public_access -role data -home-node TESO-03 -home-port e2a-1164 –address 10.63.164.19 -netmask 255.255.255.0 -failover-policy nextavail -firewall-policy mgmt -auto-revert false

2. check the Lif properties

TESO::> n i show -vserver vs1_dnfs_rac  -lif vs1_public_access

  (network interface show)

                    Vserver Name: vs1_dnfs_rac

          Logical Interface Name: vs1_public_access

                            Role: data

                   Data Protocol: nfs, cifs, fcache

                       Home Node: TESO-03

                       Home Port: e2a-1164

                    Current Node: TESO-03

                    Current Port: e2a-1164

              Operational Status: up

                 Extended Status: -

                         Is Home: true

                 Network Address: 10.63.164.19

                         Netmask: 255.255.255.0

             Bits in the Netmask: 24

                 IPv4 Link Local: -

              Routing Group Name: d10.63.164.0/24

           Administrative Status: up

                 Failover Policy: nextavail

                 Firewall Policy: mgmt

                     Auto Revert: false

   Fully Qualified DNS Zone Name: none

         DNS Query Listen Enable: false

             Failover Group Name: system-defined

                        FCP WWPN: -

                  Address family: ipv4

                         Comment:

3. enable snmp for vs1_dnfs_rac

TESO::> security login create -username public -application snmp -authmethod community -role vsadmin-readonly -vserver vs1_dnfs_rac

4. check Snmp access for vserver new lif(10.63.164.19)


[oracle@RHEL64 ~]$ snmpget -c public -v 2c 10.63.164.19 1.3.6.1.4.1.789.1.1.2.0

SNMPv2-SMI::enterprises.789.1.1.2.0 = STRING: "NetApp Release 8.2.1RC1 Cluster-Mode: Wed Nov 20 21:01:05 PST 2013"

[oracle@RHEL64 ~]$

Please let me know, if you need any help.

Regards,

Karthikeyan.N

gasparuben
9,679 Views

Hi Karthikeyan,

That helped. Thanks.

I am getting now this error:

sys@DODCDB1:SQL>  alter session set db_create_file_dest='/ORA/dbs03/RUBEN03';

Session altered.

sys@DODCDB1:SQL>  create pluggable database ruben03_clone from ruben03 snapshot copy;

create pluggable database ruben03_clone from ruben03 snapshot copy

*

ERROR at line 1:

ORA-65028: Unable to open plugin data file at path /ORA/dbs03/RUBEN03/DODCDB1/F37C479FCE7FCF05E043A506100A84E2/datafile/o1_mf_system_9k1myvhs_.dbf

ORA-17503: ksfdopn:6 Failed to open file /ORA/dbs03/RUBEN03/DODCDB1/F37C479FCE7FCF05E043A506100A84E2/datafile/o1_mf_system_9k1myvhs_.dbf

ORA-17500: ODM err:KGNFS READ FAIL:Invalid argument

I have the database open in read only mode on just one node, on the other node of the Oracle cluster it's closed.  I have tried with different db_create_file_dest but I get always the same error.

Our directory structure is like:

--CDB mounts

dbnast1332-priv:/CRS/dbs03/DODCDB1 on /CRS/dbs03/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/CRS/dbs00/DODCDB1 on /CRS/dbs00/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs02/DODCDB1 on /ORA/dbs02/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs0a/DODCDB1 on /ORA/dbs0a/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs03/DODCDB1 on /ORA/dbs03/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs04/DODCDB1 on /ORA/dbs04/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/CRS/dbs02/DODCDB1 on /CRS/dbs02/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs00/DODCDB1 on /ORA/dbs00/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

--PDB mounts

db-dbnasb402:/ORA/dbs03/RUBEN03 on /ORA/dbs03/RUBEN03 type nfs (rw,bg,hard,nointr,tcp,nfsvers=3,actimeo=0,timeo=600,rsize=65536,wsize=65536,addr=10.16.128.201)

db-dbnasb402:/ORA/dbs03/RUBEN01 on /ORA/dbs03/RUBEN01 type nfs (rw,bg,hard,nointr,tcp,nfsvers=3,actimeo=0,timeo=600,rsize=65536,wsize=65536,addr=10.16.128.201)

db-dbnasb402:/ORA/dbs03/RUBEN02 on /ORA/dbs03/RUBEN02 type nfs (rw,bg,hard,nointr,tcp,nfsvers=3,actimeo=0,timeo=600,rsize=65536,wsize=65536,addr=10.16.128.138)

Here all the availble data files:

sys@DODCDB1:SQL> l

  1  SELECT p.PDB_ID, p.PDB_NAME, d.FILE_ID, d.TABLESPACE_NAME, d.FILE_NAME

  2    FROM DBA_PDBS p, CDB_DATA_FILES d

  3    WHERE p.PDB_ID = d.CON_ID

  4*   ORDER BY p.PDB_ID

PDB_ID PDB_NAME        FILE_ID TABLESPACE_NAME                FILE_NAME

------ --------------- ------- ------------------------------ ------------------------------------------------------------------------------------------------------------------------

     2 PDB$SEED              4 SYSAUX                         /ORA/dbs03/DODCDB1/EAA8BEE1715CE8AAE043A506100AA1BF/datafile/o1_mf_sysaux__1448002357603_.dbf

     2 PDB$SEED              2 SYSTEM                         /ORA/dbs03/DODCDB1/EAA8BEE1715CE8AAE043A506100AA1BF/datafile/o1_mf_system__1447994720172_.dbf

     2 PDB$SEED              6 USERTBS                        /ORA/dbs03/DODCDB1/EAA8BEE1715CE8AAE043A506100AA1BF/datafile/o1_mf_usertbs__1448018775315_.dbf

     3 ANDREI_PDB           11 USERTBS                        /ORA/dbs03/DODCDB1/EAEA588C9581F362E043A506100A36F8/datafile/o1_mf_usertbs__1729742098491_.dbf

     3 ANDREI_PDB           10 SYSAUX                         /ORA/dbs03/DODCDB1/EAEA588C9581F362E043A506100A36F8/datafile/o1_mf_sysaux__1729524029687_.dbf

     3 ANDREI_PDB            9 SYSTEM                         /ORA/dbs03/DODCDB1/EAEA588C9581F362E043A506100A36F8/datafile/o1_mf_system__1729742100341_.dbf

     4 ANDREI_PDB2          14 USERTBS                        /ORA/dbs03/DODCDB1/EAEA588C9582F362E043A506100A36F8/datafile/o1_mf_usertbs__1729883100555_.dbf

     4 ANDREI_PDB2          12 SYSTEM                         /ORA/dbs03/DODCDB1/EAEA588C9582F362E043A506100A36F8/datafile/o1_mf_system__1729883118573_.dbf

     4 ANDREI_PDB2          13 SYSAUX                         /ORA/dbs03/DODCDB1/EAEA588C9582F362E043A506100A36F8/datafile/o1_mf_sysaux__1729883101209_.dbf

     5 RUBEN02              22 SYSAUX                         /ORA/dbs03/RUBEN02/DODCDB1/EB11E8A3070EC232E043A506100A5F02/datafile/o1_mf_sysaux__1899663749988_.dbf

     5 RUBEN02              37 TOOLS                          /ORA/dbs03/DODCDB1/EB11E8A3070EC232E043A506100A5F02/datafile/o1_mf_tools__4226018417178_.dbf

     5 RUBEN02              36 DBOD                           /ORA/dbs03/DODCDB1/EB11E8A3070EC232E043A506100A5F02/datafile/o1_mf_dbod__4226017809732_.dbf

     5 RUBEN02              24 RUBEN                          /ORA/dbs03/RUBEN02/DODCDB1/EB11E8A3070EC232E043A506100A5F02/datafile/o1_mf_ruben__1899670039558_.dbf

     5 RUBEN02              23 USERTBS                        /ORA/dbs03/RUBEN02/DODCDB1/EB11E8A3070EC232E043A506100A5F02/datafile/o1_mf_usertbs__1899663740574_.dbf

     5 RUBEN02              21 SYSTEM                         /ORA/dbs03/RUBEN02/DODCDB1/EB11E8A3070EC232E043A506100A5F02/datafile/o1_mf_system__1899663720533_.dbf

     5 RUBEN02              39 IOPS                           /ORA/dbs03/RUBEN02/DODCDB1/EB11E8A3070EC232E043A506100A5F02/datafile/o1_mf_iops__4495986687838_.dbf

     6 RUBEN01              38 IOPS                           /ORA/dbs03/RUBEN01/DODCDB1/EB11E8A3070FC232E043A506100A5F02/datafile/o1_mf_iops__4487547997417_.dbf

     6 RUBEN01              33 TPCCTAB                        /ORA/dbs03/RUBEN01/DODCDB1/EB11E8A3070FC232E043A506100A5F02/datafile/o1_mf_tpcctab__3888591021276_.dbf

     6 RUBEN01              28 RUBEN                          /ORA/dbs03/RUBEN01/DODCDB1/EB11E8A3070FC232E043A506100A5F02/datafile/o1_mf_ruben__1899690893913_.dbf

     6 RUBEN01              27 USERTBS                        /ORA/dbs03/RUBEN01/DODCDB1/EB11E8A3070FC232E043A506100A5F02/datafile/o1_mf_usertbs__1899686875908_.dbf

     6 RUBEN01              25 SYSTEM                         /ORA/dbs03/RUBEN01/DODCDB1/EB11E8A3070FC232E043A506100A5F02/datafile/o1_mf_system__1899686875586_.dbf

     6 RUBEN01              26 SYSAUX                         /ORA/dbs03/RUBEN01/DODCDB1/EB11E8A3070FC232E043A506100A5F02/datafile/o1_mf_sysaux__1899686880864_.dbf

     7 RUBEN03              29 SYSTEM                         /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_system__2683129838730_.dbf

     7 RUBEN03              30 SYSAUX                         /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_sysaux__2683129851875_.dbf

     7 RUBEN03              31 USERTBS                        /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_usertbs__2683129862767_.dbf

     7 RUBEN03              32 RUBEN                          /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_ruben__2683144897824_.dbf

     7 RUBEN03              34 DBOD                           /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_dbod__4152914319944_.dbf

     7 RUBEN03              35 TOOLS                          /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_tools__4152915012393_.dbf

     7 RUBEN03              40 IOPS                           /ORA/dbs03/RUBEN03/DODCDB1/EBC852DD72A67D43E043A506100AA921/datafile/o1_mf_iops__4498075797710_.dbf

This is what I see in the alert log for this instance:

create pluggable database ruben03_clone from ruben03 snapshot copy

create pluggable database ruben03_clone from ruben03 snapshot copy

Fri Feb 28 19:08:20 2014

Direct NFS: Read FAIL status  22

Fri Feb 28 19:08:20 2014

Direct NFS: NFSERR 305 error encountered. Server db-dbnasb402 path db-dbnasb402

ORA-65028 signalled during:  create pluggable database ruben03_clone from ruben03 snapshot copy...

Dont find anything in metalink (MOS) about this dnfs error.

The access to all mount points is via dnfs:

sys@DODCDB1:SQL> select * from v$dnfs_servers;

            ID

--------------

SVRNAME

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

DIRNAME

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

       MNTPORT        NFSPORT NFSVERSI          WTMAX          RTMAX         CON_ID

-------------- -------------- -------- -------------- -------------- --------------

             1

dbnast1332-priv

/ORA/dbs00/DODCDB1

           635           2049 NFSv3.0           65536          65536              0

             1

dbnast1332-priv

/ORA/dbs02/DODCDB1

           635           2049 NFSv3.0           65536          65536              0

             1

dbnast1332-priv

/ORA/dbs04/DODCDB1

           635           2049 NFSv3.0           65536          65536              0

             1

dbnast1332-priv

/ORA/dbs03/DODCDB1

           635           2049 NFSv3.0           65536          65536              0

             2

db-dbnasb402

/ORA/dbs03/RUBEN02

           635           2049 NFSv3.0           65536          65536              0

             2

db-dbnasb402

/ORA/dbs03/RUBEN01

           635           2049 NFSv3.0           65536          65536              0

             2

db-dbnasb402

/ORA/dbs03/RUBEN03

           635           2049 NFSv3.0           65536          65536              0

7 rows selected.

Please let me know.

Thanks for your support.

Cheers,

Ruben

nkarthik
9,679 Views

Ruben,

it look like your dnfs config issue. can you check you $ORACLE_HOME/oranfstab file content? I hope you have OMF file, please follow https://communities.netapp.com/servlet/JiveServlet/previewBody/31695-102-1-53361/OMF%20example.pdf

regards,

Karthikeyan.N

nkarthik
9,679 Views

Ruben,

Can you please also provide the $ORACLE_BASE/diag/rdbms/<sid>/<instance>/<sid>_p00*_<processid>.trc files and alert_<sid>.log file? Just want to know whether the basic communication is happening?

regards,

Karthikeyan.N

gasparuben
9,679 Views

Hi Karthikeyan,

I have created an oranfstab on both servers (I am using a RAC) at $ORACLE_HOME/dbs. I think this shouldnt be an issue though, I am positive I was always using dnfs.

server: dbnast1332-priv

path: 10.30.16.34

export: /CRS/dbs03/DODCDB mount: /CRS/dbs03/DODCDB

export: /CRS/dbs00/DODCDB mount: /CRS/dbs00/DODCDB

export: /CRS/dbs04/DODCDB mount: /CRS/dbs04/DODCDB

export: /CRS/dbs0a/DODCDB mount: /CRS/dbs0a/DODCDB

export: /CRS/dbs02/DODCDB mount: /CRS/dbs02/DODCDB

server: db-dbnasb402

path: 10.16.128.138

path: 10.16.128.201

export: /ORA/dbs03/RUBEN03 mount: /ORA/dbs03/RUBEN03

export: /ORA/dbs03/RUBEN02 mount: /ORA/dbs03/RUBEN02

export: /ORA/dbs03/RUBEN01 mount: /ORA/dbs03/RUBEN01

export: /CRS/dbs0a/DODCDB mount: /CRS/dbs0a/DODCDB

export: /ORA/dbs03/CSTEST12 mount: /ORA/dbs03/CSTEST12

server: db-dbnasb401

path: 10.16.128.136

path: 10.16.128.200

export: /ORA/dbs03/AIS12DEV mount: ORA/dbs03/AIS12DEV

Then after setting the PDB in read only and connecting to the container:

sys@DODCDB1:SQL> alter session set db_create_file_dest='/ORA/dbs03/RUBEN03';

Session altered.

sys@DODCDB1:SQL> create pluggable database ruben03_clone from ruben03 snapshot copy;

create pluggable database ruben03_clone from ruben03 snapshot copy

*

ERROR at line 1:

ORA-65028: Unable to open plugin data file at path /ORA/dbs03/RUBEN03/DODCDB1/F4985F29D35B660CE043A506100AE5BD/datafile/o1_mf_system_9l6vk1d2_.dbf

ORA-17503: ksfdopn:6 Failed to open file /ORA/dbs03/RUBEN03/DODCDB1/F4985F29D35B660CE043A506100AE5BD/datafile/o1_mf_system_9l6vk1d2_.dbf

ORA-17500: ODM err:KGNFS READ FAIL:Invalid argument

In the trace file:

Fri Mar 14 22:04:27 2014

Direct NFS: NFSERR 305 error encountered. Server db-dbnasb402 path 10.16.128.138

ORA-65028 signalled during: create pluggable database ruben03_clone from ruben03 snapshot copy...

I have also enabled:

alter system set event= '10298 trace name context forever, level 5','19392 trace name context forever, level 8','19393 trace name context forever, level 1','19394 trace name context forever, level 8','19396 trace name context forever, level 12';

How do you want me to send you the tracefiles?

Thanks for your help,

Ruben


nkarthik
7,243 Views

Ruben,

I would like to fix your issues and provide more idea about the product. shall we have a webex with conference call on next week and fix your issues related to NetApp cloning plugin for 12c? Please send me your email and communication phone to nkarthik@netapp.com

Regards,

Karthikeyan.N

gasparuben
7,243 Views

Hi Karthikeyan,

The last build you provided me works nicely. Just to recap, my test environment is Oracle RAC 12.1 and it has following file system allocation:

--CDB

dbnast1332-priv:/CRS/dbs03/DODCDB1 on /CRS/dbs03/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/CRS/dbs00/DODCDB1 on /CRS/dbs00/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs02/DODCDB1 on /ORA/dbs02/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs0a/DODCDB1 on /ORA/dbs0a/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs03/DODCDB1 on /ORA/dbs03/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs04/DODCDB1 on /ORA/dbs04/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/CRS/dbs02/DODCDB1 on /CRS/dbs02/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

dbnast1332-priv:/ORA/dbs00/DODCDB1 on /ORA/dbs00/DODCDB1 type nfs (rw,bg,hard,nointr,tcp,vers=3,actimeo=0,timeo=600,rsize=32768,wsize=32768,addr=10.30.16.34)

--PDB

db-dbnasb402:/ORA/dbs03/RUBEN03 on /ORA/dbs03/RUBEN03 type nfs (rw,bg,hard,nointr,tcp,nfsvers=3,actimeo=0,timeo=600,rsize=65536,wsize=65536,addr=10.16.128.201)

PDB and CDB  volume are located on different C-mode clusters. We are using OMF (Oracle Management Files).

Creating a PDB clone of RUBEN03 PDB can be achieved:

--pdb RUBEN03 is READ ONLY, credentials have been set in a wallet, dnfs is in use, etc... as per doc

sys@DODCDB1:SQL> alter session set db_create_file_dest='/ORA/dbs03/RUBEN03';

sys@DODCDB1:SQL> create pluggable database RUBEN03_CLONE from RUBEN03 snapshot copy;

This creates the PDB as per doc. As we are working in a RAC we need to define the new clone and  mount it on the second node of the cluster, before we trying to open the PDB there:

--/etc/fstab

db-dbnasb402:/F587948BEADC14DAE043A906100AF48C on /ORA/dbs03/RUBEN03/.oranfsclone/F587948BEADC14DAE043A906100AF48C type nfs (rw,bg,hard,rsize=65536,wsize=65536,vers=3,nointr,timeo=600,tcp,addr=10.16.128.201)

Also to notice that clones are located at the root of the vserver root_vol.

Deletion also works flawlessly:

sys@DODCDB1:SQL> drop pluggable database RUBEN03_CLONE including datafiles;

 

Pluggable database dropped.

Thanks Karthikeyan for your valuable help!

Ruben

nkarthik
7,243 Views

Thanks a lot Ruben, it was a very nice experience and it works for RAC both PDB create and delete.

Please let me know, if you need any more help in future.

Regards,

Karthikeyan.N

Public