snapcreator hangs in "Application unquiesce" phase

[ANUPMK1986]

Hi,

While running the snapshot backup for oracle database , it gets hung in Application unquiesce phase.

On checking the oracle process below were found

root     15185 14479  0 11:16 pts/0    00:00:00 /bin/su - ora<sid> -c ORACLE_HOME=/oracle/<sid>/112_64;export ORACLE_HOME;ORACLE_SID=<sid>;export ORACLE_SID;/oracle/<sid>/112_64/bin/sqlplus /nolog @/tmp/STqboyabQT.sc

ora<sid>   15186 15185  0 11:16 ?        00:00:00 /oracle/<sid>/112_64/bin/sqlplus        @/tmp/STqboyabQT.sc

The database tries to execute the command from the file /tmp/STqboyabQT.sc (alter system archive log current;) but it gets hung.

What has been done so far

1. Kill the above process and restart the snapshot backup again.
2. Restart the database

Both of the above have not helped in resolving the issue. Can you please guide in the right direction in resolving the issue?

Snapcreator version - 3.6

Oracle version  - 11.2.0.3

Regards,

Anup

[sivar]

Hello Anup,

Please provide us with the below details.

1) What is your operating System and version?

2) What shell is used for the Oracle user account?

3) Are you using this in a Oracle RAC environment?

4) Is RMAN configured?

5) What is the output of below SQL command?

    SQL> archive log list

5) Can you provide with below values from your snapcreator config file?

     ARCHIVE_LOG_ONLY=

     ARCHIVE_LOG_ENABLE=

     ARCHIVE_LOG_RETENTION=

     ARCHIVE_LOG_DIR=

     ARCHIVE_LOG_EXT=

5) Do you have Snap Creator agent logs available for this task?

6) If you could, please upload scdump (can be created using action->scdump from snapcreator GUI).

Thanks,
Siva Ramanathan

[ANUPMK1986]

Hi Siva,

Below are the details

1. OS - Linux

    Version - 6.4

2. Shell used - KSH

3. No

4. No

5.

SQL> archive log list

Database log mode              Archive Mode

Automatic archival             Enabled

Archive destination            /oracle/<SID>/oraarch/<SID>arch

Oldest online log sequence     432

Next log sequence to archive   432

Current log sequence           483

6.

ARCHIVE_LOG_ENABLE=N

ARCHIVE_LOG_RETENTION=1

ARCHIVE_LOG_DIR=/tmp/log

ARCHIVE_LOG_EXT=log

ARCHIVE_LOG_RECURSIVE_SEARCH=N

ARCHIVE_LOG_ONLY=N

I have attached the dump as requested

[sivar]

Hello Anup,

Thanks for the details. I could not figure much from the details.

Also, the scdump did not provide much clues.

Are you reusing a config file from an earlier version of SC?

If so, could you please try recreating the config file from scratch.

Any reason, why you are using 3.6p1

We have released 4.0 and 4.1 and late last week 4.1P1 versions.

I would recommend using 4.1p1 if that is a possibility.

Do these manual commands work and complete properly?

alter database end backup;

alter system archive log current;

alter database backup controlfile to trace as 'xyz_trace' reuse;

Meanwhile, if you want me to review this over a webex, please feel free to email me to sivar at netapp.com with your availability.

Thanks,
Siva Ramanathan

[ANUPMK1986]

Hi Siva,

We are not using the config file from the earlier version.

Version 3.6p1 is currently being used throughout the landscape. Not sure if we can upgrade. The snapshot backup works fine in other systems.

The command "alter system archive log current;" hangs while the other 2 commands work fine.

Regards,

Anup

[steiner]

Are you certain you aren’t out of space in the archive log destination? That’s the only reason I know that this command would hang.

[ANUPMK1986]

Hi Steiner,

The filesystem is not full. Moreover "alter system switch log file" works fine.

Regards,

Anup

[steiner]

A ‘switch logfile’ is a different operation. That’s not synchronous with an archive log operation, it simply changes the active log file.

Are you saying that ‘alter system archive log current’ is hanging at the sqlplus level too?

[ANUPMK1986]

Hi Steiner,

Yes , it is hanging at sqlplus as well

Regards,

Anup

[steiner]

That's definitely not an sc problem then. The only reason that should happen is a full archive log destination. The oracle alert log should show more details on the cause of the hang.

Sent from my mobile.