DFM database backup fails

skellner · ‎2012-01-11

Hi all,

we did a new setup of our OnCommand Server on a new hardware. Database and performance data resides on LUN that we used again. Since two days our backup fails with the following error

Jan 09 22:00:17 [dfm:ERROR]: [3994:0x7fba67c646f0]: Could not get size of performance archive directory /om-perfdata. Please make sure this directory exists.

Jan 09 22:00:17 [dfm:ERROR]: [3994:0x7fba67c646f0]: Database backup create failed

Jan 10 22:00:04 [dfm:ERROR]: [28425:0x7ffe3ff3f6f0]: Could not get size of performance archive directory /om-perfdata. Please make sure this directory exists.

The directory exists and I can get the size with the df command on the linux host

v100spldv02:/opt/NTAPdfm/log # df -m /om-perfdata

Filesystem 1M-blocks Used Available Use% Mounted on

/dev/dm-3 515054 181826 307065 38% /om-perfdata

Any hints are appreciated. Thanks in advance.

Stefan

niels · ‎2012-01-11

Hi Stefan,

does any of the following apply:

- OnCommand 5

- DFM DB and/or Perf Data stored in a directory different from the default

- DFM DB and Perf Data reside on additional LUN

- SnapShot-based backup via SnapDrive

Could be BURT 560602

regards, Niels

adaikkap · ‎2012-01-11

Hi Stefan,

Could you pls create a case and add it to the Bug Id 560602 ?

skellner · ‎2012-01-12

It seems the issue is a bit different from the above bug. dfm about shows the right location for the perfdata. However dfm backup diag shows an error.

v100spldv02:/opt/NTAPdfm/log # dfm about

Version 5.0 (5.0)

Executable Type 64-bit

Serial Number 1-50-005803

Edition Standard edition of DataFabric Manager server

Administrator Name root

Host Name v100spldv02

Host IP Address 17.247.33.200

Host Full Name v100spldv02.v100.intern

Node limit 250 (currently managing 69)

Operating System SUSE Linux Enterprise Server 11 (x86_64) 2.6.27.19-5-default x86_64

CPU Count 8

System Memory 19955 MB (load excluding cached memory: 31%)

Installation Directory /opt/NTAPdfm

657 GB free (94.0%)

Perf Data Directory /om-perfdata

Data Export Directory /opt/NTAPdfm/dataExport

Database Backup Directory /opt/NTAPdfm/data

Reports Archival Directory /opt/NTAPdfm/reports

Database Directory /om-db

96.4 GB free (76.6%)

Database Log Directory /om-db

96.4 GB free (76.6%)

Licensed Features DataFabric Manager server: installed

Storage Area Network Option: installed

File SRM Option: installed

Installed Plugins Storage System Config 6.5.1 (6.5.1) - storage systems and vFilers

Storage System Config 6.5.2 (6.5.2) - storage systems and vFilers

Storage System Config 6.5.3 (6.5.3) - storage systems and vFilers

Storage System Config 6.5.4 (6.5.4) - storage systems and vFilers

Storage System Config 6.5.5 (6.5.5) - storage systems and vFilers

Storage System Config 6.5.6 (6.5.6) - storage systems and vFilers

Storage System Config 6.5.7 (6.5.7) - storage systems and vFilers

Storage System Config 7.0 (7.0.0.1) - storage systems and vFilers

Storage System Config 7.0.1 (7.0.1.1) - storage systems and vFilers

Storage System Config 7.0.2 (7.0.2) - storage systems and vFilers

Storage System Config 7.0.3 (7.0.3) - storage systems and vFilers

Storage System Config 7.0.4 (7.0.4) - storage systems and vFilers

Storage System Config 7.0.5 (7.0.5) - storage systems and vFilers

Storage System Config 7.0.6 (7.0.6) - storage systems and vFilers

Storage System Config 7.0.7 (7.0.7) - storage systems and vFilers

Storage System Config 7.1 (7.1.0.1) - storage systems and vFilers

Storage System Config 7.1.1 (7.1.1.1) - storage systems and vFilers

Storage System Config 7.1.2 (7.1.2.1) - storage systems and vFilers

Storage System Config 7.1.3 (7.1.3) - storage systems and vFilers

Storage System Config 7.2 (7.2) - storage systems and vFilers

Storage System Config 7.2.1 (7.2.1.1) - storage systems and vFilers

Storage System Config 7.2.2 (7.2.2) - storage systems and vFilers

Storage System Config 7.2.3 (7.2.3) - storage systems and vFilers

Storage System Config 7.2.4 (7.2.4) - storage systems and vFilers

Storage System Config 7.2.5 (7.2.5.1) - storage systems and vFilers

Storage System Config 7.2.6 (7.2.6.1) - storage systems and vFilers

Storage System Config 7.2.7 (7.2.7) - storage systems and vFilers

Storage System Config 7.3 (7.3) - storage systems and vFilers

Storage System Config 7.3.1 (7.3.1.1) - storage systems and vFilers

Storage System Config 7.3.2 (7.3.2) - storage systems and vFilers

Storage System Config 7.3.3 (7.3.3) - storage systems and vFilers

Storage System Config 7.3.4 (7.3.4) - storage systems and vFilers

Storage System Config 7.3.5 (7.3.5) - storage systems and vFilers

Storage System Config 8.0 (8.0) - storage systems and vFilers

Storage System Config 8.0.1 (8.0.1) - storage systems and vFilers

v100spldv02:/opt/NTAPdfm/log # dfm backup diag

Could not open directory: /om-perfdata/backup/dev/loop2.

Directory size for perf path failed

Sybase Data Directory: /om-db/

Sybase Log Directory : /om-db/

Performance Data Directory: /om-perfdata

Script-plugins Directory: /opt/NTAPdfm/script-plugins

Reports Archive Directory: /opt/NTAPdfm/reports/

Plugins Directory: /opt/NTAPdfm/plugins

Database Hosting System: localhost

SnapDrive Installed: No

Backup Destination: /opt/NTAPdfm/data/

Number of backups possible: NA

Backup Retention Count: 2

Allowed Backup Type: Archive (ndb)

adaikkap · ‎2012-01-12

This is a perfect signature of the bug. Pls add your case. Also can you get the output of dfm options list | grep -i perf

Regards

adai

skellner · ‎2012-01-12

Hi Adai,

here's the output.

v100spldv02:/om-perfdata/backup/dev # dfm options list | grep -i perf

autosupportIncludePerf Yes

perfAdvisorEnabled Enabled

perfAdvisorMaxMonitorThreads 32

perfAdvisorPollInterval 5 minutes

perfAdvisorShowAllViews Disabled

perfAdvisorShowDiagCounters Enabled

perfAdvisorTransport httpOnly

perfAdvThreshViolationMonInterval 15 minutes

perfArchiveDir /om-perfdata

perfDataExportEnabled No

perfExportDir /opt/NTAPdfm/perfExport

perfMaxObjectInstancesInBarChart 20

perfSampleRate1 1 minute

perfSampleRate2 5 minutes

perfSampleRate3 15 minutes

perfSampleRate4 30 minutes

Is there any workaround to get the backup back running? I will create the case now und let you know the case no.

Regards

Stefan

skellner · ‎2012-01-12

Case No. 2002808883

skellner · ‎2012-01-18

Hi again,

is there any workaround to get the backup back running? Currently we have no database backup.

Thanks in advance

skellner · ‎2012-01-19

I figured out a workaround for the database backup.

stop dfm services

start sql service

Copy perfdata back to local drive

change option perfarchivedir back to local drive

start dfm services

dfm backup create

After that you can copy the perfdata back to the Lun and the database backup keeps working.