DFM database backup oddities....

[emanuel]

Hello all

We started a DFM ( 3.7.1 ) backup from the GUI - Manual backup - ( preparing to upgrade to 3.8.1 ) ... and it took a real long time to complete and there was no mechanics to tell us what stage it was at.  What is curious is about 70 minutes into it, the temp file appeared and started running.  But the backup also failed because it was unable to clear out temp files.

Curious questions include:

1.     What is exactly happening during the backup that would delay creating the TEMP file?

2.     Is the database backup process the same on the CMD LINE vr the GUI vr the installer for an upgrade?

3.     On another DFM install, the monitor db is 5.1GB in size but the backups are 17 GB in size ( from the previous night )

4.     I know there is a method to clean a DFM database that greatly reduces the size by trimming stale data - this is only done by user interjection right?

We also happened to be watching the directory from NT explorer and the command prompt ... hopefully this did not interfere with the backup process.

Event		Database Backup Failed
Source		dfm-corp
Severity		Error
About		Success/Failure status of a database backup operation
Condition		Database backup e:/Program Files/Network   Appliance/DataFabric/DFM/data/dfm_backup_2010-05-13_15-13-26.ndb creation failed.   Reason: Could not cleanup backup temporary files.
Triggered		13 May 17:04
Acknowledged		No
Deleted		No

Directory of E:\Program Files\Network Appliance\DataFabric\DFM\data\dfm_temp_dest_dir

05/13/2010  04:13 PM    <DIR>          .

05/13/2010  04:13 PM    <DIR>          ..

05/13/2010  04:11 PM     3,601,186,816 monitordb.db

05/13/2010  04:11 PM        99,180,544 monitordb.log

05/13/2010  04:58 PM     2,516,612,890 tmp.backup.file.ndb

               3 File(s)  6,216,980,250 bytes

               2 Dir(s)  262,259,101,696 bytes free

E:\Program Files\Network Appliance\DataFabric\DFM\data\dfm_temp_dest_dir>dir

Volume in drive E has no label.

Volume Serial Number is 08E1-0BD0

Directory of E:\Program Files\Network Appliance\DataFabric\DFM\data\dfm_temp_dest_dir

05/13/2010  04:13 PM    <DIR>          .

05/13/2010  04:13 PM    <DIR>          ..

05/13/2010  04:11 PM     3,601,186,816 monitordb.db

05/13/2010  04:11 PM        99,180,544 monitordb.log

05/13/2010  05:02 PM     2,651,750,877 tmp.backup.file.ndb

               3 File(s)  6,352,118,237 bytes

               2 Dir(s)  262,251,560,960 bytes free

E:\Program Files\Network Appliance\DataFabric\DFM\data\dfm_temp_dest_dir>dir

Volume in drive E has no label.

Volume Serial Number is 08E1-0BD0

Directory of E:\Program Files\Network Appliance\DataFabric\DFM\data\dfm_temp_dest_dir

05/13/2010  04:13 PM    <DIR>          .

05/13/2010  04:13 PM    <DIR>          ..

05/13/2010  04:11 PM     3,601,186,816 monitordb.db

05/13/2010  04:11 PM        99,180,544 monitordb.log

05/13/2010  05:02 PM     2,655,322,203 tmp.backup.file.ndb

               3 File(s)  6,355,689,563 bytes

               2 Dir(s)  262,248,538,112 bytes free

E:\Program Files\Network Appliance\DataFabric\DFM\data\dfm_temp_dest_dir>

[adaikkap]

Hi Emanuel

>3. On another DFM install, the monitor db is 5.1GB in size but the backups are 17 GB in size ( from the previous night )

A dfm backup contains three things.

dfm db(both monitor.db and monitor.log)

perfdata dir

scripts-plugins( this contains the scripts and its output)

So you 5.1Gb might be the size of dfm db the difference of 12Gb should be your perfdata.

Regards

adai

[prasads]

1.     What is exactly happening during the backup that would delay creating the TEMP file?

[Prasad] A copy of the DB, LOG, Performance Data files & Script files are being created to be finally zipped into .ndb file

The copy of large data(>17GB) might take time. The zip process again takes some time. And finally there is the verification process.

All these are avoided in Snapshot Based backup.

2.     Is the database backup process the same on the CMD LINE vr the GUI vr the installer for an upgrade?

[Prasad] Yes, it is the same for all.

3.     On another DFM install, the monitor db is 5.1GB in size but the backups are 17 GB in size ( from the previous night )

[Prasad] See the size of perfdata folder. It might be more than 12GB in size.

4.     I know there is a method to clean a DFM database that greatly reduces the size by trimming stale data - this is only done by user interjection right?

[Prasad] The DFM DB experts will provide more accurate answer to this question.

We also happened to be watching the directory from NT explorer and the command prompt ... hopefully this did not interfere with the backup process.

On windows when you ‘cd’ to a dir from command prompt & access the directory

& at the same time you try to delete (read cleanup) that directory from a another session, Windows will not let you do so.

It throws an error

“The process cannot access the file because it is being used by another process.”

Instead of going into the directory, just do a dir <target-directory> from the root directory.

Have you tried Snapshot Based DFM backup ?

It is much much faster & might complete in few minutes.

Message was edited by: Prasad S

[emanuel]

I have been curious about the snapshot backup but it is only for an external file system right?  in otherwords no local disk.

[adaikkap]

Yes, its for NetApp Lun(iSCSI or FCP) it drastically brings down the db back time from hours or days to few minutes.

Regards

adai

[sanjyoth]

>> I know there is a method to clean a DFM database that greatly  reduces the size by trimming stale data - this is only done by user  interjection right?

      I guess there is no CLI command or any interfaces in WebUI available in DFM which does the trimming of any sort of stale data present in DB. So yes it requires users interjection to trim the same.