Proper method of moving a DFM database

[emanuel]

Hello all

I have a customer that has ran out of disk space on the local /OPT parition and are looking to add a new harddrive for extra space.  This new drive should work for them until a more long term disk array is available.  In the meantime I found this article on how to move the DFM database from one directory to another ( basiclly another HD on the system ) ... my question is ... what about the existing perfdata collected in the /perfdata directory?

If i shut down DFM services and move the monitordb/log files can I also move the /perfdata directory to the new location?

can i create a /data directory in the destination parition and have the monitor files there and a subfolder for the perfdata?

Here is an article on moving the database:

https://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb3841

To move the database and log files from one location to different, non-default install location on the same Operations Manager server, use the dfm database command:

1. If you are moving the database file location on the      same computer, stop all services and verify all services have stopped.
   
   dfm service stop
   dfm service list
2. Move the monitordb.db and monitordb.log files to the      new location using standard file copy tools.
3. Set the dbDir and dbLogDir locations.  The new      locations in the example commands are subsets of the default install path:
   
   Windows
   dfm database set dbDir="C:\Program      Files\NetApp\DataFabric\DFM\<new_db_location>"
   dfm database set dbLogDir="C:\Program      Files\NetApp\DataFabric\DFM\<new_db_location>"
   
   Unix
   dfm database set dbDir=/opt/NTAPdfm/<new_db_location>
   dfm database set dbLogDir=/opt/NTAPdfm/<new_db_location>
4. Start Operations Manager services and verify all      services have started.
   
   dfm service start
   dfm service list
5. Monitor the Operations Manager server.

The curious thing about moving the database here is that there is no mention of moving the perfdata directory and files.  I need to look in and see if there is any other steps needed.

[bipulm]

Hello Emanuel,

There is a global option "perfArchiveDir" in DFM which specifies the existing perfdata collection directory.

You can set this global to new location so that DFM consider perfdata directory to new location. If you execute the following

commands then you can  set the new location.

$dfm option list perfArchiveDir

output:

----------

option                                          value

----------------------------                --------------------------

perfArchiveDir                      /opt/NTAPdfm/perfdata

$dfm service stop server

$dfm option set perfArchiveDir=<new_location>

$dfm service start server

The above commands will do your work. You can move the perfdata to the new location, when dfm service is not running.

Your new location for perfdata can be any directory that you have created and set in the above global.

Thanks,

Bipul

[adaikkap]

Hi Emanuel,

     Pls use the dfm datastore setup command to move the dfm db and its contents.

# dfm datastore setup help

NAME
    setup -- configure DataFabric Manager data on a different location

SYNOPSIS
    dfm datastore setup [ -n ] [ -f ] { dfm-data-dir | [ -d dbDir ]
[ -l dbLogDir ] [ -p perfArchiveDir ]
[ -s scriptDir ] [ -r reportsArchiveDir ]
[ -P pluginsDir ] }

DESCRIPTION
    -n specifies that the data present at target location will be used
        without copying original data.
    -f specifies that the data should be deleted from target location if it is not empty.
    dfm-data-dir specifies DataFabric Manager target root directory for data.
    -d specifies the new location for database data file.
    -l specifies the new location for database transaction log file.
    -p specifies the new location for perf data files.
    -s specifies the new location for script output data.
    -r specifies the new location for report archival data.
    -P specifies the new location for Storage System configuration plugins.
    Example: dfm datastore setup /opt/dfmdata/
             dfm datastore setup -d /opt/dfm/data/ -p /opt/dfm/perf/ -s /opt/dfm/script/.

#

This will take care of starting or stopping the service if required.

Regards

adai

[emanuel]

I like the look of the datastore command, it seems for me if i can to move it from one parition to another, this is all i need to do

mount new storage and it is called locally "/dfmData"

dfm datastore setup /dfmData -d /dfmData/data -i /dfmData/data-p /dfmData/perf

so in this situation i will be moving "monitordb", "monitordb.log" to /dfmData/data and all the perf data to /dfmData/perf and leaving scripts, reports, and plugins alone.  When i execute this set up command, it will shutdown DFM services for me, copy or move the data, then restart services?

I am in the 4.0 DFM docs right now looking for more info about this.

[emanuel]

I tried to run this a few times on my test system

C:\Users\emanuel>dfm datastore setup c:\dfmData -d c:\dfmData\data -i c:\dfmData\data-p c:\dfmData\perf
The setup command takes either no arguments or one argument; you gave six.
Use "dfm help" for usage information.

C:\Users\emanuel>dfm datastore setup
Error: Destination directories are missing.
Use "dfm datastore setup help" for usage information.

C:\Users\emanuel>dfm datastore setup c:\dfmData
Creating the destination data directories.
Required space for data&colon;9.95 GB, Available space:20.4 GB
Stopping all services and jobs...
Service DFMSybase: Unable to connect to Service Control Manager: 5
Error: Failed to stop DataFabric Manager Database service.

I tried looking up service control manager 5 on the NOW site but nothing is coming up.

Looking more into the docs, the SCM suggests my server is a cluster of somekind which it is not; it is a stand alone W2K8 system.

[sinhaa]

I tried it on my windows 2003 server and it worked alright.

Can you try "dfm service list" and see if all the services are stopped. If not do stop it manually using "dfm service stop <service_name>". Try datastore setup again.

Paste the output if it still fails.

For me it worked in the first time.

D:\Documents and Settings\Administrator>dfm datastore setup C:\DFM_db
Creating the destination data directories.
Required space for data&colon;205 MB, Available space:28.6 GB
Stopping all services and jobs...
Copying database files to C:\DFM_db\data.
2 File(s) copied
Copying perf data files to C:\DFM_db\perfdata.
762 File(s) copied
Copying script output files to C:\DFM_db\script-plugins.
Copying report output files to C:\DFM_db\reports.
Copying plugins files to C:\DFM_db\plugins.
106 File(s) copied
Changing database configuration settings ...
Changed dbDir to C:\DFM_db\data.
Updated dbLogDir to C:\DFM_db\data.
Starting sql service...
Changed perfArchiveDir to C:\DFM_db\perfdata.
Changed scriptDir to C:\DFM_db\script-plugins.
Changed reportsArchiveDir to C:\DFM_db\reports.
Changed pluginsDir to C:\DFM_db\plugins.
Changed databaseBackupDir to C:\DFM_db\data.
Starting services...
Completed data setup.

D:\Documents and Settings\Administrator>

[emanuel]

Thats neat how it moves everything over for you.

My concerns are compounded now ( mainly out of interest )

My client is on Red Hat so the error message may not apply to this system; however I do work with W2K8 servers as well ( production ones ) so now i am curious how it will work on these.

I think I found the problem and its kinda a big one.  I was logged into the W2K8 host as a local user "emanuel" which is a member of the local Administrators group on a standalone W2K8 box; this should give me enough permissions to do anything on the box.  However DFM might be attached to the "Administrators/root" user names which in the windows world is a bad design; in the MSFT classes for NT and 2003 server, rule number one is to always copy the Administrator user to another name.  When you copy the user, you get an exactly replica of that user under a new name.  THis allows the admins to track activities based on users.  In DFM emanuel is GlobalFullControl ( should be, i dont think anyone changed it ).

So if it is true that DFM is hardcoded to do only certain things as Administrator or Root; it can conflict with a customer's security design.

As I wait, the copying is working under the Administrator context so I think we have a winner with this functionality.  I will confirm when it is done

[adaikkap]

Hi Emanuel,

     When you move your db you must move three things namely,

the db (both monitor.db and monitor.log),the perfdir and the scripts.

As a db backup archive based or snapshot based backup contains these there.

If all these are not on a LUN or local drive db backup will fail.

Now coming to yor question the syntax you used is incorrect.

I moved it in a RHEL.

[root@lnx~]# dfm datastore setup -d /adai/data -l /adai/data -p /adai/perf -s /adai/script
Creating the destination data directories.
Required space for data&colon;789 MB, Available space:115 GB
Stopping all services and jobs...
SQL Anywhere Stop Server Utility Version 10.0.1.3960
Copying database files to /adai/data.
Copying perf data files to /adai/perf.
Copying script output files to /adai/script.
Changing database configuration settings ...
Changed dbDir to /adai/data.
Updated dbLogDir to /adai/data.
Starting sql service...
Using service restart timeout = 180 seconds.
Changed perfArchiveDir to /adai/perf.
Changed scriptDir to /adai/script.
Changed databaseBackupDir to /adai/data.
Starting services...
[root@lnx~]#

Regards

adai

[emanuel]

Adai ... interesting ... and thanks for the RHEL look.

Here is my active copy right now in W2K8 ( just finished ) ... i used the simple syntax without the database item parameters.

C:\Users\Administrator>dfm service list
sql: started
http: started
eventd: started
monitor: started
scheduler: started
server: started
watchdog: started

C:\Users\Administrator>dfm datastore setup c:\dfmData
Creating the destination data directories.
Required space for data&colon;9.89 GB, Available space:20.4 GB
Stopping all services and jobs...
Copying database files to c:\dfmData\data.
7 File(s) copied
Copying perf data files to c:\dfmData\perfdata.
2064 File(s) copied
Copying script output files to c:\dfmData\script-plugins.
Copying report output files to c:\dfmData\reports.
672 File(s) copied
Copying plugins files to c:\dfmData\plugins.
110 File(s) copied
Changing database configuration settings ...
Changed dbDir to c:\dfmData\data.
Updated dbLogDir to c:\dfmData\data.
Starting sql service...
Changed perfArchiveDir to c:\dfmData\perfdata.
Changed scriptDir to c:\dfmData\script-plugins.
Changed reportsArchiveDir to c:\dfmData\reports.
Changed pluginsDir to c:\dfmData\plugins.
Changed databaseBackupDir to c:\dfmData\data.
Starting services...

C:\Users\Administrator>

[adaikkap]

Looks like the last time you did, dfm couldn’t stop the sql service for some reason.

Regards

adai

[sinhaa]

emanuel,

          Sometimes dfm service stop fails to stop all the servies ; more often the sql service than any other I've seen.

          And this time you tried user as "Administrator" ? If its failing to work when the user is logged in as something other than "Administartor/root",  who has all the rights then this is a problem. Lets me try this scenario and I'll update the results.

[emanuel]

yea i am curious ... it may have been a mix up with my session; I have two RDP windows open to the server; one as emanuel and one as administrator; in all my tests I try to stick with "emanuel"

[sinhaa]

Emanuel,

     I created a new user as "sinhaa" and added it to group of administrators. Then tried the 'dfm datastore setup ' cmd and it worked fine.

D:\Documents and Settings\sinhaa>dfm service list

sql: started

http: started

eventd: started

monitor: started

scheduler: started

server: started

watchdog: started

D:\Documents and Settings\sinhaa>dfm datastore setup c:\dfmData

Creating the destination data directories.

Required space for data&colon;205 MB, Available space:28.4 GB

Stopping all services and jobs...

Copying database files to c:\dfmData\data.

2 File(s) copied

Copying perf data files to c:\dfmData\perfdata.

763 File(s) copied

Copying script output files to c:\dfmData\script-plugins.

Copying report output files to c:\dfmData\reports.

Copying plugins files to c:\dfmData\plugins.

106 File(s) copied

Changing database configuration settings ...

Changed dbDir to c:\dfmData\data.

Updated dbLogDir to c:\dfmData\data.

Starting sql service...

Changed perfArchiveDir to c:\dfmData\perfdata.

Changed scriptDir to c:\dfmData\script-plugins.

Changed reportsArchiveDir to c:\dfmData\reports.

Changed pluginsDir to c:\dfmData\plugins.

Changed databaseBackupDir to c:\dfmData\data.

Starting services...

Completed data setup.

Maybe the case when service sql somehow failed to stop. Sometime I see that even during upgrade.

Anyways, I hope you got your problem solved.

Warm regards,

Abhishek

[emanuel]

there is two things here...

one .... the functionality works ... it copies the DB from one location to another, makes it the new active location, and leaves the old copy intact; this is a good thing.

two .... the windows user issue i had could be a fluke or a W2K8 thing ( but jumping on as the Administrator ) worked.  My client will be using ROOT on RHEL so I do not thing there will be a user permission issue to stop services.

I will keep an eye out for these in the future and see if it happens again.

Thanks guys for your help.  Always appreciate it

[sap_user1]

Can anybody help me with following queries in case if want to do roll back .

1. 1. if even after successful or unsuccessful datastore setup, the old data remains intact ?

1. 2. In case if we face any issue and want to revert back, what do we need to do ?

1. 3. What will we approximate time required to migrate the data…Currently we have 40GB data in /opt/NTAPdfm path.

[adaikkap]

Hi

Pls find my response inline.

1. if even after successful or unsuccessful datastore setup, the old data remains intact ?

Yes,dfm datastore setup only does a copy and the original version is still preserved in /opt/NTAPdfm by default or in appropriate windows directory.

2. In case if we face any issue and want to revert back, what do we need to do ?

Again use dfm datastore setup command with no copy options -n if I remember correctly and point to /opt/NTAPdfm

3. What will we approximate time required to migrate the data…Currently we have 40GB data in /opt/NTAPdfm path.

This is same as how long a copy of 40GB will take. There is dfm process involved in this.

BTW dfm datastore setup is time tested solutions and you should have no problems with it.

Regards

adai

[sap_user1]

Thanks Adai !!