Performance Advisor data management - export/import

[mahala]

Hi All,

We are using Performance Advisor (PA) for filers' health monitoring, capturing performance data, and alerting if defined thresholds are met against any counter.

Performance data are stored for one year at filer level i.e default and 3months for volume level which we extended from default one week to three month.

DFM server is having total 174 filers out of which 55 filers are enabled for PA data capturing. We used to have performance issue on DFM server itself and sometime it doesn't respond at all and sometime responds very slow.

We noticed one thing that DFM server becomes sluggish once PA data grows much and then once we renamed/moved/removed PA data folder and start DFM server again it works very well. So looks problem is due to size of PA data but I am unable to understand why its linked with size of PA data.

Its understandable if no. of filers are more under PA monitoring but how size of PA data is causing server performance. Anyonce can comment on it if experienced same thing ?

Another question I have that if DFM server performance is really linked with size of PA data then is there any way to move out PA data on regular basis to a central location and could read by PA whenever need arises.

So I have two queries

1/ Does DFM server performance depend on size of PA data folder ?

2/ If I move out PA data folder to another locaation then is there any way to view graphs out of that data whenever need arises and then switch back to current one ?

Thanks & Regards,
Raju Singh Mahala

[anurag]

Hi Raju,

The size of the PA data can cause performance issues. Can you give me more details on the setup, here.

1. DFM version

2. Size of the PA data

3. Output of dfm about.

Do you find any specific reports or views coming up very slow ? What about the CPU and memory utilization of the dfm server ?

Unfortunately there is no way to progressively archive and purge PA data, and use it on demand for reporting and graphing.

You can export PA data into csv format periodically

Thanks,

Anurag

[mahala]

Hi Anurag,

Thanks for explanation so atleast its clear that there is no way to archive and purge PA data periodically and use it on demand.

Below are the info you requested

1. DFM version

3.7.1D10

2. Size of the PA data ( perfdata folder )

Current : 74GB, 1480Files   (This holds data since 6Apr09 as I had renamed this folder on 6Apr09. No performance degradation observed till date since then )

Past     :  279GB, 4911Files (At that time having very poor response from PA and even DFM as well when not working with PA)

PA data retention is :  one year at filer level and 3month at volume level

3. Output of dfm about

About DataFabric® Manager server
Version  3.7.1.6030 (3.7.1D10)
Serial Number  1-50-006041

Administrator Name  SHELL\raju-singh.mahala-a
Host Name  amsdc1-s-6503
Host IP Address  145.26.111.34
Host Full Name  amsdc1-s-6503.europe.shell.com
Node Limit  900 (currently managing 218)
Operating System  Microsoft Windows 2003 Enterprise Edition Service Pack 2 (Build 3790) x86 based
CPU Count  4
System Memory  4093 MB (load: 65%)
Installation Directory  E:/APPS/Network Appliance/DataFabric/DFM
67.5 GB free (13.5%)
Perf Data Directory  E:/Apps/Network Appliance/DataFabric/DFM/perfdata
Data Export Directory  E:/Apps/Network Appliance/DataFabric/DFM/dataExport
Database Backup Directory  E:\Apps\Network Appliance\DataFabric\DFM\data
Reports Archival Directory  E:\Apps\Network Appliance\DataFabric\DFM\reports
Licensed Features  Operations Manager: installed
Business Continuance Option: installed
Installed Plugins  Storage System Config 6.5.1 (6.5.1) - storage systems and vFilers
Storage System Config 6.5.2 (6.5.2) - storage systems and vFilers
Storage System Config 6.5.3 (6.5.3) - storage systems and vFilers
Storage System Config 6.5.4 (6.5.4) - storage systems and vFilers
Storage System Config 6.5.5 (6.5.5) - storage systems and vFilers
Storage System Config 6.5.6 (6.5.6) - storage systems and vFilers
Storage System Config 6.5.7 (6.5.7) - storage systems and vFilers
Storage System Config 7.0 (7.0.0.1) - storage systems and vFilers
Storage System Config 7.0.1 (7.0.1.1) - storage systems and vFilers
Storage System Config 7.0.2 (7.0.2) - storage systems and vFilers
Storage System Config 7.0.3 (7.0.3) - storage systems and vFilers
Storage System Config 7.0.4 (7.0.4) - storage systems and vFilers
Storage System Config 7.0.5 (7.0.5) - storage systems and vFilers
Storage System Config 7.0.6 (7.0.6) - storage systems and vFilers
Storage System Config 7.1 (7.1.0.1) - storage systems and vFilers
Storage System Config 7.1.1 (7.1.1.1) - storage systems and vFilers
Storage System Config 7.1.2 (7.1.2.1) - storage systems and vFilers
Storage System Config 7.2 (7.2) - storage systems and vFilers
Storage System Config 7.2.1 (7.2.1.1) - storage systems and vFilers
Storage System Config 7.2.2 (7.2.2) - storage systems and vFilers
Storage System Config 7.2.3 (7.2.3) - storage systems and vFilers
Storage System Config 7.2.4 (7.2.4) - storage systems and vFilers
Storage System Config 7.2.5 (7.2.5.1) - storage systems and vFilers
Storage System Config 7.2.6 (7.2.6.1) - storage systems and vFilers
Storage System Config 7.3.1 (7.3.1) - storage systems and vFilers
NetCache Config 5.3.1 (5.3.1R4) - NetCache appliances
NetCache Config 5.4 (5.4R3) - NetCache appliances
NetCache Config 5.5 (5.5R6) - NetCache appliances
NetCache Config 5.6 (5.6R1) - NetCache appliances
NetCache Config 5.6.1 (5.6.1) - NetCache appliances
NetCache Config 5.6.2 (5.6.2R1) - NetCache appliances
NetCache Config 6.0 (6.0) - NetCache appliances
NetCache Config 6.0.1 (6.0.1) - NetCache appliances
NetCache Config 6.0.2 (6.0.2) - NetCache appliances
NetCache Config 6.0.3 (6.0.3) - NetCache appliances
NetCache Config 6.0.4 (6.0.4) - NetCache appliances
NetCache Config 6.0.5 (6.0.5) - NetCache appliances
NetCache Config 6.1 (6.1RC2) - NetCache appliances
NetCache Config 6.1.1 (6.1.1RC1) - NetCache appliances

4/ Do you find any specific reports or views coming up very slow ? What about the CPU and memory utilization of the dfm server ?

Didn't notice anythings special and most of the find whole DFM responds slow against any click. Yes I noticed CPU and RAM utilization quite high of the server that time.

I have few more queires

1/ I couldn't understand how size of PA data affects performance of DFM server if we are not working with PA.

I understand if we are working with PA views then may affect the performance as huge data needs to load in memory.

2/ Can you provide some URLs or bestPractice guides to design a scalable and clean solution for DFM infrastructure with PA and having in mind that as of now around 200 filers need to monitor and will grow in future. Main goal is how to keep DFM infrastructure in control while using PA

3/ Can below workaround work ?

stop DFM service

rename "perdata" folder

start DFM service back

Then whenever need arises to look into old data

stop DFM service

point "perfArchiveDir" options to old "perfdata" folder

stop PA monitoring so it doesn't write anything in this perfdata folder

start DFM services back

once work done revert back to original

stop DFM service

point "perfArchiveDir" options back to current perfdata folder

start DFM services and PA monitoring

4/ As data can be exported in csv format on periodic basis from PA so Can we read those exported data back in PA and plot graph if required ?

Thanks & Regards,
Raju

[mahala]

Hi Guys,

Any comment on "impact of PA data size on DFM server performance".

If this is the case then how to design a scalable DFM infrastructure. Any ideas or things need to take into consideration during designing scalable solution for PA infra.

Anybody came across any technical doc talks about designing PA infra.

Thanks & Regards,
Raju

[harish]

Hi Raju,

I have not come across a  situation where the Performance Advisor slows down when large amounts of data are being collected. Can you please submit a burt? We will take a look.

In the meanwhile, I would like to understand your requirements at a high level like the list of counters you want to collect, how often you want to collect them, how long you want the data to be retained, the thresholds you have, etc. Based on these inputs, I will try to suggest ways to improve the scalability of Performance Advisor.

Regarding your other question on renaming performance advisor folder and then re-using it later, note that this is not supported.

Let me know if you have any other queries.

Regards

Harish

[mahala]

Hi Harish,

Ok I will open RFE as you requested but my intension is to first find out how "PA Data size" may affects the performance of DFM server because that may provide good direction to design scalable PA environment.

We haven't set any custom view so only default views but extended the data retention at volume level for 3months from default one week. So at filer level retention was one year and volume level retention was three months.

As of now around 200filers are registered in DFM and around 60filers out of those are enabled for PA.

Once PA data size grew more than 200GB, DFM server started performing very slow every now and then. It was not always slow but many time started responding very slow. So sometime we used to restart the services or sometime improved its own.

So after sometime we renamed the PA data folder by stoping DFM services and found DFM performing well again without any issues.

It will be really very helpful if you can provide some ideas to prepare scalable PA infra. Please let me know if any other info you want from me.

It will also be helpful if you can provide any technical docs as well.

Thanks & Regards,
Raju

[mahala]

Hi Harish,

I have created RFE#364629  as you asked for.

Thanks & Regards,
Raju