Active IQ Unified Manager Discussions

dfm interface slow

BrendonHiggins
6,981 Views

I have added 4 new filers to my estate and ops manager has become very slow.  There are only 15 filers in total and the DFM database monitordb.db is now 370 Mb in size on a high spec, single task server with 4Gb of ram.  This server is only used for Ops and performance manager and both apps are running standard monitoring and are not killing the box poling every second, etc.

Any ideas why the interface just sticks at "loading"?  Please see image.

We have also move some volumes between aggregates by using snapmirror and rename.

Thanks

Bren

12 REPLIES 12

RichardSopp
6,946 Views

Which version of DFM are you using?

BrendonHiggins
6,946 Views

3.7 - Sorry should have put it in the post.

dhank
6,946 Views

Can you provide the NMC logs for further debugging ? If you are able to reproduce this issue, set the  log level as ALL (default is ERROR) in the log4j.properties in the NMC home directory and send them.

BrendonHiggins
6,945 Views

Sorry I forgot about this issue but have started using DFM again and the problems are worse than ever.

The E:\Program Files\NetApp\DataFabric\DFM\data\monitordb.db is 395 Mb yet the dbsrv9.exe process only shows 353 Mb in windows task manager.

The windows service "DFM Sybase ASA" - DataFabric Manager database server starts the database with the switch below.

"E:\Program Files\NetApp\DataFabric\Sybase\ASA\win32\dbsrv9.exe" -hvDFMSybase

Has anyone played with the cache options to speed up the databse?

http://www.ianywhere.com/developer/product_manuals/sqlanywhere/0902/en/html/dbdaen9/00000146.htm

Thanks

Bren

pascalduk
6,945 Views

Maybe you already checked it, but is a virusscanner perhaps active?

Running on linux here: 28 filers, 1.8 GB database and 125 GB of PA data. All running smooth.

BrendonHiggins
6,946 Views

AV does not scan the database folder.

Thanks for the idea however.

Bren

BrendonHiggins
6,946 Views

Going to deploy v4 of Operations manager.  Many changes to improve performance in this release.

Bren

martin_fisher
6,945 Views

Just wndering how anyone got on with v4 of DFM and if it has improved performance.

was v4 installed on a Linux OS rather than windows.

Im having performance issues with W2K3 server running DFM v3.x monitoring approx 10 appliances, mostly it appears DB/SQL related.

Thanks.

Martin

pbhanu
6,945 Views

Hi Martin,

Can you paste the output of 'dfm about' in the DFM version that you are currently using.

Looks like in current version, there is not enough free space in the installation directory.

A sample output :

[root@lnx186-143 script]# dfm about
CPU Count                        1
System Memory                    2026 MB (load excluding cached memory: 69%)
Installation Directory           /opt/NTAPdfm
                                 2.17 GB free (17.1%)

Regards,

Bhanu

martin_fisher
6,381 Views

Hi Bhanu

As requested:

About DataFabric® Manager server
Version3.6.0.4808 (3.6)
Serial Number
Administrator Namenone
Host Name
Host IP Address
Host Full Name
Node Limit10
Operating SystemMicrosoft Windows 2003 Service Pack 2 (Build 3790)
CPU Count2
System Memory3839 MB (load: 26%)
Installation DirectoryE:/Program Files/DFM
8.07 GB free (20.2%)
Licensed FeaturesOperations Manager: installed
Business Continuance Option: installed
File SRM Option: installed
Protection Manager: your dataprote

adaikkap
6,381 Views

DFM version 3.8.1 and version 4.0 has lot of scalability and performance improvements.

Can you paste a dfm diag output of yours to see how big is your environment ?

Atleast the objects count part of the dfm diag output.

Regards

adai

martin_fisher
6,381 Views

Hi Adai

Bit of a big post, but most details

autosupport.installTime05 Oct 20072007-10-05 11:49:41.352000
autosupport.lastStatusSendTime05 Oct 20071290697983
autoThrottleMultiplier29 Nov 15:25100
generationId05 Oct 2007641
generationId.alonso05 Oct 200788
generationId.freya-reloaded05 Oct 2007104
generationId.gayatri05 Oct 200721
generationId.janneau05 Oct 200777
generationId.kahlua05 Oct 200716
history.rollup.aggrHistoryMonth29 Nov 15:35Timeslot 29 Nov 16:00: 28 records in 0 seconds (100000  rps)
history.rollup.aggrHistoryQuarter29 Nov 15:35Timeslot 30 Nov 00:00: 28 records in 0 seconds (100000  rps)
history.rollup.aggrHistoryWeek29 Nov 15:35Timeslot 29 Nov 16:00: 19 records in 0 seconds (100000  rps)
history.rollup.aggrHistoryYear29 Nov 15:35Timeslot 01 Dec 00:00: 28 records in 0 seconds (100000  rps)
history.rollup.cpuHistoryMonth29 Nov 15:29Timeslot 29 Nov 16:00: 14 records in 0 seconds (100000  rps)
history.rollup.cpuHistoryQuarter29 Nov 15:29Timeslot 30 Nov 00:00: 14 records in 0 seconds (100000  rps)
history.rollup.cpuHistoryWeek29 Nov 15:29Timeslot 29 Nov 16:00: 14 records in 0 seconds (100000  rps)
history.rollup.cpuHistoryYear29 Nov 15:29Timeslot 01 Dec 00:00: 14 records in 0 seconds (100000  rps)
history.rollup.dfHistoryMonth29 Nov 15:35Timeslot 29 Nov 16:00: 968 records in 0.109 seconds  (8881 rps)
history.rollup.dfHistoryQuarter29 Nov 15:35Timeslot 30 Nov 00:00: 986 records in 0.078 seconds  (12641 rps)
history.rollup.dfHistoryWeek29 Nov 15:35Timeslot 29 Nov 16:00: 706 records in 0.078 seconds  (9051 rps)
history.rollup.dfHistoryYear29 Nov 15:35Timeslot 01 Dec 00:00: 1014 records in 0.172 seconds  (5895 rps)
history.rollup.fcswHistoryMonth29 Nov 15:29Timeslot 29 Nov 16:00: 0 records in 0 seconds (100000  rps)
history.rollup.fcswHistoryQuarter29 Nov 15:29Timeslot 30 Nov 00:00: 0 records in 0 seconds (100000  rps)
history.rollup.fcswHistoryWeek29 Nov 15:29Timeslot 29 Nov 16:00: 0 records in 0 seconds (100000  rps)
history.rollup.fcswHistoryYear29 Nov 15:29Timeslot 01 Dec 00:00: 0 records in 0 seconds (100000  rps)
history.rollup.historyMonth29 Nov 15:23Timeslot 29 Nov 16:00: 402 records in 0.047 seconds  (8553 rps)
history.rollup.historyQuarter29 Nov 15:23Timeslot 30 Nov 00:00: 405 records in 0.063 seconds  (6429 rps)
history.rollup.historyWeek29 Nov 15:23Timeslot 29 Nov 16:00: 285 records in 0 seconds (100000  rps)
history.rollup.historyYear29 Nov 15:23Timeslot 01 Dec 00:00: 408 records in 0.078 seconds  (5231 rps)
history.rollup.ifHistoryMonth29 Nov 15:29Timeslot 29 Nov 16:00: 84 records in 0.031 seconds  (2710 rps)
history.rollup.ifHistoryQuarter29 Nov 15:29Timeslot 30 Nov 00:00: 84 records in 0 seconds (100000  rps)
history.rollup.ifHistoryWeek29 Nov 15:29Timeslot 29 Nov 16:00: 84 records in 0 seconds (100000  rps)
history.rollup.ifHistoryYear29 Nov 15:29Timeslot 01 Dec 00:00: 84 records in 0 seconds (100000  rps)
history.rollup.initiatorHistoryMonth29 Nov 15:29Timeslot 29 Nov 16:00: 0 records in 0 seconds (100000  rps)
history.rollup.initiatorHistoryQuarter29 Nov 15:29Timeslot 30 Nov 00:00: 0 records in 0 seconds (100000  rps)
history.rollup.initiatorHistoryWeek29 Nov 15:29Timeslot 29 Nov 16:00: 0 records in 0 seconds (100000  rps)
history.rollup.initiatorHistoryYear29 Nov 15:29Timeslot 01 Dec 00:00: 0 records in 0 seconds (100000  rps)
history.rollup.lunHistoryMonth29 Nov 15:35Timeslot 29 Nov 16:00: 343 records in 0 seconds (100000  rps)
history.rollup.lunHistoryQuarter29 Nov 15:35Timeslot 30 Nov 00:00: 347 records in 0.031 seconds  (11193 rps)
history.rollup.lunHistoryWeek29 Nov 15:35Timeslot 29 Nov 16:00: 343 records in 0.015 seconds  (22867 rps)
history.rollup.lunHistoryYear29 Nov 15:35Timeslot 01 Dec 00:00: 349 records in 0 seconds (100000  rps)
history.rollup.qtreeHistoryMonth29 Nov 15:05Timeslot 29 Nov 16:00: 0 records in 0 seconds (100000  rps)
history.rollup.qtreeHistoryQuarter29 Nov 15:05Timeslot 30 Nov 00:00: 0 records in 0 seconds (100000  rps)
history.rollup.qtreeHistoryWeek29 Nov 15:05Timeslot 29 Nov 16:00: 0 records in 0 seconds (100000  rps)
history.rollup.qtreeHistoryYear29 Nov 15:05Timeslot 01 Dec 00:00: 0 records in 0 seconds (100000  rps)
history.rollup.quotaHistoryMonth29 Nov 15:05Timeslot 29 Nov 16:00: 673 records in 0.187 seconds  (3599 rps)
history.rollup.quotaHistoryQuarter29 Nov 15:05Timeslot 30 Nov 00:00: 673 records in 0.156 seconds  (4314 rps)
history.rollup.quotaHistoryWeek29 Nov 15:05Timeslot 29 Nov 16:00: 0 records in 0 seconds (100000  rps)
history.rollup.quotaHistoryYear29 Nov 15:05Timeslot 01 Dec 00:00: 1522 records in 0.047 seconds  (32383 rps)
history.rollup.snapmirrorHistoryMonth29 Nov 15:35Timeslot 29 Nov 16:00: 109 records in 0 seconds (100000  rps)
history.rollup.snapmirrorHistoryQuarter29 Nov 15:35Timeslot 30 Nov 00:00: 109 records in 0 seconds (100000  rps)
history.rollup.snapmirrorHistoryWeek29 Nov 15:35Timeslot 29 Nov 16:00: 109 records in 0 seconds (100000  rps)
history.rollup.snapmirrorHistoryYear29 Nov 15:35Timeslot 01 Dec 00:00: 109 records in 0.031 seconds  (3516 rps)
history.rollup.vfilerHistoryMonth29 Nov 15:11Timeslot 29 Nov 16:00: 5 records in 0 seconds (100000  rps)
history.rollup.vfilerHistoryQuarter29 Nov 15:11Timeslot 30 Nov 00:00: 5 records in 0 seconds (100000  rps)
history.rollup.vfilerHistoryWeek29 Nov 15:11Timeslot 29 Nov 16:00: 5 records in 0.047 seconds (106  rps)
history.rollup.vfilerHistoryYear29 Nov 15:11Timeslot 01 Dec 00:00: 5 records in 0 seconds (100000  rps)
history.rollup.volHistoryMonth29 Nov 15:35Timeslot 29 Nov 16:00: 332 records in 0.015 seconds  (22133 rps)
history.rollup.volHistoryQuarter29 Nov 15:35Timeslot 30 Nov 00:00: 341 records in 0 seconds (100000  rps)
history.rollup.volHistoryWeek29 Nov 15:35Timeslot 29 Nov 16:00: 219 records in 0.031 seconds  (7065 rps)
history.rollup.volHistoryYear29 Nov 15:35Timeslot 01 Dec 00:00: 352 records in 0.016 seconds  (22000 rps)
history.stats.aggrHistory29 Nov 06:160.265 seconds
history.stats.cpuHistory29 Nov 03:130.333 seconds
history.stats.dfHistory29 Nov 04:154.7 seconds
history.stats.fcswHistory29 Nov 01:100 seconds
history.stats.history28 Nov 19:021.74 seconds
history.stats.ifHistory29 Nov 00:090.688 seconds
history.stats.initiatorHistory29 Nov 02:120 seconds
history.stats.lunHistory28 Nov 23:071.66 seconds
history.stats.qtreeHistory28 Nov 20:030 seconds
history.stats.quotaHistory28 Nov 21:0511.3 seconds
history.stats.snapmirrorHistory28 Nov 22:060.844 seconds
history.stats.vfilerHistory29 Nov 07:180.156 seconds
history.stats.volHistory29 Nov 05:162.62 seconds
monitor.dbck.checkObjFlags29 Nov 12:43succeeded=4 failed=0 elapsed=0s
monitor.dbck.consolidateHistory29 Nov 15:46succeeded=205 failed=0 elapsed=0s
monitor.dbck.deleteDupQuotaUsers29 Nov 15:37succeeded=76 failed=0 elapsed=0s
monitor.dbck.mgmtStationName29 Nov 15:31succeeded=21 failed=0 elapsed=0s
monitor.dbck.preCacheGroups29 Nov 15:45succeeded=77 failed=0 elapsed=0s
monitor.dbck.rebuildHistoryStatistics29 Nov 15:30succeeded=21 failed=0 elapsed=0s
monitor.dbck.removeOrphans29 Nov 15:09succeeded=21 failed=0 elapsed=83s
monitor.dbck.verifyDbcache29 Nov 15:33succeeded=21 failed=0 elapsed=0s
monitor.discover.network26 Oct 2007none
objectMap.verifier.current29 Nov 15:3371
postTransformLogicSuccess05 Oct 20071
postupgrade.dp_policies.create_samples05 Oct 2007no
Global Options
Option Name Option Value
agentHostCIFSAccount
agentHostLoginguest
agentHostPort
agentHostTransporthttp
agentMonInterval00:02:00
aggrFullThreshold85
aggrFullThresholdInterval00:00:00
aggrNearlyFullThreshold75
aggrNearlyOvercommittedThreshold190
aggrOvercommittedThreshold200
aggrSnapshotFullThreshold90
aggrSnapshotNearlyFullThreshold80
alertFromOperationsManager
apacheUserName
auditLogEnabledYes
auditLogForeverNo
authUsePamno
autosupportAdminContact
autosupportContentcomplete
autosupportDestinationEmailautosupport@netapp.com
autosupportDestinationURLsupport.netapp.com/asupprod/post/1.0/postAsup
autosupportEnabledYes
autosupportMonInterval00:02:00
autosupportProtocolhttps
autosupportRetryCount3
autosupportRetryDelay00:15:00
backupDirMonInterval08:00:00
backupRetentionCount1
cacheAdminPort3132
cacheAdminUseradmin
cacheClearLocalConfigno
cacheMonInterval00:05:00
cacheSecureAdminPort3443
cacheUseDfmHostnameno
ccMonInterval04:00:00
cfMonInterval00:05:00
chargebackDayOfMonth1
chargebackIncrementDaily
chargebackRate8.54
cmsAclEnableNo
cmsDir
cpuBusyThresholdInterval00:15:00
cpuMonInterval00:05:00
cpuTooBusyThreshold95
credCacheTTL00:20:00
currencyFormat£ #,###.##
currentEventsCacheSize
databaseBackupDbengWaitTime600
databaseBackupDirE:\Program Files\DFM\data
dfMonInterval00:30:00
discoverAgentsNo
discoverAppInitEnabledYes
discoverAutoSnmpNo
discoverCacheAdminPort3132
discoverEnabledYes
discoverHostsNo
discoverHttpEnabledNo
discoverInterval00:15:00
discoverNetworksNo
discoverSANYes
discoverTimeout00:00:05
discoverVfilersYes
diskMonInterval04:00:00
dsConformanceMonInterval01:00:00
dsProtectionMonInterval00:15:00
envMonInterval00:05:00
eventsPurgeInterval00:00:00
favoriteMaxReports25
fcMonInterval00:05:00
filerConfigSaveLocalChangesyes
fsMonInterval00:15:00
groupTreeShowStatusEnabled
growthRateSensitivity2
guiRefreshInterval00:02:00
hbaportTooBusyThreshold90
hostAdminTransporthttp
hostEnclosureDiscoveryEventsDisabledForCluster
hostLoginProtocolrsh
hostPingMethodecho_snmp
hostRBACMonInterval1:00:00:00
httpEnabledYes
httpPort8080
httpsEnabledNo
httpsPort8081
ifMonInterval00:01:00
ldapBaseDN
ldapBindDN
ldapBindPass
ldapEnabledNo
ldapGid
ldapMemberuniqueMember
ldapUgidCN
ldapUidUID
ldapVersion3
licenseExpireWarningThreshold5
licenseMonInterval04:00:00
localHostName
lunMonInterval00:30:00
monMinFreeBytes10000.0
monMinFreePercent10.0
monSNMPRetries4
monSNMPTimeout5
ncfileDir
ndmpDataUseAllInterfaces0
ndmpMonInterval00:30:00
networkDiscoveryLimit15
nodesRemainingWarningThreshold1
opsMonInterval00:10:00
ownerEmailFieldNameownerEmail
perfAdvisorEnabledYes
perfAdvisorMaxMonitorThreads32
perfAdvisorPollInterval00:05:00
perfAdvisorShowAllViewsNo
perfAdvisorShowDiagCountersNo
perfAdvisorTransporthttpOnly
perfArchiveDirE:/Program Files/DFM/perfdata
pingMonInterval00:01:00
pingMonRetryDelay3
pingMonTimeout3
pluginsDir
pmQSMBackupPreferredYes
profileTTL45:00:00:00
qtreeFullThreshold90
qtreeFullThresholdInterval00:00:00
qtreeGrowthEventMinChangePct1
qtreeMonInterval08:00:00
qtreeNearlyFullThreshold80
recentMaxReports25
reportsArchiveDirF:\Archive
respoolFullThreshold90
respoolNearlyFullThreshold80
respoolSpaceMonInterval1:00:00
SANHostMonInterval00:05:00
SANHostMonSnapshotLUNsYes
sanManagerHostname
sanManagerLogin
scriptDir
scriptPath
serverHTTPEnabledYes
serverHTTPPort8088
serverHTTPSEnabledYes
serverHTTPSPort8488
shareMonInterval01:00:00
SMTPServerBackup
SMTPServerName
SMTPServerPort25
snapmirrorMonInterval00:30:00
snapshotMonInterval00:30:00
snapvaultMonInterval00:30:00
snmpTrapListenerEnabledYes
snmpTrapListenerPort162
snmpTrapRcvdMaxPerWindow250
snmpTrapRcvdWindowSize00:05:00
softwareDir
srmLargestDirs100
srmLargestFiles20
srmLeastRecentlyAccessedFiles20
srmLeastRecentlyModifiedFiles20
srmMonInterval00:10:00
srmRecentlyModifiedFiles20
statusMonInterval00:10:00
sysInfoMonInterval01:00:00
useHostsEquivNo
userEmailDefaultDomain
userEnableAlertsNo
userFullThreshold90
userNearlyFullThreshold80
userQuotaMonInterval1:00:00:00
vFilerMonInterval01:00:00
volFullThreshold90
volFullThresholdInterval00:00:00
volGrowthEventMinChangePct20
volNearlyFullThreshold85
volNearlyNoFirstSnapThreshold80
volNearlyOvercommittedThreshold95
volNoFirstSnapThreshold90
volOvercommittedThreshold100
volReserveDepletedThreshold90
volReserveNearlyDepletedThreshold80
volSnapshotCountThreshold250
volSnapshotFullThreshold90
volSnapshotTooOldThreshold42:00:00:00
Object Counts
Object Type Count
Administrator6
Aggregate56
DP Policy20
DP Schedule18
DP Throttle2
FCP Target12
Host15
Initiator Group93
Interface108
Lun Path450
Mgmt Station1
Mirror109
Network7
OSSV Hosts0
Primary Filers0
Qtree2
QuotaUser2614
report schedule11
Resource Group44
Role22
schedule6
Secondary Filers0
SnapMirror Rels194
SnapVault Rels0
UserQuota1522
vFilers5
Volume894
Zapi Hosts6
Monitoring Timestamps
Timestamp Name Interval Last Updated Error if older than ...
cacheTimestamp5 minutes
29 Nov 15:42
ccTimestamp4 hours
29 Nov 11:47
cfTimestamp5 minutes29 Nov 15:46Normal29 Nov 15:42
cpuTimestamp5 minutes29 Nov 15:47Normal29 Nov 15:42
dfTimestamp30 minutes29 Nov 15:45Normal29 Nov 15:17
diskTimestamp4 hours29 Nov 15:42Normal29 Nov 11:47
envTimestamp5 minutes29 Nov 15:47Normal29 Nov 15:42
fcTimestamp5 minutes29 Nov 15:46Normal29 Nov 15:42
fsTimestamp15 minutes29 Nov 15:46Normal29 Nov 15:32
hostPingTimestamp1 minute29 Nov 15:47Normal29 Nov 15:46
ifTimestamp1 minute29 Nov 15:47Normal29 Nov 15:46
licenseTimestamp4 hours29 Nov 15:37Normal29 Nov 11:47
lunTimestamp30 minutes29 Nov 15:30Normal29 Nov 15:17
opsTimestamp10 minutes29 Nov 15:46Normal29 Nov 15:37
qtreeTimestamp8 hours29 Nov 11:38Normal29 Nov 07:47
rbacTimestamp1 day29 Nov 11:36Normal28 Nov 15:47
userQuotaTimestamp1 day29 Nov 12:06Normal28 Nov 15:47
sanhostTimestamp5 minutes
29 Nov 15:42
snapmirrorTimestamp30 minutes29 Nov 15:46Normal29 Nov 15:17
snapshotTimestamp30 minutes29 Nov 15:46Normal29 Nov 15:17
statusTimestamp10 minutes29 Nov 15:46Normal29 Nov 15:37
sysInfoTimestamp1 hour29 Nov 15:41Normal29 Nov 14:47
svTimestamp30 minutes
29 Nov 15:17
svMonTimestamp8 hours
29 Nov 07:47
xmlQtreeTimestamp8 hours29 Nov 13:42Normal29 Nov 07:47
vFilerTimestamp1 hour29 Nov 15:42Normal29 Nov 14:47
Database
monitordb.db2.73 GB
dbFileVersion9
ConnCount25 connections
MaxCacheSize523264 KBytes
CurrentCacheSize523264 KBytes
PeakCacheSize523264 KBytes
PageSize8192 Bytes
Logs
discovery 140 KB26 Oct 2007
DFMMonitor 2.40 MB29 Nov 15:45
DFMEvent 2.78 MB29 Nov 10:33
DFMServer 1.08 MB29 Nov 11:19
DFMScheduler 222 KB30 Oct 14:27
DFMWatchDog 1.16 MB29 Nov 15:47
dfm 297 KB29 Nov 15:40
sybase 5.00 MB29 Nov 15:13
pingmon 205 KB28 Nov 21:00
audit 318 KB29 Nov 15:47
vfmon 46.4 KB22 Aug 13:27
Services
sqlNormal Started
httpNormal Started
eventdNormal Started
monitorNormal Started
schedulerNormal Started
serverNormal Started
watchdogNormal Started
Time Since Confirmed Alive
Eventd5 seconds
Monitor2 seconds
Scheduler12 seconds
Server51 seconds
Watchdog1 second
Events
DP Job Information
Job State Count
Jobs Running0
Jobs Completed Total0
Jobs Aborted Total0
Jobs Aborting Total0
Jobs Completed Today0
Jobs Aborted Today0
Jobs Aborting Today0
Data Set Protection Status
Protection State Count
Protected0
Unprotected0
Version Numbers
Program Version
Version3.6.0.4808 (3.6)
Public