Operations Manager Efficiency Dashboard

[J_ABELLA]

I have just upgraded to the new Datafabric manager, Version 5.1.0.15008 we have been running the

Operations Manager Storage Efficiency Dashboard Plugin hourly without issues for a few months. After the upgrade the script fails with no errors given.

I have tried uninstalling it and reinstalling it with no luck.

Our DFM server runs on Windows 2008.

Any advice on how to resolve it?

>dfm script list
ID   Script Name                                        Version
---- -------------------------------------------------- ----------
10245 Operations Manager Efficiency Dashboard            1.3.0

>dfm script job start 10245
Script Job 10253 started.

>dfm script job list
ID   Schedule Name             Script Name               Start Time   Status
---- ------------------------- ------------------------- ------------ ----------
10253 Manual                    Operations Manager Efficiency Dashboard 28 Aug 14:03 Errors
10251 OM - Dashboard (10246)    Operations Manager Efficiency Dashboard 28 Aug 13:56 Errors
10250 Manual                    Operations Manager Efficiency Dashboard 28 Aug 13:40 Errors
10248 OM - Dashboard (10246)    Operations Manager Efficiency Dashboard 28 Aug 13:33 Errors
10247 OM - Dashboard (10246)    Operations Manager Efficiency Dashboard 28 Aug 13:32 Errors

I do have these errros in the dfm error.log but the timestamps do not coincide with the running of the script:  The client IP is my current workstation IP.

[Tue Aug 28 09:47:15 2012] [notice] Child 3784: Listening on port 8080.

[Tue Aug 28 09:48:54 2012] [error] [client 10.230.31.48] File does not exist: D:/Program Files (x86)/DFM/web/Contents, referer: http://wn1151.ourdomain:8080/dashboard.html

[Tue Aug 28 13:24:57 2012] [error] [client 10.230.31.48] File does not exist: D:/Program Files (x86)/DFM/web/Contents, referer: http://wn1151.ourdomain:8080/dashboard.html

[Tue Aug 28 14:18:43 2012] [error] [client 10.230.31.48] File does not exist: D:/Program Files (x86)/DFM/web/Contents, referer: http://wn1151.ourdomain:8080/dashboard.html

[J_ABELLA · Accepted solution]

Well I was left to answer my own question but I thought I would share how I was able to resolve my issue for some other poor admin.  I attempted to open a ticket with Netapp support and was bombarded with information gathering before I could get any usefull analysis or even a technical discussion on what was occuring.  I understand the need to collect some preliminary information but found this hurtle to get assistance annoying at best.  Also IMHO they should have asked me to go ahead and run the DMFDC tool available in the downloads to collect this info instead of piece meal it manually.

So I don't want to bore you with my rant so I have placed Netapp's information request email below this but now I am going to tell you how I resolved my issue:

Since I had uninstalled and reinstalled the dfmeff script I found the D:\Program Files (x86)\DFM\script-plugins\dfmeff directory only had a few files contained in it.  When I went to run dfmeff.exe it stated it couldn not run becuase it couldn't find dbtool10.dll.  I performed a search for the file on my system C: and 😧 drives and couldn't locate the .dll.  I then performed a websearch to get an idea of what the .dll even is.  The .dll is part of SQL anywhere, I then returned to do another search but this time because I was lazy I did a search for only dbtool and found a dbtool11.dll.  I took a copy of the file and renamed the copy to dbtool10.dll and left it in the same directory, I then reran dfmeff.exe which uncompressed all the .xml files and now the efficiency dashboard script runs as expected.

------------------------------------------

Here is the email of information requested by support regarding my issue:

Summary of Environment

Note: All the commands referred below are to be
run on the DFM server CLI.

Framing Questions:

1. What is the specific issue being experienced? Provide a detailed
   description along with the associated errors. Be verbose and collect examples.
   Include screen shots where necessary.
   • What are the steps leading to the specific issue?  This
     question is directed at understanding the workflow that leads to the specific
     issue and not just the issue itself.
   • Did this configuration ever work? If yes, when did it stop
     working?
   • What is the frequency of the failure? (How often, is it
     intermittent?)
   • Does it work in another setup/ configuration? (For example, a
     lab reproduction).
   • What changed since it worked previously?
   • When did the issue occur? Be as specific as possible, with the
     date and time; this will help NetApp when examining the log files.
   • What other errors, if any, are observed?
   • Does the issue affect every storage controller, some storage
     controllers, or only one storage controller? Identify which ones, by name,
     serial number and IP address.
   • Does the issue affect all the OnCommand Host Package clients,
     some OnCommand Host Package clients, or only one OnCommand Host Package client?
     Identify the specific client(s) by name and function.
   • Has isolation of the issue been attempted, such as suspending
     other services or operations to simplify the environment? Be as detailed as
     possible to establish timelines for the activity.
   • Have you referenced the product documentation, Knowledge Base or
     BUG Tools (public BURTS) to verify if the issue encountered is a known behavior
     or has a documented workaround?
2. What are the DataFabric Manager server version, host platform
   and Operating System in use? Information can be found by using ‘dfm diag’ or
   ‘dfm about’ for CLI methods or by going to Help > About, and
   then clicking Configuration for the UI method.
3. Are you running OnCommand Standard or the Express edition?
   Are you running previous release of Operations Manager (if so, what is the
   release)?
4. Which release of Data ONTAP is the affected storage controller
   running?
5. Are you running a supported configuration as specified in the
   OnCommand Core and Host Package Installation and Setup Guide and/ or the NetApp
   Interoperability Matrix Tool?
6. Does the OnCommand server meet the minimum requirements as
   specified within the OnCommand Core and Host Package Installation and Setup Guide
   and/or NetApp Interoperability Matrix Tool?
7. Is the DataFabric Manager server deployed on a system that is
   running no other applications?
   
   Note:
   Best practices and product documentation indicate that the DataFabric Manager
   server be the only application running on the server.
8. If OnCommand worked properly before the reported behavior, were
   any changes made to the storage system or the OnCommand server, such as patches
   to the host OS, product reconfiguration, network, or other maintenance?
9. Is there a firewall installed on the network that could be
   affecting communication between the storage system and the DataFabric Manager
   server?
10. Could this be a browser issue? Are you using one of the
    supported browsers? Have you tried clearing the browser cache?
    
    Note: OnCommand
    Core Package supports a reduced set of browser and versions as compared to the
    existing DFM product. Check the NetApp Interoperability Matrix Tool for a list of supported browsers and
    versions.

SECTION B

Data Collection:

Given the complex nature of
this product with regard to all the integrated pieces, collecting all logs in
at the same time for inspection after the issue has been observed is required
to continue troubleshooting.  It may be necessary to capture data in
multiple iterations, and it is best to capture data from all the involved
systems at the same time for inspection.

1. If the DataFabric Manager server operating system is Windows,
   execute msinfo32.exe, save
   the output as a *.nfo file,
   and upload the archive. For more information, see article  1010090: How to upload a file to NetApp.
2. Download and extract the DataFabric Manager Data Collector
   (dfmdc) from the Utility Toolchest in the Downloads link in the NetApp Support site,
   to the DataFabric Manager server, and run it per instructions on the download
   page. If the host is Windows, dfmdc must be run as Administrator from the CLI
   and not be impacted by UAC. Right-click the command prompt shortcut, and select Run as Administrator. Navigate to the
   directory where the dfmdc data files were extracted and begin the data
   collection. The dfmdc tool creates a single file archive as the output. Upload
   the archive per Section - 3: Sending Data to NetApp, below.
3. Collect the Host Services information from the DataFabric
   Manager server. Archive/ ZIP the following into a single file named hostserviceinfo, and add the appropriate
   extension. Upload the archive per Section - 3: Sending Data to NetApp, below.
   • dfm
hs list-a
   • dfm
hs diag <hostserviceID> (column one from dfm hs list is the ID)
   • dfm
hs controller list
   • dfm
hs plugin list
4. Collect the Virtual Sever Object information from the DataFabric
   Manager server. Archive/ ZIP the following into a single file name vservinof and add
   the appropriate extension. Upload the archive per Section - 3: Sending Data to
   NetApp, below.
   • dfm
vm list –a
   • dfm
vmdatastore list –a
   • dfm
vcenter list –a
   • dfm
hypervisor list –a
   • dfm
datacenter list –a
   • dfm
vdisk list -a
5. Generate an AutoSupport from all storage controllers involved in
   the issue. Execute the command at the same time as the data collection process
   is in progress, and tag the AutoSupport with the case number (<> are delimiters and should not be entered in
   the actual command):
   
   options autosupport.doit <casenumber>
6. Download and run the Data Collection Tool for Windows
   (ONTAPWinDC) from the Utility Toolchest in the Downloads link in the NetApp Support site,
   to each host involved in the issue (each host would be where the OnCommand Host
   Package was installed for integration with this DataFabric Manager server).
   Upload the archive per Section - 3: Sending Data to NetApp, below.
7. Collect the following logs from the same hosts where ONTAPWinDC
   was executed. Archive/ ZIP the following into a single file named hostpkglogs, and add the appropriate
   extension. Upload the archive per Section - 3: Sending Data to NetApp, below.
   • C:\Program
Files NetApp\OnCommand Host Package\VMware Plugin\log\*
   • C:\Program
Files NetApp\OnCommand Host Package\VMware UI Plugin\log\*
   • C:\Program
Files NetApp\OnCommand Host Package\SnapDrive\snpdrvdbg.log
8. If the issue involves the NetApp Host Agent (NHA), collect the
   following:
   • The NHA, by default, is installed in:
     • C:\Program Files\Network
Appliance\Agent\ on
       Windows (older DFM releases)
     • C:\Program
Files\NetApp\DataFabric\Agent\ on
       Windows (newer DFM releases)
     • /opt/NTAPagent/ on Solaris and Linux
   • Collect the output of the following commands:
     • Windows: ntap_agent.exe query and ntap_agent.exe version
     • Linux/
       Solaris: ntap_agent query and ntap_agent version
   • Collect ntap_agent.log file (Windows).
   • Collect ntap_agent.logntap_agent.acl, and ntap_agent.cfg files (Linux/ Solaris).
   • Combine all files/ output into a folder named 'nha' and upload
     the archive per Section - 3: Sending Data to NetApp, below.
9. If the issue involves the NetApp Management Console (NMC),
   collect the following:
   • The NMC, by default, is installed in:
     • C:\Program Files\Network
Appliance\Management Console\ on
       Windows (older DFM releases)
     • C:\Program
Files\NetApp\DataFabric\Management Console\ on Windows (newer DFM releases)
     • <dir_where_the_downloaded_installer_is_run> on Solaris and Linux
   • Collect the following files/ directories from the <installdir> on Windows and Linux/ Solaris:
     • Entire logs dir, boot.propertieslog4j.properties, and vendor.properties files
     • If
       present, all hs_err_*****.log files
   • Combine all files/ output into a folder named 'nmc' and upload
     the archive per Section - 3: Sending Data to NetApp, below.
10. Depending on archive file sizes, the individual archives above
    might be combined into a single archive for upload. Do not mix individual
    files; rather combine the individual archives into a single master archive for
    a particular data collection request.

[allison]

James Abella We would like to follow-up on this case. Can you give me your case support number for to be investigated further? Thanks.

[J_ABELLA]

Sure Allison, here you go: 2003494058.

[allison]

Sorry you had a less than stellar experience with our Support team. I can imagine how frustrated you were and we have passed this information along to our Support team. Thank you for sharing your knowledge with the Community for others who may encounter the same issue. Please feel free to reach out to me if you have any other feedback or concerns. Thanks.

~allison