NetApp OneCollect: The tool for easy, consolidated diagnostic data collection

[uttamd]

Introduction: 

NetApp OneCollect is a data collection tool that collects data from ONTAP, E-Series, Solid Fire, various operating system hosts and switches.  OneCollect tool can be used by Customers, Partners and Internal users.  The collected data is used for troubleshooting, solution validation, migration and upgrade assessments.  This data will be stored in ASUP and presented for use by various other tools like SmartSolve, ConfigAdvisor and Unified Parser.

Key Features

• Can be installed on Windows, Linux and OS X(Mac)
• Collects data from ONTAP, E-Series, SolidFire, solution based collection including FlexPod, MCC.
• Collects data from host Operating Systems including Windows, Linux, VMware ESX, AIX, HP-UX, Solaris.
• Collects FC and Network data from Cisco and Brocade switches.
• Collect data can be viewed using OneCollect raw data viewer and SmartSolve.
• Supports auto discovery of environment, parallel execution and scheduling collection jobs.
• Sends collected data to NetApp AutoSupport for use by assessment and troubleshooting tools.

GA

• March 14, 2017

NetApp® OneCollect 1.0

• Requires NetApp Support Site account

Support and Questions

• Community-based support only at this time
• If you have any questions please post a reply to this article or contact ng-OneCollect-support@netapp.com

[chao]

Thanks a lot for your sharing.  That's wonderful tool and hope the official version could be released sooner.  here are a few request from team.

1. Could onecollect collect the existing ASUP?  especially those weekly ASUP

2. the ASUP uploaded has the title "Onecollect notification ...".  Could myAUSP or other tools accept and handle this kind ASUP?  I remember many tools only accept weekly ASUP.

3.  For host and switch data, Could onecollect replace nSANity?  if so, could it save the output as nSANity format because many other tools accept nSANity output.

4.  is there a plan to collect perfstat?  

[Pradeep_Palukuri]

Hi Chao,

Thank you for your interest in OneCollect.  OneCollect 1.0 GA is planned on 3/14. Please find the responses below.

1. Could onecollect collect the existing ASUP?  especially those weekly ASUP

<Pradeep> Currently it doesn't collect but its in the roadmap

2. the ASUP uploaded has the title "Onecollect notification ...".  Could myAUSP or other tools accept and handle this kind ASUP?  I remember many tools only accept weekly ASUP.

<Pradeep> SmartSolve and Unified Parser can search and retrieve OneCollect ASUPs and parse the data.  We have MyASUP in the roadmap for not just displaying raw ASUPs but also possibly use some data within OneCollect and not present in regular ASUPs.

3.  For host and switch data, Could onecollect replace nSANity?  if so, could it save the output as nSANity format because many other tools accept nSANity output.

<Pradeep> Yes. OneCollect tool is replacing nSANity. The output formats will be supported by Unified Parser.  SANview is going away and most other tools are part of governance to consume new OneCollect format.  It's not a major change as we using nSANity code for mostly underneath but there are few schema changes.

4.  is there a plan to collect perfstat?

<Pradeep>  No.  Persftat is going away.   We should be using PerfArchive starting from 8.3.  Anything prior to 8.3 will use existing PerfStat 8.4 release which is the last release and no further updates planned.

Thanks,

Pradeep,

Product Manager

OneCollect and Perf Supportability Tools

[chao]

Many Thanks Pradeep.  Does onecollect have plan to support to retrieve existing PerfArchive?  BTW, out-of-box tool (ex. perfstat) is still necessary for those secure site.  Please consider about that.

[Pradeep_Palukuri]

Hi Chao,

Yes, we do have in the roadmap to collect perfarchives and core files but may be not in the upcoming 1.1 release.  I am working on gathering requirements for next release.  Please let me know if you need anything else.  I can share the current PRD.  Please let me know your email address. 

For secure sites, we need to collect perfarchives and upload manually.  Current perfstat and latx will exist but no further enhancements coming in it.

Thanks,

Pradeep.

[fail]

Congratulations on the release of OneCollect!  Awesome job!  Awesome tool!

[aleeyee]

Great tool! I am using this now more often for collecting and upload to UP.   I am however coming across an issue where all three tools are exhibiting the same behvior (nsanity,ict, one collect) on a specific set of ESXi hosts for a client.  The error that I get from all of these tools when I run it against the ESX host is the same.   The ESX hosts are running 5.5u2.  The error seems to be happening on post processing of esxcfg-info command.   I have run the command manually it finishes on the host successfully.  I am hoping you can shed some light as it seems all three tools use the same code base and exhibit the same behavior.   

ESX Host 1

03-18 01:48:51:INFO: Vmware:miavm101.bftg.com: Running esxcfg-info -a -F xml
03-18 01:58:50:INFO: Failed to run command: esxcfg-info -a -F xml. Exception: 'Timeout waiting for command to complete'
03-18 01:58:54:WARNING: Error while post-processing command: esxcfg-info -a -F xml. Error: Opening and ending tag mismatch: vm-disk-
resource line 2515059 and command_out, line 2515061, column 15

ESX Host 2

03-18 10:14:20:INFO: Failed to run command: esxcfg-info -a -F xml. Exception: 'Timeout waiting for command to complete'
03-18 10:14:29:WARNING: Error while post-processing command: esxcfg-info -a -F xml. Error: Opening and ending tag mismatch: world-32
967-stats- line 2104651 and command_out, line 2104667, column 15

[NithinR]

Hi Aleeyee,

   Thanks for the feedback! Looks like the log you posted is from ICT, but the root cause is clear- 'Timeout waiting for command to complete' and OneCollect would show same error. This means that command output did not complete within the set time limit (10min in case of OneCollect/nsanity). We have set a time limit of max 10min and a size limit of 100MB for output from any command; this is to avoid the case where a command would produce enormous/infinite output and to maintain sanity of the result file.

     However we do have a workaround for this in OneCollect, if you are sure the command is taking more time and has a definite output within the size limit. Please follow the steps below:

1. Stop OneCollect Process (From task manager in Windows or using kill in Linux/Mac)

2. Edit the CONFIG file in the path <user_home_directory>\NetApp\OneCollect\Config and add the below line at the end of file e.g to increase to 20min (maintain same case/syntax)

COLLECTION_TIMEOUT=1200

3. Restart OneCollect. (By clicking OneCollect.exe under C:\Program Files(x86)\NetApp\OneCollect in Windows, ./onecollect from install dir on Mac/Linux)

Please let us know if that works.

Thanks,

   Nithin

[chao]

Hi Sir

I found the uploaded OneCollect ASUP was moved to individual category and could only be searched by entering SN/Jobname... in OneCollect field.  is it true?  

Partner said they uploaded OneCollect ASUP but I can't fine them.  is there a log show me what happened? May I manually upload ASUP file collected by OneCollect?  

[Pradeep_Palukuri]

Hi Chao,

I believe you meant SmartSolve.  Yes, you can search using serial number or job name.  Search by case id is coming in couple of weeks.

You can get the job name from Partner by which you can search it.  Collection Logs are available in user system <home directory>/OneCollectData/LogFiles diectory.

Thanks,

Pradeep.

[gmilazzoitag]

Hi,

great tool!
Is there some relationship with the OneCollect Field I've seen in SmartSolve portal?
If yes, which ones?

I've tried to input a serial name but there are no result.

[Pradeep_Palukuri]

 Hi @gmilazzoitag,

Thank you very much for the feeback.

Yes, Onecollect ASUPs are integrated in SmartSolve.  You can search using serial number, job name or case id.

It will display only OneCollect AutoSupports.  To be able to search OneCollect Autosupports, there must be OneCollect Autosupports uploaded already using OneCollect tool (OneCollect tool can upload ASUPs after data collection).  There will be one autosupport per device that is collected.

Thanks,

Pradeep.

[JasonDrury]

As a partner I've asked a customer to run OneCollect against the devices in a SAN environment we're planning an upgrade for.  But now I have no idea where to go to get to the data.  Customer has said it's gone to NetApp but I can't work out which portal I should use to get to the ASUP with the OneCollect output data.

Alternatively, there doesn't seem to be any easy way for me to get the customer to send me the results files and import them into OneCollect for me to view locally.

Thanks,

Jason

[Pradeep_Palukuri]

Hi Jason,

Customer should be giving you the Job name which he created in OneCollect tool.  If you have access to SmartSolve tool, you can search using Job name or NetApp serial number in SmartSolve and view the data.  You can also search ONTAP data collected using OneCollect tool in Unified Parser by specifying ONTAP serial number.

Thanks,

Pradeep.

[aleeyee]

Hi Pradeep,

 Just an update and sorry for the late response.  The collection_timeout worked perfectly. Also like the feature to skip certain command collections.  

Thanks,

Amiel

[aborzenkov]

Am I right that Unified Parser is accessible only to NetApp employees? I get empty page when follow link from field portal to Unified Parser with my partner account.

[JasonDrury]

Hi aborzenkov.

On top of that, SmartSolve is only available to Support Services Certified (SSC) Partners and original equipment manufacturer(OEM) partners.  So Premier partners with a Professional Services focus such as ourselves can't even use it to view the OneCollect data.  Therefore the new tools for increasing our chances of being successful with migrations and transitions are kinda pointless.

Cheers,

Jason

[Pradeep_Palukuri]

Unified Parser is available to Customers and all Partners too.  Please email to ng-OneCollect-Support2@netapp.com with your NetApp SSO user id.  We will take it up with Unified Parser team and let you know.

Thanks,

Pradeep.

[chao]

I was confused by Onecollect plug-ins.  the URL in the InA manual does not work ( http://mysupport.netapp.com/NOW/download/tools/Onecollect/) . in same manual, it said there are the independent plug-in installation packages are available for the following plug-ins:

• ONTAP 7-Mode

• Clustered Data ONTAP

• E-Series

• FlexPod

• MetroCluster in Clustered Data ONTAP

I just wonder if basic Onecollect could collect data from above system?  at least I found nothing incorrect to collect cDOT system.

[NithinR]

Hi Chao,

    Thanks for the feedback. The documentation errors with respect to wrong URL and the confusion on plug-ins available for download are fixed and we will update the manual in portal soon.

     However, to clarify your question on plug-ins, all the profiles that you see in the OneCollect tool can be considered as plug-ins and we can selectively update each of those profiles if there are any updates to commands. In this release, we have bundled all the plugins together with the framework and are installed by default. So there is no need to install any of these separately unless there is an update which would be notified to users (under 'Notifications' tab in the tool when there is active network connection).

Thanks,

   Nithin

[JustinM]

Hi,

I have a requirement to provide a list from my virtual environment including the following:

Server Name

Operating System

DISM version

SnapDrive version

SnapManager for SQL version

I have a predetermined list of VMs to target. 

Based on the outcome of this list will determine if I need to perform any upgrades.  Can OneCollect provide me with this information or is there another tool available?  Alternatively should I be looking at writing a PowerShell script to do this for me?

I look forward to your responses.

Regards

Justin

[nigelg1965]

Hi

I can see how this is better than the previous app.

But is there a way to get rid of the annoying "help" overlays!

I'm sick of clicking on "I got", 'cause I have got it and I don't need the app to keep holding my hand now!

[DiegoFerreyra]

Hello,

Does OneCollect capture host performance statistics?

Thanks,

Diego

[Pradeep_Palukuri]

@JustinM Yes. OneCollect can do in the upcoming release which is scheduled end of this month.

[Pradeep_Palukuri]

@nigelg1965 The message should not come up again once you click on "I got it".  Please let us know if it is coming up even after that.

Thanks,

Pradeep.

[Pradeep_Palukuri]

@DiegoFerreyra  Hi Diego, OneCollect currently doesn't collect host performance stats but we do have this in the roadmap and shoud be available soon.

Thanks,

Pradeep.

[QL-Ryan_Rothert]

The import devices from CSV seems to be broken.  Any update on this? 

http://community.netapp.com/t5/OnCommand-Storage-Management-Software-Discussions/OneCollect-csv-import-option-giving-an-error/m-p/131239

[Claas]

Hi,

do you think it will be possible to have in the future a kind of portable app version of OneCollect?

It is not everywere possible to install application without a vast amount of organizational effort, but portable apps would be much easier.

Any hint when ASUP download from ActiveIQ will be implemented ?

Thank you

Best regards

Claas

[kayyumanwar]

Hi Team,

I am useing Onecollect to collect data for FLI ,HDS to Netapp migration .. But I see I dont have option to add HDS system to onecollect.

Am I missing something .. Please guide .

[NithinR]

Hi @kayyumanwar

     Thanks for your feedback! HDS collection is not currently supported. We can add it to our future roadmap.

Thanks,

   Nithin

[NithinR]

Hi @Claas

    THanks for your feedback! We can add these to our roadmap. We are planning to make the data directory path configurable like the install directory path, so that it would be possible to have all binaries and data in a mapped network location or pen drive.

Thanks,

   Nithin

[Claas]

Hello

@NithinRthank you for your quick reply, Perfect match, would be really helping in a bunch of customer situations.

Looking forward to the next Version

Cheers

Claas

[Tas]

I cannot get OneCollect running.  I've tried, Centos 6.7, RHEL 7.4 and WinSrv12.

I will concentrate on Windows for now:

I've disabled SEP.

I installed the package and ran it. 

When I run the shortcut, it opens, 127.0.0.1:8044/data-collection and I get the spin wheel.  It just sits there.

a. HOME does nothing

b. SETTINGS takes me to Data Collection Jobs, of which there are none, and no controls

c. ADMIN does nothing

d. DATA VIEW is empty:

  1. If I select a Solution Based Collection Type, Profile, SubProfile and Persona drop-downs are blank.

[NithinR]

Hi @Tas,

     Just to rule out any browser compatibility issues, the supported browsers are Firefox version 55 or higher, Chrome version 63 or higher and latest Microsoft Edge browser. If the browser falls under supported category, please try to delete or backup database file located at <user home dir>\NetApp\OneCollect\Db after stopping OneCollect process and restart the OneCollect process. Let us know if that works!

Thanks,

   Nithin

[Tas]

Well, you may have hit the nail on the head.  The Install guide states Firefox 35 or later.  I am running 39.  I will see about getting 55.

Actually, upgrading to Firefox x64 v56 on Windows Server 2012, and upgrading .NET 4.5.2 did it.

This is a solution.

[Tas]

So trying to run the utility is frustrating.

1.  Tried on our two-node (one HA-pair) AFF cluster, on both a node's management port and the cluster management port, and it will run then get stuck.  I've re-ran several times, usually stops between 20-35%, but it has gone to 52% once and stopped there.

2.  Tried on our eight node FAS cluster, same as above.  It runs for a while and then gets stuck

The job log does not show an error.  It shows the last command it was running.

3.  Tried it on a brand new test cluster with no SVM or SVM LIF's, it ran, but of course there is nothing on that.

[NithinR]

Hi @Tas,

     Sorry for the inconvenience caused! When the jobs were stuck forever, did the memory usage on your system shot up or the application hung? We will work on having a custom debug build and then we could have webex to troubleshoot and get it working. Please message you email id.

Thanks,

   Nithin

[Ajay_sasit]

Hi Team,

I am getting following error during data collection process and job completes with partial status. Using version 1.2

- Error is for the VMware ESXi host.

    "Failed executing 'esxcfg-mpath -v' on sasesx19. Error - esxcfg-mpath: invalid option -- 'v' esxcfg-mpath [--path=]"

It would be good to have feature where when to collect data for a vcenter server, it also collects all the required informaiton for the ESXi hosts too? Can this be achieved?

Thanks 

Ajay

[kayyumanwar]

Hi Team,

I am running OneCollect 1.3 against RHEL6 and windows R12, Both have Netapp luns LVM and MPIO with 3 paths etc .. All good configured.

Now when I ran the data Collect I got data .. But for RHEL I am faceing issue that I don't see Netapp luns .I can see one VM lun which is OS lun.

And If I switch to command view mode , I can see lun ,MPIO paths , device etc .. but not in advance/export view.

Am I missing anything here , Please help.

[NithinR]

@kayyumanwar

  Hi,

       Could you please send across the collected data files to ng-OneCollect-Support@netapp.com? The data files could be found at <user home directory>\OneCollectData\JobFiles\<Job name>.

       That would help us troubleshoot/fix the issue!

Thanks,

   Nithin

[kayyumanwar]

I guess this path is not valid for 1.3 ,

<user home directory>\OneCollectData\JobFiles\<Job name>.

Please check .

[kayyumanwar]

I found it , will email you details , thanks

[OGill]

Hello,

as far as I can see even with the current version 1.5 of OneCollect you can't collect perfarchives (aka Counter Manager data) from FAS-systems (be it 7-Mode or cDOT).

Is this feature still on the roadmap?

For some customers it simply is not allowed to connect their systems to ActiveIQ to get the fancy performance graphs.

And I can't use OCUM to get performance data since it does not support 7-Mode systems.

And Perfstats are also quite an effort if you for example want a week of data. The output gets really huge and the processing afterwards also takes some time.

Currently in these situations I always have to the mount the correct etc-path and then collect the Performance Archiver files manually which is quite the effort.

Would be awesome if you could tell OneCollect to download the perfarchives from say the latest X hours/days.

Then you could use CMPG to produce some performance graphs.

best regards

Oli