Failed to Access Event Log

[MURADAKRAM]

We are running SCOM 2012 and have 4 Management Servers in a single Management Group with one acting as the RMS emulator. After installing the OnCommend Plug-in and importing the MPs, we started getting the following WARNING alerts on 2 Management Servers. There is no "OnCommand Event logs" on these MS so my question is why the HealthService is looking for the OnCommand Event logs on these servers? (Did i missed something during the install?)

OnCommand event log

Description:

The Windows Event Log Provider is still unable to open the OnCommand event log on computer 'MS02.prod.domain.local'. The Provider has been unable to open the OnCommand event log for 720 seconds. Most recent error details: The specified channel could not be found. Check channel configuration. One or more workflows were affected by this. Workflow name: many Instance name: many Instance ID: many Management group: SCOMPROD

Event Data:

< DataItem type =" System.XmlData " time =" 2012-08-01T10:22:23.2691671-05:00 " sourceHealthServiceId =" 989E0FC3-A9DF-0F4A-B28B-283AF428418C " >
< EventData >
  < Data > SAVOPROD </ Data >
  < Data > many </ Data >
  < Data > many </ Data >
  < Data > many </ Data >
  < Data > OnCommand </ Data >
  < Data > 720 </ Data >
  < Data > The specified channel could not be found. Check channel configuration. </ Data >
  < Data > MS02.prod.domain.local </ Data >
  < Data />
  </ EventData >
  </ DataItem >

[jausch]

The OnCommand event log is how we translate OnCommand events into windows style events.  SCOM consumes events from the Windows Event Viewer.

If you look at your SCOM server and open windows event viewer, you will see an "Applications and Services" Log called "OnCommand" and if you look inside you will see all the alerts that we are collecting for SCOM.  This error means that the MP cannot open this Windows Event log for some reason.

Can you look at these servers to see if that log actually exists and if it's populated?

Alex

[MURADAKRAM]

Alex, I do see the OnCommand Event container, but it's only on the RMS Emulator server where OnCommand Plug-in is installed, but looks like the Healthservice is looking for this "OnCommand Event Log Container" on every SCOm Management Server. Should the OnCommand Plug-in install had created this on every SCOM MS? If not then why is it looking for it on every server?

Murad

[jausch · Accepted solution]

OK, it looks like SCOM is trying to run the discovery task on all your management servers and you only have SCOM installed on one of them.

When you enabled the discovery task, did you enable it for all management servers or just the one with OCPM installed?  You should have enabled it just for the one where OCPM is installed.

You can check this by going to "Authoring, Management Pack Objects, Rules."  You'll see a huge list of rules.  If you search for rules called "Data ONTAP" and you should see a list of just our rules.  Under the heading "Type:  Management Server" there should be two rules, "Data ONTAP: Discovery Rule" and "Data ONTAP Virtualization:  Discovery Rule."  If you right click on the "Data ONTAP: Discovery Rule" you should be able to select "Overrides Summary."  As part of the installation process, the IAG asks you to create an override to enable the rule, so you should see a single override there.  Double click on it and you should get a dialog box called "Override Properties."  Within that dialog box, there is a field towards the top labeled "Overrides Target."  The Overrides target should be just the management server where OCPM is installed.  If the overrides target says "Class:  Management Server" then you've targeted all management servers which would cause the error above.

If that is the case, you need to delete this override and create a new one just targeted at the specific management server where OCPM is installed.

Alex

[MURADAKRAM]

That was it Alex! I actually enabled this rule on my second MS thinking that I don't want to overload the RMS Emulator with  OnCommand discovery, but I didn't realize that this discovery rule can only be run/enabled on the MS where OCPM is installed. With that said I deleted the override and recreated it on the RMS server.

So that means if I ever fail-over my RMS Emulator role to another server I have to modify this rule again right?

Thanks for your help, I will watch it for a day just to make sure the error doesn't pop-up again.

Murad

[jausch]

Yes, exactly.

What many people do is to install OCPM on two servers but leave the second one inactive.  You can then enable the rule on your backup very quickly via the UI or via a PS Script.

We don't support active-active controllers in this version.  Working on it for a future release.

Alex

[MURADAKRAM]

Alex, does this also mean that when I add a controller using the "Data ONTAP: Add Controller" option from the SCOM console task bar, I should only select the Management Server where OCPM is installed even though I see all 4 MS in the list?

Our SCOM environment is spanning multiple data centers (Dallas and Chicago) and I was hoping that I could use the MS in Dallas to monitor our NetApp storage in Dallas to avoid additional traffic over the WAN link, but I guess that's not possible either because MS in Dallas doesn't have the OnCommand Eventlog container :-(.

How can I modify which SCOM MS a NetApp controller is reporting too?

Aren't I keeping you busy on Friday 🙂

[jausch]

No, you can have only one active management server.  In the case of a DR situation, you will need to enable the discovery rule on the new server.  The discovery task will then grab all the ONTAP objects and begin collecting stats for them.

Yes, when you add a controller, you should assign them to your one active management server.

Alex

[MURADAKRAM]

Hi Alex,

Because initially I had the discovery rule enabled on my 2nd Management Server, the discovery task is still associated with that server even though I deleted and recreated the discovery rule and enabled it only to run on my RMS emulator server. I also remove the controllers and readded them with no luck.

Any idea how can I update the Discovery task to run on the correct server?? If you look at the image below you will see the "Data ONTAP: Discovery Rules" is only enabled on the elkscomp01 (RMS) server, but if you look at the "Data ONTAP: Run Discovery Task" you will see that's it's stil listing my 2nd MS (elkscomp02) machine which is why it's still complaining about not be able to find the OnCommand Event log on this machine 😞

[ramdass]

Hi Murad,

When you click on the "Run discovery task" make sure in your Management Server view you click elkscomp01 before clicking the task. It seems as though you may have elkscomp02 when you click Run discovery task.

Please let me know if that works.

Cheers,

Dennis

[MURADAKRAM]

That was it Dennis, thanks!

[ramdass]

Great. Thanks Murad! We'll be sure to update our documentation to make this clear.

Cheers,

Dennis.

[MarioTunes]

Hello Alex,

I need your advice. We have SCOM 2012 R2 environment where we installed Data ONTAP MP and our equipment under monitroing.

We have a customer in nontrusted domain and we use SCOM Gateway to monitor it. We use 'Customer-GW.localdomain.local' as management server. When we add NETAPP storages to the monitoring system we got following alert:

"The Windows Event Log Provider is still unable to open the OnCommand event log on computer 'Customer-GW.localdomain.local'. The Provider has been unable to open the OnCommand event log for 1440 seconds.

Most recent error details: The specified channel could not be found. Check channel configuration.
 

One or more workflows were affected by this.  

Workflow name: many
Instance name: many
Instance ID: many
Management group: GroupName"

There is no "OnCommand" event log on server "Customer-GW.localdomain.local". What is the right way to setup NetApp monitoring in this configuration?

Thank you

[StewartMilne]

We also experienced a similar issue with the OnCommand 4.1.1 management pack:

The Windows Event Log Provider is still unable to open the OnCommand event log on computer 'xxxxx'.
The Provider has been unable to open the OnCommand event log for 64800 seconds.

Most recent error details: The specified channel could not be found. Check channel configuration.

One or more workflows were affected by this.

Workflow name: DataONTAP.PRO.Discovery.Rule

The management server in question does not have the OnCommand plugin installed, nor is it a member of the "Clustered Data ONTAP: Management Server Resource Pool" so I found it strange that the management pack rules were being executed on all management servers instead of being scoped to the resource pool. This was fixed with an override in the discovery rule to disable it for the management server in question.

[Andre]

we are experiencing the same problem.

created a new override - it is only pointing to the management server that has OCPM installed, but we still see event 26004 on all other management servers and gateways.

even checked the SQL DB:

SELECT * FROM [OperationsManager].[dbo].[MT_DataONTAP$ManagementServer]

ot returns only the managment server with OCPM installed.

we also have clustered data ontap installed. the "same" management server is the only one on the resource pool for clustered data ONTAP:Mangement server resource pool.

have you managed to resolve the notifications ??