Web authentication problem after upgrading to OnCommand Core 5.0

[pennington]

Has anyone else encountered the following issue?

After upgrading from DFM/Operations Manager 4.x to OnCommand Core 5.0.1, I am encountering an unusual issue where I log into the web page without a problem but after some period time, I am automatically logged out and can no longer back in.  I usually can not log back in for about 24 hours or after I restart the server.  The server is running Windows 2008 R2.

I am using Internet Explorer 8 but I do not believe this is a browser issue.

When I examine the dfmserver.log file shortly after I cannot log in, I find the following messages:

Jun 29 13:20:11 [dfmserver: WARN]: [1796:0xc5c]: LookupAccountName failed for foo, error = 1332 No mapping between account names and security IDs was done.

Jun 29 13:20:11 [dfmserver:DEBUG]: [1796:0xc5c]: Failed to parse administrator name: foo

Jun 29 13:20:11 [dfmserver: WARN]: [1796:0xc5c]: LookupAccountName failed for foo, error = 1332 No mapping between account names and security IDs was done.

Jun 29 13:20:11 [dfmserver:DEBUG]: [1796:0xc5c]: Failed to parse administrator name: foo

Jun 29 13:20:11 [dfmserver:DEBUG]: [1796:0xc5c]: Login failed (http) for foo: incorrect password.

Jun 29 13:20:11 [dfmserver: INFO]: [1796:0xc5c]: Logon request for VDFM02\administrator (VDFM02\administrator) denied: 1326, Logon failure: unknown user name or bad password.

The server name is VDFM02 and I am using the local admin account in the example above.  The same problem occurs when I use my Active Directory login which is what I normally used without problems before I performed the upgrade.

[pukale]

when you hit this issue once again, could  you please check the status of http and webui by running below command?

# dfm service list

also, Please let us know o/p of "dfm option list | findstr -i http"

Thanks

Santosh

[pennington]

Hello,

The http and webui services are running.  Below is the output you requested from the two commands.

C:\>dfm service list

sql: started

webui: started

http: started

eventd: started

monitor: started

scheduler: started

server: started

watchdog: started

C:\>dfm option list | findstr -i http

agentHostTransport                    https

autosupportProtocol                   https

hostAdminTransport                    http

httpEnabled                           Yes

httpPort                              8080

httpsEnabled                          Yes

httpsPort                             8443

perfAdvisorTransport                  httpsOk

serverHTTPEnabled                     Enabled

serverHTTPPort                        8088

serverHTTPSEnabled                    Enabled

serverHTTPSPort                       8488

-Lia

[pennington]

Some additional information I discovered today after I failed to login thru the web interface....

In the dfm.log file, the following error message is present at the same time I attempted to log in to the web gui:

"Jul 05 17:28:46 [dfm:ERROR]: [3752:0x134c]: Error updating profile: [Sybase][ODBC Driver][SQL Anywhere]Index 'upUniqueConstraint' for table 'userProfiles' would not be unique (4294967100)"

And there is also present a log file named "jetty-2012_07_05.log"  which contains the following:

2012-07-05 17:28:34,922 [30549415@qtp-10933534-6: FATAL]: com.netapp.nwf.dfmui.server.log.BrowserLogServlet: Thu Jul 05 16:51:28 MDT 2012 running from https://denvdfm02.leprino.local:8443/start.html com.netapp.nwf.client.userinterface.application.Application$Method.ON_UNCAUGHT_EXCEPTION, com.google.gwt.core.client.JavaScriptException: (Error): Out of stack space

number: -2146828260

description: Out of stack space

2012-07-05 17:28:34,922 [30549415@qtp-10933534-6: FATAL]: com.netapp.nwf.dfmui.server.log.BrowserLogServlet: Thu Jul 05 16:51:58 MDT 2012 running from https://denvdfm02.leprino.local:8443/start.html com.netapp.nwf.client.userinterface.application.Application$Method.ON_UNCAUGHT_EXCEPTION, com.google.gwt.core.client.JavaScriptException: (Error): Out of stack space

number: -2146828260

description: Out of stack space

2012-07-05 17:28:34,922 [30549415@qtp-10933534-6: FATAL]: com.netapp.nwf.dfmui.server.log.BrowserLogServlet: Thu Jul 05 17:28:26 MDT 2012 running from https://denvdfm02.leprino.local:8443/start.html com.netapp.nwf.client.userinterface.application.Application$Method.ON_UNCAUGHT_EXCEPTION, com.google.gwt.user.client.ui.AttachDetachException: One or more exceptions caught, see full set in AttachDetachException#getCauses

[adaikkap]

My suggestion is to open a case with NetApp Global support for this issue as it would need some data collection and correlation.

Regards

adai

[markkulacz]

Any progress or solution on this issue?

Running into exactly the same problem with OnCommand Core 5.0 with a different client.

This message has been around for several years with MS AD authentication. It doesnt appear to be NetApp OpsManager/OnCommand specific. For example, read http://social.msdn.microsoft.com/Forums/en-US/sqlsecurity/thread/97f73ec5-5016-4366-af0f-d240ccdf0a0d/ and http://connect.microsoft.com/SQLServer/feedback/details/355414/no-mapping-between-account-names-and-security-ids-was-don (dates back to 2008!). Often these problems are related to a DB-like application performing user authentication and have frequently involves MS SQL. However, its really not a SQL issue. It appears to be something in the user account/AD config that has become stale and/or inconsistent.

[adaikkap]

Hi Mark,

     I Recommend you to upgrade to 5.0.2 which has some critical securtiy related bug fixes and open a case with NetApp Support on the same.

Regards

adai

[pennington]

Mark,

There has been no progress on this issue and upgrading to 5.0.2 did not solve the issue.

I have a case open now with NetApp support, so hopefully I'll get a resolution soon.

Thanks for the links!

Regards,

Lia

[lacherejc]

Hello,

Support have you solved the problem ?

I met same issue on version 5.1.

Thanks

JC

[lacherejc]

Hello again

I solved my problem.. McAfee block process that send email !!

So be carefull with your antivirus

JC

[SANTHANAK]

lacherejc , could you please provide us some more information on this one ?

This seems to be affecting in larger portion of the new upgrades, we upgraded nearly 3-4 machines and looks like we face this issue in all of the machines I did checked all the user level authentication and all seems to be set correctly. Could be some kind of bug in the software which couldnt find the exact user/password correctly.

.@Lia , do you have any update from support. Did they got anything new on this issue. Please let us know if you find anything from support.

-Santhana

[adaikkap]

Hi Santhana,

     What version of OCUM are you running ? From which version did you upgrade from ?

Regards

adai

[SANTHANAK]

Adai,

This was upgraded from  dfm 4.0 (4.0D15)   to dfm 5.1 (5.1) . As of now have upgrade nearly 4 machines and in all of them , could see similarities like this error message poped up immediately when the installation completed.

Mar 31 11:07:16 [dfmserver: INFO]: [30192:0x2ad5d6f6af50]: Starting Host Service Processes.

Mar 31 11:07:18 [dfmserver:ERROR]: [30192:0x41414940]: Login failed for root: Invalid password: Authentication failure

Mar 31 11:07:18 [dfmserver:DEBUG]: [30192:0x41414940]: Login failed (http) for root: incorrect password.

Mar 31 11:07:20 [dfmserver:ERROR]: [30192:0x41515940]: Login failed for root: Invalid password: Authentication failure

Mar 31 11:07:20 [dfmserver:DEBUG]: [30192:0x41515940]: Login failed (http) for root: incorrect password.

Mar 31 11:07:23 [dfmserver:ERROR]: [30192:0x41c1c940]: Login failed for root: Invalid password: Authentication failure

also dfmserver is crashing very frequently. There is no change( no extra load or host is been added) other than upgrade  .

Thanks,

Santhana

[adaikkap]

Hi Santhana,

     Are the servers windows ? We disabled local administrator access on windows from version 5.1. Can you check that is causing issue for your. Pls refer this kb article.

Though nothing should change on upgrade.

https://kb.netapp.com/support/index?page=content&id=1013744

I also recommend you to open a case with support.

Regards

adai

[pennington]

Santhana,

I only got a workaround from support for the problem.

Essentially your profile is updated in the OnCommand/DFM database when you login but does not get updated or "cleaned-up" and then has to be manually deleted.

So on the server, run the command "dfm profile list"; then delete the profile-id for the user locked out using the "dfm profile delete" command.

C:\>dfm profile list
Profile ID   Administrator Name Client Address   First Used   Last Used
------------ ------------------ ---------------- ------------ ------------
75K08PAH2ZW  lipennin           xx.xx.xx.xx      19 Feb 09:07 19 Feb 09:07

C:\>dfm profile delete 75K08PAH2ZW
Deleted 75K08PAH2ZW profile.

Regards, Lia