Solved: NetApp-Harvest OCUM server not reachable

Arjen_1 · ‎2016-06-29

I have an issue with the connection to OCUM. After a Ububtu update their seems to be an issue with the connection to the OCUM server.

I tested the following:

- Ping and telnet the ocum server

- Login to the ocum server using my netapp-harvest user and password I created for this

- Went true the setup guide to see if there is a step about config that might give me a clue of a some config that might be missing

- checked the log file and found the following alert:

[2016-05-28 06:55:14] [WARNING] [sysinfo] system-about API failed with reason: No response received from server
[2016-05-28 06:55:14] [WARNING] [main] system-info update failed; will try again in 10 seconds.
[2016-05-28 06:55:24] [WARNING] [sysinfo] system-about API failed with reason: No response received from server
[2016-05-28 06:55:24] [WARNING] [main] system-info update failed; will try again in 10 seconds.
[2016-05-28 06:55:34] [WARNING] [sysinfo] system-about API failed with reason: No response received from server

This is what is in the config file:

#====== OnCommand Unified Manager (OCUM) for cDOT capacity info ===============

[OCUM_EVO]
hostname = 10.13.150.47
site = AMS01
host_type = OCUM
data_update_freq = 900
normalized_xfer = gb_per_sec

It always worked so I think there is some parameter missing somewhere but I have no clue where

madden · ‎2016-06-29

Hi @Arjen_1

Harvest uses Perl libraries (Net::SSLeay) that in turn use openssl. My experience is that negociation issues at the ssl layer don't always get communicated back very well and and result in a more generic message like you mention. Can you check if using curl (1st line in bold below) works?

# curl -k -v https://10.64.28.77/
* About to connect() to 10.64.28.77 port 443 (#0)
* Trying 10.64.28.77...
* Connected to 10.64.28.77 (10.64.28.77) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject: CN=sdt-um.nltestlab.hq.netapp.com
* start date: Feb 27 19:03:02 2014 GMT
* expire date: Feb 27 19:03:02 2019 GMT
* common name: sdt-um.nltestlab.hq.netapp.com
* issuer: CN=sdt-um.nltestlab.hq.netapp.com
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.64.28.77
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Cache-Control: no-cache, no-store, must-revalidate
< Pragma: no-cache
< Expires: 0
< Location: /um/?redirectUrl=/
< Date: Wed, 29 Jun 2016 13:03:10 GMT
< Connection: keep-alive
< Transfer-Encoding: chunked
<
* Connection #0 to host 10.64.28.77 left intact

I just want to understand if the SSL server on the OCUM server is responding in a healthy way.

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

Arjen_1 · ‎2016-06-30

Hi @madden

Here is the output:

# curl -k -v https://10.13.150.47
* Rebuilt URL to: https://10.13.150.47/
* Hostname was NOT found in DNS cache
* Trying 10.13.150.47...
* Connected to 10.13.150.47 (10.13.150.47) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS alert, Server hello (2):
* error:14082174:SSL routines:SSL3_CHECK_CERT_AND_ALGORITHM:dh key too small
* Closing connection 0
curl: (35) error:14082174:SSL routines:SSL3_CHECK_CERT_AND_ALGORITHM:dh key too small

That does not look like the reply expected 😞

Regards,

Arjen

madden · ‎2016-07-01

Hi @Arjen_1

I did some googling and found this:

http://www.ubuntu.com/usn/usn-2639-1/

"As a security improvement, this update also modifies OpenSSL behaviour to
reject DH key sizes below 768 bits, preventing a possible downgrade
attack."

So I think the package update on your Ubuntu poller host might have upgraded ssl to a release that requires a minimum DH key size that is not supported by your OCUM host.

What platform (VA or on what OS+level) and release of OCUM are you running?

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

Arjen_1 · ‎2016-07-04

Hi @madden,

I checked the version and I was running 6.2. I upgraded to version 6.4P1 but that caused errors:

[2016-07-04 09:07:41] [NORMAL ] WORKER STARTED [Version: 1.2.2] [Conf: netapp-harvest.conf] [Poller: OCUM_EVO]
[2016-07-04 09:07:41] [NORMAL ] [main] Poller will monitor a [OCUM] at [10.13.150.47:443]
[2016-07-04 09:07:41] [NORMAL ] [main] Poller will use [password] authentication with username [netapp-harvest] and password [**********]
[2016-07-04 09:07:42] [WARNING] [sysinfo] Discovered [sbp-cluster1] on OCUM server with no matching conf section; to collect this cluster please add a section
[2016-07-04 09:07:42] [NORMAL ] [main] Collection of system info from [10.13.150.47] running [6.4P1] successful.
[2016-07-04 09:07:42] [ERROR ] [main] No best-fit collection template found (same generation and major release, minor same or less) found in [/opt/netapp-harvest/template/default]. Exiting;
[2016-07-04 09:17:26] [NORMAL ] WORKER STARTED [Version: 1.2.2] [Conf: netapp-harvest.conf] [Poller: OCUM_EVO]
[2016-07-04 09:17:26] [NORMAL ] [main] Poller will monitor a [OCUM] at [10.13.150.47:443]
[2016-07-04 09:17:26] [NORMAL ] [main] Poller will use [password] authentication with username [netapp-harvest] and password [**********]
[2016-07-04 09:17:28] [NORMAL ] [main] Collection of system info from [10.13.150.47] running [6.4P1] successful.
[2016-07-04 09:17:28] [ERROR ] [main] No best-fit collection template found (same generation and major release, minor same or less) found in [/opt/netapp-harvest/template/default]. Exiting;

So I downgraded to version 6.3P2 and now I do not get errors and it is collecting data so it seems:

[2016-07-04 11:22:59] [NORMAL ] WORKER STARTED [Version: 1.2.2] [Conf: netapp-harvest.conf] [Poller: OCUM_EVO]
[2016-07-04 11:22:59] [NORMAL ] [main] Poller will monitor a [OCUM] at [10.13.150.47:443]
[2016-07-04 11:22:59] [NORMAL ] [main] Poller will use [password] authentication with username [netapp-harvest] and password [**********]
[2016-07-04 11:23:01] [NORMAL ] [main] Collection of system info from [10.13.150.47] running [6.3P2] successful.
[2016-07-04 11:23:01] [NORMAL ] [main] Using best-fit collection template: [ocum-6.3.0.conf]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.SIN01.**************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.WDC01.**************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.AMS01.****************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.FRA01.***************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.AMS01.****************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.AMS01.*************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.AMS01.***************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.***************]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.***********]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.*********]
[2016-07-04 11:23:01] [NORMAL ] [main] Calculated graphite_root [netapp.capacity.**********
[2016-07-04 11:23:01] [NORMAL ] [main] Using graphite_meta_metrics_root [netapp.poller.capacity.**********]
[2016-07-04 11:23:01] [NORMAL ] [main] Startup complete. Polling for new data every [900] seconds.

And now I have info again in harvest.

madden · ‎2016-07-04

Hi @Arjen_1

Great to hear you got it working; it seems that the HTTPS feature support in newer versions of OCUM was the fix to be compatible with your updated client. Also, if you'd like to use OCUM 6.4 you can simply rename the configuration file; see this thread. I will include 6.4 support natively in the next Toolchest release.

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

View solution in original post