harvest problem with pollers and /etc/issue file

StevePutre · ‎2017-08-28

Hi:

I have a problem with Hravest 1.3, where polling fails after I start the poller processes. I had seen a discussion where the problem was identified as characters in the host's /etc/issue file interfering with the http requests. I think the problem may be that our company's default /etc/issue file has one single-quote character in it (used as an apostrophe).

I can temporarily work around the problem by editing /etc/issue and replacing its contents. The pollers then will start and will work OK. However, our company has a lot of security controls in place, and the /etc/issue file gets replaced with the corporate standard file once per day. So the next time the host reboots or the pollers are restarted, I have the same problem.

Is there a more permanent fix for this problem? Can the webserver or the perl scripts or whatever has the problem, be configured to ignore /etc/issue for example?

Tks,

-Steve

madden · ‎2017-08-30

Hi @StevePutre

I haven't heard of this issue before with Harvest. Can you let me know log entries (see /opt/netapp-harvest/log/) that point to this being the problem? You might also want to start the poller with the -v flag to get verbose log entries.

Cheers,

Chris Madden

Solution Architect - 3rd Platform - Systems Engineering NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

StevePutre · ‎2017-08-31

Thanks for the help, Chris. Actually, when i first saw this problem I think it was a forum post which tipped me off that /etc/issue was the cause. But I have been unable to find that post again.

Here is what the log looks like when the poller is failing:

...

[2017-08-31 13:42:42] [DEBUG ] [conf] Line [79] is Section [lshnap006]
[2017-08-31 13:42:42] [DEBUG ] [conf] Line [80] in Section [lshnap006] has Key/Value pair [hostname]=[10.24.192.8]
[2017-08-31 13:42:42] [DEBUG ] [conf] Line [81] in Section [lshnap006] has Key/Value pair [group]=[LSH]
[2017-08-31 13:42:42] [DEBUG ] [conf] Line [82] in Section [lshnap006] has Key/Value pair [username]=[harvest]
[2017-08-31 13:42:42] [DEBUG ] [conf] Line [83] in Section [lshnap006] has Key/Value pair [password]=[**********]
[2017-08-31 13:42:42] [DEBUG ] [conf] Line [84] in Section [lshnap006] has Key/Value pair [template]=[7dot-8.2.0-cifs-stats.conf]
[2017-08-31 13:42:42] [NORMAL ] [main] Poller will monitor a [FILER] at [10.24.65.185:443]
[2017-08-31 13:42:42] [NORMAL ] [main] Poller will use [password] authentication with username [harvest] and password [**********]
[2017-08-31 13:42:42] [DEBUG ] [sysinfo] Updating system-info cache
[2017-08-31 13:42:42] [DEBUG ] [connect] Using HTTP/1.0 for communication (either set earlier or only version supported by SDK).
[2017-08-31 13:42:42] [WARNING] [sysinfo] Update of system-info cache DOT Version failed with reason: Couldn't find end of Start Tag netapp
[2017-08-31 13:42:42] [WARNING] [main] system-info update failed; will try again in 10 seconds.
[2017-08-31 13:42:52] [DEBUG ] [sysinfo] Updating system-info cache
[2017-08-31 13:42:52] [DEBUG ] [connect] Using HTTP/1.0 for communication (either set earlier or only version supported by SDK).
[2017-08-31 13:42:53] [WARNING] [sysinfo] Update of system-info cache DOT Version failed with reason: Couldn't find end of Start Tag netapp
[2017-08-31 13:42:53] [WARNING] [main] system-info update failed; will try again in 10 seconds.

I have confirmed that the problem is being caused by the single-quote being used as an apostrophe. If I just remove that character from issue, the poller starts OK. It is not surprising, because any shell utility which parses text can easily get confused when it encounters a quote and goes looking forever for the matching end-quote.

Here is the problematic /etc/issue (sanitized). There is a single-quote on the first line.

CONSISTENT WITH XXXXX'S POLICY ON PRIVACY, MESSAGES AND OTHER
DOCUMENTS ON THE NETWORK MAY BE REVIEWED BY XXXXX MANAGEMENT
UNDER THE FOLLOWING CIRCUMSTANCES: WHEN AUTHORIZED BY THE
CREATOR OR RECEIVER; WHERE A BUSINESS-RELATED NEED EXISTS;
WHEN REQUIRED BY REGULATORY OR LEGAL AUTHORITY; OR WHEN
REQUESTED BY XXXXX COUNSEL TO PROTECT THE LEGAL INTERESTS
OF XXXXX.

USE OF THE SYSTEM BY XXXXX PERSONNEL OTHER THAN FOR BUSINESS
PURPOSES MAY SUBJECT THE USER TO DISCIPLINARY ACTION, INCLUDING
DISCHARGE.

Another detail: once the pollers are started, they will continue to run OK even though the /etc/issue file is returned to its 'problem' state with the ' character. This tells me that it is something being evaluated or stored at startup.

You may be able to reproduce the problem by creating /etc/issue with one single-quote character in it

madden · ‎2017-09-01

Hi @StevePutre

Gotcha, from your original post I thought the problem was the /etc/issue file on the linux host not on the 7-mode filer. This error is being returned by the NetApp Perl SDK not by Harvest. I tried to reproduce on my lab 8.2.3P3 7-mode system with an /etc/issue file matching your post, using the NetApp 5.6 SDK on my poller host, and could not.

So maybe this problem is already fixed?

The version you have can be found by running:

grep NMSDK_VERSION /opt/netapp-harvest/lib/NaServer.pm

If needed, update and see if that fixes.

If it doesn't, I think you have two options

(a) open a support case and report the problem against the Perl SDK, or try to hack a fix in there yourself.

(b) change your issue text slightly to avoid the apostrophe that seems to break everything

Cheers,

Chris Madden

Solution Architect - 3rd Platform - Systems Engineering NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!