Subscribe

update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

[ Edited ]

I'm having issues with fetching capacity data from OCUM. I see following errors in netapp-harvest log:

 

[2015-11-24 12:00:01] [WARNING] [aggregate] update failed with reason: in Zapi::invoke failed to connect SSL
[2015-11-24 12:00:01] [WARNING] [aggregate] data-list update failed.
[2015-11-24 12:15:00] [WARNING] [aggregate] update failed with reason: in Zapi::invoke failed to connect SSL
[2015-11-24 12:15:00] [WARNING] [aggregate] data-list update failed.
[2015-11-24 12:30:00] [WARNING] [aggregate] update failed with reason: in Zapi::invoke failed to connect SSL
[2015-11-24 12:30:00] [WARNING] [aggregate] data-list update failed.

 

I can access OCUM via SSL without any issues. Seems that it does capture data from time to time - but fails like that most of the time.

Re: update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

Hi,

 

What is the ONTAP version you are running ?

 

If possible, can you please create support bundle and put it in a shared path to debug it further ?

 

Regards,

Saravanan

Re: update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

Saravanan,

 

It only fails to caputre from OCUM, data captures from NetApp filers work fine.

 

Regards

JL

Re: update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

[ Edited ]

Hi,

 

The error “in Zapi::invoke failed to connect SSL“  comes from the SDK:

 

      Net::SSLeay::connect($ssl) or return $self->fail_response(13001,
           "in Zapi::invoke failed to connect SSL $!")

 

 

 

From that SDK code there should be a reason “$!” but none is provided making it hard to know why it aborted.

 

If you restart the poller does the problem go away?  

 

Is collection ok, then not ok, then ok again without restarting the poller?  Or is it ok, until it breaks when you start getting this error?

 

I think this is the fault of some intermediate module (either NetApp SDK or Net:: SSLeay from the internet)...

 

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

 

Re: update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

Chris,

 

After restart, it captures data once or twice, and then it stops. Sometimes restart helps, sometimes not. Seems that it can capture some data, and can't some. I do have all the dedupe and compression savings data, I don't seem to get used aggregate data. After it breaks it doesn't seem to come back.

 

Regards

JL

Re: update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

Hi JL,

 

I think the issue might be related to SSL libraries in use or those libraries trying to failback to something less secure than TLS (which isn't supported by OCUM or cDOT) for some reason.

 

Can you check/upgrade to the latest packages related to SSL?

--perl-Net-SSLeay

--openssl

 

Can you run harvest from a different host platform (so different linux box or distro, or from windows) to see if the behavior changes?  

 

Are you sure the network is error free between the poller and OCUM?

 

Can you try a more frequent poll interval and see if you have any change in behavior?

 

Cheers,

Chris

 

 

 

 

Re: update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

Chris,

 

I'm running that on Ubuntu 15.10 (64-bit), Network is error free between those hosts - i'm 100% sure.

 

Versions:

ii openssl 1.0.2d-0ubuntu1 amd64 Secure Sockets Layer toolkit - cryptographic utility

ii libnet-ssleay-perl 1.70-1 amd64 Perl module for Secure Sockets Layer (SSL)

 

I changed from 15 minutes to 10 and errors are still there.

 

I will try running harvest from a different host (maybe Centos 7.x)

Re: update failed with reason: in Zapi::invoke failed to connect SSL netapp-harvest OCUM 6.3RC1

Chris,

 

I've moved netapp-harvest to centos 7, i've changed pooling time to 5 minutes, and I can still notice those issues:

 

[2015-11-26 10:25:01] [WARNING] [aggregate] update failed with reason: in Zapi::invoke failed to connect SSL
[2015-11-26 10:25:01] [WARNING] [aggregate] data-list update failed.
[2015-11-26 10:30:05] [WARNING] [lun] update failed with reason: in Zapi::invoke failed to connect SSL
[2015-11-26 10:30:05] [WARNING] [lun] data-list update failed.
[2015-11-26 10:35:10] [WARNING] [qtree] update failed with reason: in Zapi::invoke failed to connect SSL
[2015-11-26 10:35:10] [WARNING] [qtree] data-list update failed.

 

versions:

openssl-1.0.1e-42.el7.9.x86_64
perl-Net-SSLeay-1.55-3.el7.x86_64

 

Could that be caused by the amount of data it has to capture from OCUM?