VMware Solutions Discussions

VSC 5.0 SSL Handshake Failures

JBROADWAY
19,317 Views

I'd just like to document a problem and a work around / solution I found with VSC 5.0 and vCenter 5.5 in case anyone else is experiencing something similar as I did not see anything for this in the bug tracker for VSC.

Setup:

vCenter 5.5 on win2k8r2

VSC on a separate win2k8r2 server

cDOT 8.2.1

Problems encountered:

Unable to to backup systems from within vCenter,

Clicking on "Backup and recovery Configuration" option in the VSC plugin in vCenter gives an "unable to connect to virtual storage console"  error

Running commands such as "smvi backup list" as well as other smvi commands from the VSC server produces an error and throws java exceptions about handshake failures and a connection has been closed in the logs.

Specific issue (found using Wireshark):

After the tcp three way handshake, VSC by default sends a SSL client hello handshake proposing SSLv3 (0x0300) to vCenter.  vCenter would then FIN,ACK the connection immediately causing VSC to issue a fatal alert handshake failure packet back to vCenter and the client programs to fail / error out.  There was also periodic background traffic between the two servers that I noticed would always propose and negotiate at TLSv1 so I guessed that forcing the client initiated connections to negotiate at TLSv1 might fix the issue and it did.

Solution / Workaround:

Open

%Programfiles%\Netapp\Virtual Storage Console\smvi\server\etc\wrapper.conf

and find the "wrapper.java.additional.X" statements (should be 4), add the following statement.

wrapper.java.additional.5=-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2

Open

%Programfiles%\Netapp\Virtual Storage Console\wrapper\wrapper.conf

and find the "wrapper.java.additional.X" statements (should be 7) add following additional statement.

wrapper.java.additional.8=-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2

Restart both VSC services or reboot.

VSC will now propose TLSv1.2 (0x0303) by default and vCenter will negotiate the connection down to TLSv1.0 (0x0301).  If you remove TLSv1 from those statements, you'll get the handshake failures again as it appears vCenter 5.5 will only use TLSv1.  The "Backup and Recovery" configuration will now display in vCenter, smvi commands complete successfully and backup jobs can be created from within vCenter.

I had added the -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 in a couple other of places (control panel java applet for both the system jre and the one located in the VSC installation folder) and the smvi.bat / .cmd files but the two wrapper.conf files appear to be the only place where you can force specific java run time options for VSC.

12 REPLIES 12

CSEIZMAIR
18,909 Views

Thanks for the detailed fix. It also helped me with the same errors.

rubinsed1
17,926 Views

This didn't help me with getting VSC to communicate with the filers over TLS per the vulnerabilities listed in https://kb.netapp.com/support/index?page=content&id=9010008 and sslv3 being disabled on them.

I wonder if there is some other way to get that to work.

Coldfirex
17,251 Views

Worked here with VSC 4.2.2 and VCenter 5.1 u3 that is set to use TLSv1.  Thanks!

JCobe
16,059 Views

Used this fixing a problem after updating to vCenter 5.5U3B. I had to remove the TLS v1.1 and 1.2 arguments though. The lines looked like this:

wrapper.java.additional.5=-Dhttps.protocols=TLSv1

 

I was getting a HTTP 500 error (Failed to retrieve backup list). The Netapp Interop matrix doesnt have 5.5U3B in the listing and probably should be since 5.5U3B removes SSLv3.

dkomanek
15,835 Views

It resolved my problem after upgrading vCenter to 5.5 update 3b, thank you very much.

 

It was necessary to mention only TLSv1 in the configuration files (as is mentioned in one of the replies to this thread) and to restart the whole windows server. Restarting VSC server service was not sufficient in my case.

 

Thanks again,

 

D.

KvK
15,678 Views

Hello,

 

Thanks for this info.

The same problems occurred with VSC 4.2.2 when we updated vCenter 5.5 U3 to  vCenter 5.5 U3b

I used your work around / solution to solv our problem 

 

Regards,

 

René

inva
15,366 Views

Does the work around work, if the netapp ontap version is 7.3.5.1 ? i tried, it didn't work , and i have VSC 4.2.2 / vSphere 5.5 U3B

GenosMan
15,350 Views

Inva,

 

That is quite an old version of VSC which may be part of the problem, but I would check to make sure any newer versions work with the version of OnTap you have deployed before updating VSC.

thigian007
15,492 Views

I had this same exact problem after upgrading VCenter to 5.5 Update 3b. this should be an official Netapp article. Worked like a charm.

georgevj
9,754 Views

This is documented in the following KB article

https://kb.netapp.com/support/index?page=content&id=2026327

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.
Cannot find the answer you need? No need to open a support case - just CHAT and we’ll handle it for you.

ThomasHoffmannKBB
9,723 Views

Thank you for this solution. It helped me to resolve the Problem as described.

Best regerds

Thomas

geluyan
8,878 Views

solved the same issue by one of our customer. thx! This is one for my documention 😄

Public