Acquiring Symmetrix data sources in OCI 7.0.2

JBARBALACE · ‎2015-05-04

Hello,

Are there any know issues with Symmetrix acquisition with 7.0.2? We had successful acquisition with 7.0.1, then when we upgraded to 7.0.2 we started seeing this error:

"Failed to execute external utility: Failed to execute Sym CLI symcfg, because The remote client/server handshake failed. Please consult symapi and storsvd log files"

So, consulted the logs, and maybe I just don't know what I'm looking for - is there a particular error message that I should see in these logs related to this issue?

I have the 7.0.2 patches file from Ostiguy, but not sure if the patch contained for the Symmetrix is relevant, or if there is something else going on. Nothing has changed. Same server, same netcnfg file. Now, we also had 7.0.2 running on a test server in our lab environment with zero issues - acquiring happily with no errors. I just upgraded to 7.1.0 in our test environment, and still happily acquiring.

The OCI server apparently still has permissions to access the SE server, although I'm having the storage team review and ensure that this is the case and that nothing funny has happened since it seems like if it was failing in one place, it should be failing in all places. We still have 6.4.3 running in production, and we've had no issues there. All acquiring successfully.

Anyone else come across this?

Thanks!

Julia

ostiguy · ‎2015-05-04

Hey Julia, Handshake messages = TLS/SSL negotiation failure. Why do TLS/SSL negotiations fail? DNS misconfiguration - ever go to a web site where you get a message that the digital certificate hostname does not match the hostname in the URL? Certificate expiration - didja renew your digital certificates? Clock drift - if you have a stupendously inaccurate clock (VM's are set to sync time to ESX host, one esx host X is NOT configured for NTP, and is set to GMT timezone - if your VM vmotions to host X, its clock could shift forward 7 hours - if you created a digital certificate within < 7 hours, that host will not perceive the digital certificate as valid because its start time is in the future). Solutions Enabler mutually negotiates SSL/TLS since SE 7.0. Any DNS misconfiguration between the client and server can cause handshake messages. On both the OCI server / point of acquisition , and the production SE server, review the storsrvd*.log* files - you are looking for an error like received XXXXXX, expected _______ Which means XXXXX was in the digital cert, but reverse DNS says ______ is the name for reverse DNS record for the IP address. This is the cause of 90% of SE problems with OCI. Ancient SE versions will have expired certs - I reallly hope SE 7.3 or 7.4 are in play - their cert signing certs are expired, and are another cause of issues Matt

JBARBALACE · ‎2015-05-04

Thanks for the quick reply! I'll run this past the team and see if they can review the SE server against the OCI server. As far as the obvious, the only change I made was installing 7.0.2, but maybe there was something else going on on the SE server and it turned into a perfect storm to create an issue. I'm not aware that I created any new certificates, but maybe on the SE server? I'll take a look.

Thanks!

ostiguy · ‎2015-05-04

I strongly doubt this was OCI related - this is really a SE <-> SE issue.

Since SE 7.0, mutual SSL/TLS negotiation occurs, so if either side dislikes the other side's certificate, your handshake message results

Matt

JBARBALACE · ‎2015-05-04

Thanks, Matt.

So, to clarify, it could be that whatever certificates are running on the OCI server might not be compatible with the certs on the SE server? Vice versa?

We went through something like this once before, where we updated the certs on the OCI server and the SE server, and it did not go well. We ended up rolling back because it caused the data sources to fail in our Lab and Production environment. It is just strange that it is only happening in one place after the 7.0.2 install, when we have OCI (in one version or another) running on three different servers. Which makes me think and agree that it's probably not an OCI issue alone.