Mismatched certificate between ONTAP and StorageGrid

[heightsnj]

In this FabricPool over Cluster and SG object storage environment, We renewed the certificate before it got expired on SG, but did not update or install just renewed one on Cluster. So, there is a mismatch. As the result, data couldn’t be tiered to SG. My question is, could it also result in not being able to read from SG if I set the tiering policy on a volume to be Auto and assuming data were already in SG? Why?

Thanks for your input!

[paul_stejskal]

Yes because the datastore would be not available. Generally this isn't noticed on NAS environments until those files are tried to be accessed.

[heightsnj]

@paul_stejskal The result caused a serious issue here, because a critical file that Oracle depends on happed to be on this volume. When SG was not even readable, that caused DB crash or couldn't be started up.

I am hoping there would  be more clear warnings when we renew the certificate on SG, something like "you will have to install /update the new one on the cluster as well".  These two things HAVE TO be done together. Neglected doing it on the cluster caused data inaccessible is pretty dramatic. The document is not so straightforward on that. 

[paul_stejskal]

I'm so sorry to hear that. 😞 Which document are you referring to? I'll get it updated or any documentation you referred to.

As far as improving behavior/alerting, I believe we have a RFE on this. Did you happen to open a case for this?

[heightsnj]

Only after the fact, we found the kb that addressed the issue

 https://kb.netapp.com/Advice_and_Troubleshooting/Hybrid_Cloud_Infrastructure/StorageGRID/How_to_Configure_StorageGRID_server_certificates_on_FabricPoo...

Yes we did open the case for help, but, unfortunately it was neglected by the engineer as well. 

[aronk]

I am sorry that this caused an issue for you.  I am looking at the StorageGRID documentation to add a warning note for certificate changes and updates.

[paul_stejskal]

There needs to be better handling of this in ONTAP really.

[heightsnj]

Yeah, either on ONTAP or SG side. 

We have had some issues for tiering, for instance in the situation when SG was getting full, in cases like that, it would not stop us to read, only stop us to tiering which is OK. But, in this case, we even couldn't read which was pretty dramatic, as I said earlier. 

[heightsnj]

@aronk @paul_stejskal Looking through KB again:

https://kb.netapp.com/Advice_and_Troubleshooting/Hybrid_Cloud_Infrastructure/StorageGRID/How_to_Configure_StorageGRID_server_certificates_on_FabricPoo...

Particularly on step 2, In my case, the certificate on SG got renewed successfully. It had not been installed/updated on the cluster as I said earlier, but, the certificate on the cluster was not expired yet despite mismatching. Would  that cause inaccessible as well?

I just wanted to make absolutely sure that was the cause. Forgive my persistence. 

[paul_stejskal]

If it wasn't expired it would still work. If the SG cert got renewed, than ONTAP and SG fell out of sync and that broke the communication.

[heightsnj]

@paul_stejskal 

Could you please explain to me how RFE works out?

[Mjizzini]

A request to add capabilities or improve performance beyond the specifications of the Products is referred to as an Enhancement Request RFE.

Creating an RFE is no guarantee for it getting implemented in the version it is targeted to.

RFE can be requested by customer, partner, or internal team.

If an RFE is already open,  you can subscribe to it to get email updates when they happen.

[paul_stejskal]

In addition to what Mo just described, once the RFE is open, the account team can help push if it is a super urgent issue for your business.