Windows Server 2012 servers stopped seeing LUNs that were mapped from NeptApp FAS2552 ONTAP 9.1

[areti]

Hello!

There was an emergency restart of the equipment (servers, FC switches). After that, two Windows Server 2012 servers stopped seeing LUNs that were mapped from NeptApp FAS2552 (ONTAP 9.1 Cluster Mode) . I tried re-presenting LUNs to servers, but with no success. The servers can't see these LUNs.

The netap itself did not reboot, but in the logs events of disconnection of links on fс ports were recorded (servers and fс-switches were rebooted).

[areti]

On the same NetAPP 2552, other LUNs for ESX VMware servers are presented, and everything works fine.

And the Windows 2012 server, which does not see LUNs on NetApp 2552, works without any problems with LUNs on NetApp2240 and FAS2750, which are also connected to the same FC switches.

The only problem is in the bundle of NetApp2552 and Windows Server 2012 - they do not see each other in the SAN network, and this happened yesterday after an emergency restart of FC switches and Windows Server 2012 servers.

[areti]

I created a new LUN and presented it to the server, anyway the server does not see this LUN.
And when choosing an initiator, there is no WWWN of this server in the drop-down list, you can only enter it manually. The conclusion is that the FAS2552 and the server do not see each other over the SAN network, but they work fine with other devices in this SAN network.

[Ontapforrum · Accepted solution]

Please share this output:

::> network fcp adapter show

[areti]

fas2552::> network fcp adapter show
Connection Host
Node        Adapter  Established  Port Address
------------  -------         -----------       ------------
fas2552-01         0c            loop         0
fas2552-01         0d            loop         0
fas2552-01         0e            ptp            11700
fas2552-01         0f            ptp             11700
fas2552-02          0c           loop          0
fas2552-02          0d          loop           0
fas2552-02         0e            ptp             10000
fas2552-02         0f             ptp             10000
8 entries were displayed.

[Ontapforrum]

Sorry I needed to check 'operations status' of those ports, please give me this:

::> network fcp adapter show -fields status-oper

[not_a_Lone_wolf]

@areti Compare the output of the above command with the one mentioned in the following kb :https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Systems/FAS_Systems/FCP_target_adapter_showing_'LINK_DISCONNECTED'_after_'scsitarget.isp...

[not_a_Lone_wolf]

++ https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Systems/FAS_Systems/What_is_the_difference_between_LINK_NOT_CONNECTED_and_LINK_DISCONNEC...

[areti]

fas2552::> network fcp adapter show -fields status-oper
node adapter status-oper
---------- ------- ------------------
fas2552-01 0c link not connected
fas2552-01 0d link not connected
fas2552-01 0e online
fas2552-01 0f online
fas2552-02 0c link not connected
fas2552-02 0d link not connected
fas2552-02 0e online
fas2552-02 0f online
8 entries were displayed.

[not_a_Lone_wolf]

Thanks for sharing the output. Kindly compare the output with the kb links shared above. You would get the answer

[Ontapforrum]

Possible cause for link disconnected in your output: 

If an FCP adapter is in a 'LINK DISCONNECTED' state, it means the adapter was taken down due to a fibre channel link offline event. The link is still physically connected. This condition should recover once the issue that caused the offline event is cleared, but can require manual intervention in some cases.

Attempt a bounce of the connected port from the fabric switch side, then re-assess connection:

Cisco Example:
switch# config terminal
switch(config)# interface fc<slot/port #>
switch(config-if)# shutdown
switch(config-if)# no shutdown

Brocade Example:
switch:admin> portdisable <index # OR slot/port #>
switch:admin> portenable <index # OR slot/port #>

If issue persists and port does not come online, physical inspection and re-seat of cable and SFPs on both ends may be required. Repeat the port bounce process after re-seat. If issue still persists, please contact NetApp Technical Support and reference this article for further assistance.

Courtesy:
https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Systems/FAS_Systems/FCP_target_adapter_showing_'LINK_DISCONNECTED'_after_'scsitarget.isp...

[areti]

On each node, two links are involved, and they have the online status.
Four VMware ESXi servers see their datastore on this FAS2552 and the virtual machines are running.

But other physical servers (Windows Servers) stopped seeing their LUNs.
I even took a new physical server out of the box today and deployed Windows Server 2016 from scratch.
Configured Zoning on FC switches.
And the new server is also not found in initiators on this storage FAS2552.
And on another FAS2750, which is connected to the same switches, everything works without problems.

It seems that the FC ports on the FAS2552 came in with a stupor after the emergency reboot of the FC switches.
Perhaps rebooting the nodes on the FAS2552 will help? But I want to solve the problem without such drastic measures. Virtual machines work on this storage FAS2552.

Another error in the logs:

-------------------------------------------------------------------------------------------------------------------------------------------------------
Event:
mgmtgwd.certificate.expired: A digital certificate with Fully Qualified Domain Name (FQDN) svm_fcp, Serial Number 05474F89E30D9D, Certificate Authority 'svm_fcp' and type server for Vserver svm_fcp has expired.
Message Name:
mgmtgwd.certificate.expired

Sequence Number:
484933

Description:
This message occurs when a digital certificate for a Vserver expires. Client-server communication is not secure.

Action:
Install a new digital certificate on the system using the 'security certificate create' or 'security certificate install' command.
---------------------------------------------------------------------------------------------------------------------------------------------------------------

It appeared in the logs for a very long time. Can't it be the cause of the this failure?

[areti]

Each node uses links 0e and 0f, and they have online status.

Links 0a and 0d are not connected because they have never been used.

We use the FC protocol.

[Ontapforrum]

Ok. If each node uses only '0e' and '0f', then physical link is ok.  All the errors related to ports (that aren't connected) can be ignored. Now, could you list the 'fc data lif' that connects to those Windows Server 2012 boxes.

[Ontapforrum]

Hosts are unable to login to FCP LIF due to Brocade defect DEFECT000387237 [This could be your issue if you are using BROCADE ?]

If yes: Then, In my last reply, I asked for 'fc data lifs', simply bounce the LIFs (Logical layer), that should do the trick.

How to bounce:
::> net int modify -vserver <vserver_name> -lif <lif_name> -status-admin down
::> net int modify -vserver <vserver_name> -lif <lif_name> -status-admin up

[areti]

Ontapforrum, thank you for help!

The commands you suggested helped:
::> net int modify -vserver <vserver_name> -lif <lif_name> -status-admin down
::> net int modify -vserver <vserver_name> -lif <lif_name> -status-admin up

Problem solved!

Brocade defect bug DEFECT000387237 article:

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/Hosts_are_unable_to_login_to_FCP_LIF_due_to_Brocade_defect_DEFECT00038...