ONTAP Hardware

MetroCluster cabling - adapter mixing

IGORSTOJNOV
4,559 Views

Hello, all!

I've followed up on TR-3548 with regards to Stretch MetroCluster cabling and now I have a setup exactly like the one below:

I find it curious that for each controller, all FiberBridge connections, both local and remote, are connected to the same FC adapter [slot 3, in this case]. The same applies for DS14 shelf loop [slot 2].

Is this best practice or some kind of prerequisite? I wondering if it wouldn't be more resilient to mix up FiberBridge and DS14 connections between those 2 adapters, so that in case of failure for one adapter - there's still one local connection left to FiberBridge stack or DS14 loop?

Thanks!

5 REPLIES 5

aborzenkov
4,559 Views

Personally I would split connection to each loop/stack across two adapters. May be this was created as example of adding SAS to existing DS14 cluster, where you would avoid moving existing connections.

I sent comment to TR publisher, let’s see if we get any response ☺

IGORSTOJNOV
4,559 Views

Thanks for the feedback!

Sounds logical to proceed that way, but I now have one concern...

     Aggregate sas3 (online, raid_dp, mirrored) (block checksums)

       Plex /sas3/plex0 (online, normal, active, pool0)

         RAID group /sas3/plex0/rg0 (normal, block checksums)

           RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

           --------- ------          ------------- ---- ---- ---- ----- --------------    --------------

           dparity   4a.125L1        4a    12  0   FC:A   0   SAS 15000 560000/1146880000 560208/1147307688

           parity    4c.125L54       4c    12  1   FC:B   0   SAS 15000 560000/1146880000 560208/1147307688

           data      4a.125L3        4a    12  2   FC:A   0   SAS 15000 560000/1146880000 560208/1147307688

           data     4c.125L56       4c    12  3   FC:B   0   SAS 15000 560000/1146880000 560208/1147307688

           data      4c.125L57       4c    12  4   FC:B   0   SAS 15000 560000/1146880000 560208/1147307688

When I switch adapters, one of these paths will become say, 5a.

Will this cause any disruption on existing aggregate configuration?

More precisely, when I boot up the controllers will they see all over their old aggregates or might they panic over something?


aborzenkov
4,559 Views

If you do it offline - no.

IGORSTOJNOV
4,559 Views

Yes, it will be done offline. We have downtime scheduled during which we plan to physically migrate one half of the MetroCluster, along with some network equipment and server resources - to the new building.

Hence my idea to use that opportunity to re-wire the shelves/controllers in a more resilient way. The only thing I could see as a potential issue was aggregate "persistency", or more simply put, disruption of disk paths (on SAS shelves) connected via FiberBridge... I wouldn't like to trigger any events that would cause further downtime.  Is there any precaution I could take during boot-up sequence, maybe?

DOMINIC_WYSS
4,559 Views

Is there any precaution I could take during boot-up sequence, maybe?

you could boot up into maintenance mode first and check if you can see all disks.

it doesn't matter on which ports it will find the disks (software disk ownership), so I don't expect an issue.

Public