MetroCluster cabling - adapter mixing

IGORSTOJNOV · ‎2014-07-29

Hello, all!

I've followed up on TR-3548 with regards to Stretch MetroCluster cabling and now I have a setup exactly like the one below:

I find it curious that for each controller, all FiberBridge connections, both local and remote, are connected to the same FC adapter [slot 3, in this case]. The same applies for DS14 shelf loop [slot 2].

Is this best practice or some kind of prerequisite? I wondering if it wouldn't be more resilient to mix up FiberBridge and DS14 connections between those 2 adapters, so that in case of failure for one adapter - there's still one local connection left to FiberBridge stack or DS14 loop?

Thanks!

aborzenkov · ‎2014-07-29

Personally I would split connection to each loop/stack across two adapters. May be this was created as example of adding SAS to existing DS14 cluster, where you would avoid moving existing connections.

I sent comment to TR publisher, let’s see if we get any response ☺

IGORSTOJNOV · ‎2014-07-30

Thanks for the feedback!

Sounds logical to proceed that way, but I now have one concern...

Aggregate sas3 (online, raid_dp, mirrored) (block checksums)

Plex /sas3/plex0 (online, normal, active, pool0)

RAID group /sas3/plex0/rg0 (normal, block checksums)

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)

--------- ------ ------------- ---- ---- ---- ----- -------------- --------------

dparity 4a.125L1 4a 12 0 FC:A 0 SAS 15000 560000/1146880000 560208/1147307688

parity 4c.125L54 4c 12 1 FC:B 0 SAS 15000 560000/1146880000 560208/1147307688

data 4a.125L3 4a 12 2 FC:A 0 SAS 15000 560000/1146880000 560208/1147307688

data 4c.125L56 4c 12 3 FC:B 0 SAS 15000 560000/1146880000 560208/1147307688

data 4c.125L57 4c 12 4 FC:B 0 SAS 15000 560000/1146880000 560208/1147307688

When I switch adapters, one of these paths will become say, 5a.

Will this cause any disruption on existing aggregate configuration?

More precisely, when I boot up the controllers will they see all over their old aggregates or might they panic over something?

aborzenkov · ‎2014-07-30

If you do it offline - no.

IGORSTOJNOV · ‎2014-07-30

Yes, it will be done offline. We have downtime scheduled during which we plan to physically migrate one half of the MetroCluster, along with some network equipment and server resources - to the new building.

Hence my idea to use that opportunity to re-wire the shelves/controllers in a more resilient way. The only thing I could see as a potential issue was aggregate "persistency", or more simply put, disruption of disk paths (on SAS shelves) connected via FiberBridge... I wouldn't like to trigger any events that would cause further downtime. Is there any precaution I could take during boot-up sequence, maybe?

DOMINIC_WYSS · ‎2014-08-14

Is there any precaution I could take during boot-up sequence, maybe?

you could boot up into maintenance mode first and check if you can see all disks.

it doesn't matter on which ports it will find the disks (software disk ownership), so I don't expect an issue.