ACP cabling with a FAS2040

[CCIE17603]

Hi Everyone,

I was looking for some clear documentation on how to cable the SAS and ACP connections on a DS4243 to a FAS2040 when I came upon the Universal SAS and ACP Cabling guide. For the most part this guide was helpful except when it came to the ACP cabling instructuons. The FAS2040 only has one SAS connection per controller so the guide was good enough to call that model out and make special reference to it.

But when it came to the ACP cabling I found it difficult to follow. I understand that there are differnt cabling procedures depending if you have 1 controller or 2 controllers, attached to 1 disk shelf or multiple shelves; and the 2040 in itself is unique in that it only has 1 SAS port per controller, so you can only daisy chain the controllers to the shelves, but you cant create a complete loop because of this SAS port limitation.

However, the Universal SAS and ACP Cabling guide is unclear on how to connect the ACP. It shows that you daisy chain the shelves (and make an inter-connection if you have multiple stacks) but it doesnt show the controller connections to the shelves. I guess it just assumes you know what to do there??? Its definitely not explicit.

Can anyone point me in the right direction. What I have is an HA (2 controller) FAS2040 with 2 DS4243s in 1 stack. When I was done the only thing I could figure out was to daisy chain the ACPs in the exact same method as the SAS ports which in the end showed 2 active ACPs but only Partial Connectivity. I'm not convinced this is correct.

Thanks.

[aborzenkov · Accepted solution]

Actually, the guide is quite clear.

Controller-to-stack connections: Each storage system controller is

connected to each stack of disk shelves through a dedicated Ethernet port:

- Controller 1/A always connects to the top shelf IOM A square port in a

stack.

- Controller 2/B always connects to the bottom shelf IOM B circle port in a

stack.

In essence, you daisy all IOMs of all shelves and then connect two remaining ports to both controllers. If you have single controller, you connect just one port. How exactly you daisy chain does not really matter, but keeping suggested order makes it easier to support.

[CCIE17603]

Humm... I think I see what you are saying. So if I refer to p29 of the Universal SAS and ACP Cabling guide. See the image

You are saying connect the top controller's e0P port to the top shelf's Square port, and connect the bottom shelf's Circle port to the bottom controller's e0P port?

Thanks for clarifying.

[aborzenkov]

Right. If you have multiple stack, interconnect (daisy chain) all stacks and connect two remaining ports on the first and last shelves.

[CCIE17603]

Yeah, I see what you are saying now.  Page 31 explains it. I think the thing that tripped me up was they only show a 2 stack example.

Thanks very much for your time.

[scottgelb]

Most of us have to reference the guide even after several installs...especially with multiple stacks and acp cabling between stacks. The universal cabling guide is a great resource

On my wish list to NetApp are switches (Lsi has a sas switch. Not sure if the engenio acquisition included that product) for sas and acp...if that ever happens cabling would be much easier... We had an fc switch solution before with ds14s (not metrocluster but multiple fas controllers to the same fabric like vseries but it was discontinued).

[aborzenkov]

I am not at all convinced that having switch would be advantageous. Right now you can use short shelf-to-shelf cables for the most parts, which leaves only 4 long SAS cables for stack-to-controller. With switch you would basically need the same long SAS cables for every shelf; this would quickly become an issue due to cable inflexibility.

And ACP cabling is very simple once you understood the basic principles. I initially made an error in trying to memorize it instead of just understanding it ☺

[scottgelb]

With larger installs it can be a cabling mess. Especially with 3-4 racks and running and labeling all the cables. It works but I'd prefer a switch setup. V-series installs always look cleaner. I heard we may see something down the road but I think that is dependent on the sas switch technology.

NetApp originally tried the switches for ds14s for multiple controllers to share disks but it didn't go anywhere ad went eol after a short while.