I know that max 84 FC disks can be connected per FC loop,
How many disks for SAS loops?
for example if FAS3170A is with 840 disks, How many SAS HBA need?
For SAS (DS4243), NetApp uses the term stack, as opposed to a loop (loop is a term used for DS14 disk shelves)
With the exception of FAS2XXX systems, each SAS stack supports a maximum of 10 disk shelves, which brings the # of disk drives to 240.
I believe for FAS2XXX, the limit is 4 DS4243 per stack, which brings the # of disk drives to 96.
For a FAS3170A with 840 disk drives, you'll need 3.5 loops (4 loops). So per controller, you'll need 8 ports (4 x 2), which is 2 quad port SAS HBAs per controller.
He....was just discussing this with a customer earlier this week. Some details are.... (a lot overlapping with the previous post)
Great post - added a lot of clarity to my response. Thank you!
I think this point itself might warrant a separate thread. This ability of mix-n-match SAS/SATA shelves within a stack might well bring forth it's own set of interesting scenarios. While I do agree that they'll support mixing and matching shelves in a stack - I am sure there are some best practices and some more will come out.
(sort of like, they gave us the rope, we can chose to use it or hang ourselves with it )
For e.g. 1 filer, 3 DS4243 SATA and 3 DS4243 SAS Shelves. Which is better in the following scenarios?
If ports are not an issue (yeah - right!), then the order might be 1), 2). But if they are, then I would reckon that 2) would be more preferable than 3) and I also recall reading somewhere about this.
The recommendation is to NOT switch switch drive types more than once in a stack. So SAS --> SAS --> SATA then you should only add sata after... It would be interesting to see if support will let us hot break the middle of a stack connection to insert shelves in the middle to keep disk types contiguous in the stack... however this often isn't feasible since shelf-ids are already set and shelves likely have to move in the rack.
I often keep all FC/SAS on one head and all SATA on the other head in configurations...we have several customers with high performance needs and have sometimes had issues where the CP (consistency ever 10 seconds or nvram half full) to FC drives have higher latency since they have to wait for the CP to complete to all drives. For many this isn't an issue and perfstat is the best way to see what is happening (and the GSC performance team does a great job helping identify this edge case).
Yeah....the internal guides were clear about the best practice being to only have one SAS/SATA transition in a stack but did leave open what exactly happens if you don't (I'm more wondering for curiosity....the best practice is quite clear.
What I'm thinking of right now actually is to put the controller at the bottom of top or bottom of the rack and put the disk shelves in the middle -- so you can either grow up with one type of disk or grow up with another type of disk (I already will do the same thing in DS14 racks actually). The whole hot-break question is very interesting though.
Alternatively, there are longer SAS cables and the only requirement is that the shelves be connected up in the right order....which doesn't necessarily have to correspond to their physical location in the rack. That would lead to messier cabling but does ultimately give enough flexibility for almost any scenario I think.
What will be interesting though is the physical rack space -- if you d a 31x0 system, you're at 6U already. Given a standard 42U rack, that only leaves you with enough space for 10 DS4243 shelves....makes me think we'll see the first SAS stack often be 9 shelves. The next stack (in an adjacent rack) will grow up to 10 shelves/40U.
Hmm....very good to know. So then...if my choice is between multiple transitions (i.e. existing environment where not laid out well physically) vs. a nasty cabling mess, which is practically better?
I can find plenty of statements that there should be only one transition from SAS to SATA in a stack but I can't find anything to tell me why that is.
If this is not required why is it recommended?
Unfortunately, Adam Fox is no longer with NetApp.
At the time he wrote this comment, the only supported configuration was one crossover per SAS stack. A few months ago that limitation was removed.
It's still a best practice to avoid mixing SAS and SATA in a stack when you can and limiting the number of crossovers when you can't. You always want to try to design disk hardware and software ownership to segregate I/O for slow drives from the I/O for fast drives. Think of it like allowing a Toyota Prius cars into a NASCAR race - at some point the Prius is going to create a chokepoint and slow everyone behind them.
All the documents and diagrams I've seen for systems with only SAS shelves have the controller about 1/2 - 3/4 of the way up in the cabinet (e.g, U24+ rather than U19+). Placing the controller in the top of the cabinet may give you a few more feet/meters on the SAS cable, but then you have to stand on a ladder to do system maintenance. Placing the controller at the bottom of the cabinet is never a good idea because the controller(s) become the world's most expensive vacuum cleaners (and I've seen systems complain about being too cold if they are too close to the A/C outlets)
He he...expensive vacuum cleaners. If anything, that feels like a blinding statement of the obvious....blinding because it seems stunningly obvious but it never crossed my mind until you mentioned it.
I'll have to ponder where I want to stick controllers now....I kind of like the 3/4 up the rack for easy maintenance/installs/card adds...so hmm....hmm....