Ian
My name is Trey. I am a Consulting SE at NetApp focused on Ethernet Storage Technologies. I am also a former Cisco employee so can answer some of the architecture specifics of the 3750E. One of the changes or introductions with the E version of the 3750 was the enhancement to the stacking architecture Cisco calls Stackwise to Stackwise+. The Stackwise interconnect in the older 3750 (non E series) had a 32 Gigabit per second bandwidth. Now that 32Gbps path was actually two counter-rotating 16Gbps paths yielding a bi-directional total.
Check out this link for details in differences between Stackwise and Stackwise+ on the 3750E platform.
http://www1.cisco.com/en/US/prod/collateral/switches/ps5718/ps5023/prod_white_paper09186a00801b096a.pdf
There are several enhancements of the Stackwise+ technology and the Catalyst 3750E. I find that many customers haven't gotten the update on what differences actually existed between the 3750 and the E-series so sometimes folks referring to the 3750E will quote specs from the 3750. Here are a few differences.
Catalyst 3750G
Port ASIC controls ring, There is no switch fabric
3750 has external TCAMs
3750 only runs StackWise mode 32Gbps (Bi-Directional 16Gbps)
1:1.6 Oversubscription - Blocking Architecture
Source Strip of Frame from Stacking Ring
Catalyst 3750E
Switch fabric allows local switching
128Gbps Stacking Bandwidth (Bi-Directional 64Gbps)
Non-Blocking Switch Architecture
Destination Strip - Spacial Reuse from Stacking Ring
So, those differences produce the following architecture limits in the 3750 that are not present in the 3750E.
3750 does not have a local switch fabric preventing it from being able to locally switch, it must send a frame on the ring to determine its destination. The sources of this are centered around the TCAM subsystem which the 3750 uses versus the 3750E having TCAMs in local port ASICs. TCAMs are basically the memory in a switch that is used to perform lookups to execute L2 switching or L3 routing. < Trying not to write a book here.....
Read this for an explanation of resource limits which can be reached on the 3750 (not the E series)
http://www.cisco.com/en/US/products/hw/switches/ps5023/products_tech_note09186a00801e7bb9.shtml
Now, with regard to use of 3750E with a 10Gbps workload on ports which are considered uplinks. You are correct in that Cisco is positioning the 3750E 10G ports as uplinks. The changes in the E architecture enable the switch to be used in different, more data center oriented workloads. I would tell you that it is always preferred that we deploy Nexus 5000s for data center Ethernet Storage connectivity. However, realities are that many have 3750s which they are using in data centers. The E Series introduction of 10Gbps and architecture changes enable it to support 10Gbps successfully and can tell you that we have several customers who are using E series switches for 10Gbps connectivity. They are running fine and happy 3750E with NetApp customers. The link to provided to the episteme.arstechnica.com site was/is a configuration issue versus architecture of the switch. The topic on running LACP is odd to me because when I research the releases presently available for the 3750E I see no restriction documented for LACP. There is even a section regarding EtherChannel Configuration Guidelines which don't depict this limitation. However, not having access to the 3750E immediately I can state that if you can't configure LACP the simply use Static Etherchannels. I find that there is so much confusion in general about this because so many people use so many different terms. In a nutshell, it is our suggestion that you always run LACP because of the benefits of that protocol, right up until the point that you can't run it. When you can't simply run static etherchannels. The configuration document linked to for the 3750E above actually states this as well.
LACP in Cisco command lingo is "channel-group mode active" or "channel-group mode passive". When you are configuring LACP to a NetApp always use "active".
Static Etherchannels in Cisco command lingo is "channel-group mode on"
I detail this entire concept here along with all the associated NetApp commands:
http://communities.netapp.com/blogs/ethernetstorageguy/2009/04/04/multimode-vif-survival-guide
With regard to configuration best practices in general the same best practices that exist for Gigabit Ethernet deployments with NetApp apply to 10Gbps. There aren't typically any differences that we would call out.
I have documented a few recomendations for mixing 10G with 1G configs here:
http://communities.netapp.com/blogs/ethernetstorageguy/2009/09/13/8021q-vlan-trunking-with-mixed-jumbo-standard-ethernet-frames-using-second-level-vif...
The concept of Multi-Chassis Port Aggregation is documented here as there is a different technology for just about every type of Cisco switch you wish to leverage this functionality with:
http://communities.netapp.com/blogs/ethernetstorageguy/2009/09/23/virtual-port-channels-vpc-cross-stack-etherchannels-multichassis-etherchannels-mec--...
Now our best practices for VMware vSphere have recently been updated and the formal release of our best practices guide is scheduled for February 18th. It has been available on lulu.com for a few weeks now ( TR-3749 ) < for those of you who want it in book form. Next week you can get it online via PDF from our Technical Library in TR-3749 we document our Networking best practices as they relate to vSphere with NetApp for NFS to include 10Gbps and Jumbo Frames.
Finally, I would suggest a couple of things regarding cabling connectivity as it is a hot topic because of cost. If the 3750E is near (within 5 meters) of the NetApp then you are a candidate for using twinax cabling. Twinax cabling is available in 1, 3 and 5 meter lengths as is far less expensive than the optical 10G interfaces you would typically be required to deploy. When using twinax cabling the cost of 3 1Gbps ports is the equivilent of a single 10Gbps port. Just doing some rough math. A twinax cable includes the SFP+ interface at both ends of the cable for about $260 list price. When using optical 10G you must purchase a 10G optical SFP both both sides of the connection which are typically around $1900 list each. When specifically talking about NetApp 10G interfaces we sell a 10G card which supports twinax cabling and actually sell Cisco branded twinax cables. I have detailed the different card options here: NetApp 10G Options
You will see a X1107 10G card which supports twinax and should be used if you are within the distance requirements. That same X1107 card can have a optical SFP inserted to support longer distances. Now other NetApp 10G cards don't have a modular interface selection type so the cost of that optic is often bundled into the total price of the card, so using twinax saves you money.
Now, if you have followed along to this point you are likely going to say "Trey, twinax is great but it don't work in a Catalyst 3750E, they have X2 slots for 10G". Nice catch which is why Cisco released the OneX converter, another GEM of a solution which converts an X2 slot into a SFP+ interface and supports the use of twinax cabling. You will notice on the link I provide below there is also a compatibility matrix for this and every other 10G optic Cisco sells. Go to the link and look up your switch (3750E is on it).
OneX Converter
http://www.cisco.com/en/US/prod/collateral/modules/ps5455/data_sheet_c78-547521.html
Hope this helps a little,
Trey
