Network and Storage Protocols

FAS2020 Optimize Storage Capacity and Upgrade optimization for the Future. (RAID-DP & RAID4 Active/Passive Controllers)

adrianlowe
16,417 Views

Just out of interest, on the FAS2020 the common issue which crops up is with the available "useable storage capacity".


I have lots of recommendations around the FAS2020 depending on what the customers requirements are.

Typically if the customer has just the FAS2020 with 12 internal disks with no additional disk shelves. They typically want to maximize the capacity available from the controller/s.

Depending on the customers performance requirement from the Fibre/Ethernet ports/controllers CPU. If one controller is able to handle the work load I would recommend the following setup:

An Active/Passive Configuration.

  • Configuring Controller 1 with RAID-DP with 9 disks with 1 spare.
  • Configuring Controller 2 with RAID4 as 2 disks.
  • Ensuring cluster failover configuration is set up to failover.

This is to maximize the useable storage capacity and Spindle performance from the internal disks.

Once an additional disk shelf has been added. I would recommend creating a new aggregate consisting of all the disks in the new disk shelf.

I would migrate controller 2's root volume to the new aggregate.
Destroy the RAID-4 aggregate to re-claim the internal disks for the internal aggregate on controller 1.

I have added the suggestion above to be useful to others. Any additional recommendations to improve this would be appreciated. Else I hope it helps others achieve the best they can with their storage device

Thanks in advance,

Adrian

15 REPLIES 15

rorzmcgauze
16,395 Views

Good post.

This is the same advice I give our customers when they are "worried" about the capacity of the 2000 series. Also feel you need to stress the active passive bit with them but capacity over performance is the usuial driving factor at this end of the market.

I believe there is also an option you can turn off on ctrl2 so it stops error messages about no hot spares.

dwork
16,395 Views

Great information guys.  I'm fairly new to the storage arena and could use some recommendations.  I'll be migrating data from an older EMC box to a FAS2020 HA with 12x600 15K drives.  The eventual load is roughly 1TB of VM storage and 2TB of CIFS shares (before any dedup, etc).  Your post seems to be the right way to go and leave the second controller for failover.  However, the VM and CIFS networks are on different VLANs and I'd like to have redundancy on the physical links.  So I guess the question is:

A) Active/Passive and setup a vif with VLAN tagging for both VLANS (Cisco 3560 switch)?

B) Active/Active and VM's on one controller and CIFS on the other (obviously losing some disk space)?

Thanks in advance.

adrianlowe
16,395 Views

Hello,

Both setups up would be suitable. It may depend and be determined by the capacity requirement. Option 1 would provide a higher useable capacity.

If the capacity isn't too critical yet bandwidth is then you may want to opt for options 2 to enable you to achieve a higher throughput i.e. 2GB total from Controller 1 and 2GB total from controller 2 as well.

Overall it is a trade of depending on your customers requirement/s. If you intend to upgrade in the future adding additional disk shelves and performance isn't exceeding 2gb throughput then I would personally go with options 1 so you can add an additional disk shelf at a later day and get the storage up in an Active/Active congiuration with higher capacity across the board and utilising all the ethernetports.

Hope this helps

adrianlowe
16,395 Views

p.s. don't forget you can just apply VIF's on the ports on both controllers to protect the physical port redunancy.

dwork
16,395 Views

Your help has been much appreciated and I hope I can ask for little more advice.

I've setup an Active/Passive setup with most of the disks on Controller 1.

e0a and e0b are vif lacp and I've created vif-a, vif-b VLANs on top.

I've setup Etherchannel on the Cisco side (3560G) and all is well.  If I unplug one interface, there's no downtime.  Awesome (and there is a similar setup on Controller 2 so it fails over).

However, it appears that it is not aggregating the links.  On Cisco while I'm transferring some massive data, the 2nd port doesn't pass traffic, only the first.

Is this to be expected?  Does the 2nd adapter only kick in when the 1st is maxed?

aborzenkov
16,395 Views

yes, this is expected. 3560 supports packet distribution by MAC or IP - in both cases all traffic between the same pair of systems will always use the same physical interface (unless interface failed).

On NetApp side you could additionally set distribution to round-robin (which is not recommended) or port which will take in account TCP/UDP port. The latter may offer better distribution across aggregate members if traffic is multithreaded.

Do not forget that load distribution for incoming traffic (from NetApp point of view) is configured on switch - there is nothing Netapp can do about it; and load distribution for outgoing traffic is configured on NetApp - again, switch does not do anything here.

In general load distribution is effective only with large number of clients; but every single client will normally run over dedicated interface.

adrianlowe
16,395 Views

Hello Dwork,

On top of the previous recommendations you can also apply IP aliasing. Typical 1 IP alias per the # of physical interfaces per VIF. i.e. for a FAS2020 it would be 1 VIF IP plus 1 additional IP alias. This is if the VIF is set to IP based and not MAC based of course 😄

This enables the system to load balance a bit better when serving data due to the IP pairing when getting requests in from multiple hosts.

Does this help answer your question 🙂

I hope it helps,

Adrian

kwellmaker
16,395 Views

Can you point me to this option to turn off the messages about not having a spare?

aborzenkov
16,395 Views

Turning off "not enough spares" warning is not possible with RAID4.

IRVING_POPOVETSKY
11,175 Views

I've done this on two FAS2040 filers I run.   Once with a NetApp engineer (protecting the innocent) and the second time by myself.

The process:

  1. Configure Controller A as normal  (9-disk RAID DP)
  2. Configure Controller B as normal (3-disk RAID DP) - but don't put any data here yet
  3. Convert Controller B's aggregate to raid4   (from maintenance mode:  aggr options aggr0 raidtype raid4)
  4. one of the disks will be converted to a spare, FIGURE OUT WHICH ONE BEFORE PROCEEDING
  5. still in maintenance mode on controller B,  remove ownership of the spare disk from controller B  (still maintenance mode:  disk remove_ownership disk_id)
  6. assign ownership of the disk to controller A  (disk assign disk_id)

Some Advice

  1. Don't put anything you care about on Controller B
  2. Learn to love the permanent Low Spares warning on Controller B

aborzenkov
11,175 Views

Nitpicking – you do not need to be in maintenance mode for any of this ☺

IRVING_POPOVETSKY
11,176 Views

I'm sure NetApp tech support really appreciates you saying that

Fair enough,  you can use "priv set advanced".   I actually did it this way.   Maintenance mode is just safer, probably best for those starting out with their new baby-FAS filers.

KARLOCHACON
11,175 Views

follow this trick cool now I have more one disk for my aggr0 in Controller1

Still my big doubt is why I get just a little space with 12 Hard Drive SATA 1TB

Controller 1 - 9 Disk - DP - Usable space 3.5 TB.... - 2 Parity 1 Spare.....leaving 6 Disk for Data so why 3.5 TB usable?

Controller 2 -  3 Disk - RAID 4 - Usable like 630 GB.

anyone can explain me this? I am getting questions why is this behavior......

Weird thing is why every aggregate needs a spare... even when is DP which is similar to RAID 6 so 2 disk to fail...

thanks guys please help me with the question above about space utilization

adrianlowe
11,175 Views

Hello Carlos,

A few more recommendations for you too... Theres no real need to have 3 disks for controller 2. If its not really running the production data then you can assign the spare disks to controller01. The controller02 is literally sat there waiting for controller01 to fail but isn't really hosting anything i.e. the Active/Passive configuration above. The RAID4 should also provide some level of failure on the controller02.

Regarding the controller01... in this setup this is hosting your critical data and application data so you want to cover all aspects. the NetApp best practice is to actually have 2 spare disks per aggregate BUT given the limited number of disks and capacity you would see from a small aggregate, many people opt to have the 1 spare disk because of the support contract (either Next business day or same day 4 hour replacement)

There are a few other things you can try to reclaim space, one is the aggregate snap reserve. No one ever restores an entire aggregate from the aggregate snapreserve because it is typically done at the flexvol layer now so it can be more granular. therefore you can reclaim this storage space. Depending on what Data Ontap version your running, this is usually 5% of the entire aggregate space.


I personally have two methods for this...
1. If a customer is really tight on space and really needs that additional capacity... I remove the Aggregate snapreserve, this then allows 5% additional usable capacity.
2. If the customer is not tight on capacity and does not have the urgent requirement, then I leave the 5% snapreserve on the aggregate. Why I hear you ask... this is because I like to see it as a safety net... so if and when I/another consultant returns to do a Data ONTAP upgrade, The upgrade requires a min of 3% free space per aggregate on the controller... if its 6months, 1 year etc time and the customer has max out the storage... at least we can proceed with the requested upgrade and hopefully followed by some form of audit to see if they are utilising the controller storage efficiency in the best manor. I hope this helps,

I know this doesn't answer your question as to why you only get x amount of usable capacity against the RAW figures. There are multiple other blogs on this though:
https://communities.netapp.com/message/10834
https://communities.netapp.com/community/netapp-blogs/efficiency/blog/2009/08/10/how-to-measure-storage-efficiency--part-i--raw-storage
http://netapp-blog.blogspot.com/2009/03/netapp-usable-space-calculation-formula.html

adrianlowe
11,175 Views

Hello All,

I just thought I would update this for those of you who find it useful. This post was ideally discussing the easiest way to configure the FAS2020 with the maximum capacity and performance with the easiest method to expand capacity in the future. By the way this post also applies to the FAS2040 or any controllers which has internal disks.

I mentioned the following before:

This is to maximize the useable storage capacity and Spindle performance from the internal disks.

Once an additional disk shelf has been added. I would recommend creating a new aggregate consisting of all the disks in the new disk shelf.

I would migrate controller 2's root volume to the new aggregate.
Destroy the RAID-4 aggregate to re-claim the internal disks for the internal aggregate on controller 1.

Although this method is achievable and is straight forward, I thought I would post a few other options (given that the option above isn't the most ideal method, despite being the easiest).

Option 1. Do as the post suggests. Create a new RAID DP aggregate (for controller02) on the new disk shelf, migrate the root volume over from the old RAID4 aggregate. Trash the old RAID 4 aggregate, reassign the RAID4 disks to controller01 and assign the disks to the original RAID DP aggregate (controller01).

Pros: This will ensure you have a clear logical aggregate layout i.e. all internal disks will be controller01s aggregate with a spare disks, all external disks will be controller02s, aggregate with a spare.

Cons: This would potentially create a hot spot on the original RAID DP aggregate (controller01) and therefore you may experience some performance issues on that aggregate.

Option 2. Leave the Existing configuration as it is (10 disks on controller01, 9 in aggregate and 1 spare. With 2 disks on controller02 in a RAID4 config). When you add the additional shelf/storage (providing they are the same disk type/size as the internal disks AND providing the RAID4 aggregate only consists of the root volume i.e. no actual production data) you can just upgrade the RAID4 to a RAID DP aggregate and then expand the aggregate to scale up across the new disk shelves.

Pros: Easy to configure with the least amount of disruption. You achieve the same capacity whilst protecting both controllers with RAID DP with spares.

Cons: This will not provide a nice aggregate layout and AND this is also dependent upon having the same disk size and type internally as well as externally else it wouldn’t work and option 1 or 3 would need to be considered.

Option 3. Create a new RAID DP aggregate (on controller01) with the new disks (depending on the type & size – ideally you need to meet the same capacity and performance as the original RAID DP aggregate (on controller01)) but obviously now you will have a slightly larger aggregate than the original RAID DP aggregate on the internal disks. You can then migrate all the data AND root volume over to the newly create aggregate. Once data and root vol have migrated successfully. Reboot the controller etc to ensure it is stable. You can then offline, destroy and zero the original raid-DP aggregate (owned by controller01), and then re-assign the disks to controller02 to upgrade the original RAID4 aggregate to RAID DP and expand the aggregate to with the internal disks.

Pros: This will ideally provide a clean aggregate layout (all internal disks on one aggregate (controller02) and all external disks on another aggregate (controller01) ), it will also avoid having any hot spots on the aggregates.

Cons: This will be the most disruptive and taskly method but with provide the best results.

I hope this helps with existing or future installations. Again as always any additional suggestions or recommendations are appreciated.

P.S. I'm sure many of you are happy not the FAS2040 now sit where the FAS2020 to help with the network configurations etc

Public