ONTAP Discussions

Adding 1TB Drive Shelf to Aggr built with 500GB Drive Shelf.

jmanzano07
6,088 Views

Hello Everyone,

          Just to verify and make sure I have this down correctly before we purchase the hardware.  
    
          We are looking to grow an existing Aggregate ( 1 RG, RAID-DP, 1 HS Disk)   that is built from a 500 GB SATA Shelf , with a new 1TB Drive SATA Shelf that we will purchase.

          Now from what I understand ....

          We will need to move the Hot Spare that is currently allocated on a 500 Gb Drive ,to one of the 1 TB drives, as well as allocate an additional 1 TB Drive as a Hot Spare to give us the total of 2 Hot Spares which can be used for either the 500 GB or 1 TB shelf.                                       

          We will need to Grow the existing Raid Group to 14 from 13 if we would like to use the newly available 500 GB ( old Hot Spare ) drive as a Data Drive , or leave as is to use as Hot Spare drive ?                   

               Currently the Aggr has a single Raid Group which is maxed, so when the new Disks are added , a second RG will be created with it's own Parity and Double Parity Drives, so the Parity and Double Parity drives of the 500 GB Raid Group do not have to be moved as no 1TB DATA Drives will be added to the existing Raid Group
                 

                Does it sound about right ?  Also, would you recommded holding off 1TB Drive Shelf ?

              

                We are down to only 3 additional Shelves that can be added total, so I’m trying to maximize the space we can get out of the remaining purchases .

             Thanks Guys  - Jose

1 ACCEPTED SOLUTION

amiller_1
6,088 Views

To be honest, I'd probably just make it a separate aggregate -- 13 disks with 1 hot spare. While you could use a 1 TB hot spare for the 500 GB disks, it's not ideal as a 1 TB hot spare replacing a 500 GB disk ends up as being a 500 GB usable disk.

Also, while you can do it by using separate RAID groups, I generally try to stay away from mixing disk sizes in an aggregate....just helps keep me sane.

Back to the setup, I'd probably....

  • keep the existing aggr as is -- 13 disks with 1 hot spare
  • add the 1 TB tray creating a new 13 disk aggregate (leaving 1 hot spare)

That would get you the exact same usable space (more actually as you'd one (1) 500 GB spare and (1) 1 TB spare) and keep things simple overall.

View solution in original post

8 REPLIES 8

amiller_1
6,089 Views

To be honest, I'd probably just make it a separate aggregate -- 13 disks with 1 hot spare. While you could use a 1 TB hot spare for the 500 GB disks, it's not ideal as a 1 TB hot spare replacing a 500 GB disk ends up as being a 500 GB usable disk.

Also, while you can do it by using separate RAID groups, I generally try to stay away from mixing disk sizes in an aggregate....just helps keep me sane.

Back to the setup, I'd probably....

  • keep the existing aggr as is -- 13 disks with 1 hot spare
  • add the 1 TB tray creating a new 13 disk aggregate (leaving 1 hot spare)

That would get you the exact same usable space (more actually as you'd one (1) 500 GB spare and (1) 1 TB spare) and keep things simple overall.

madden
6,088 Views

Two more comments:

1) Plan to keep at least one spare disk of each disk type.  If you only have 1TB disks as spares they will be selected if a 500GB drive fails, but the replacement drive you'll get from your support provider or NetApp will only be a 500GB.

2) If you will add a 2nd raidgroup to the existing aggr with these 1TB disks (which is perfectly fine and supported) pay attention to the maximum aggr size allowed on the Data ONTAP release you're running

jmanzano07
6,088 Views

Thanks for replying guys !

    My main reasoning behind the idea of moving the existing 500 GB Hot Spare to a DATA disk and allocating 2 drives from the 1 TB shelf is that it would give me 2 Hot Spares that could be used for failures on either the 500 GB Shelf , or the 1 TB Shelf.  If not, we have the scenario of having 2 disk types ( 500GB/1 Shelf + 1TB/ 1 Shelf , Aggr is RAID+DP ) that we will only have a single HS Drive for.

    I know the "best practice" recommendation is to have 2 Hot Spare disks per Disk Type, but do you guys think it warrants losing an additional 1 TB drive to be used as Hot Spare ?


    I was told we are limited to 16 TB Aggr for the release that we are running which is 7.2.5.1,

   ** I would like to verify if anyone could please point me to where the info is located for the release versions **


   So based on the 16 TB , I came up with the following. ….

       Our existing Aggregate  that is built off a single 500 GB Drive Shelf shows up as 3.99 TB’s, so I used the following to see if we went over.

       1 TB drives “ Right Size” down to 847 GB  , now I substrate the 10% ONTAP overhead it leaves me with 762 GB per drive. Ok , so now I subtract 3 ( 2 x RAID-DP, 1 HS) Drives from the Shelf which leaves me with 11,
       so 11 x 762 = 8.38 TB + 3.99 =  12.37 TB 

        Hopefuly it's right ..... 

        Also, dismissing the question of adding the new shelf to an existing 500GB Drive Aggregate, do you guys see any reason (Issues/ Failure Rate) to stay away from the 1 TB Drive Shelf on the new purchase and maybe going with the 750B or another 500GB  Shelf?

radek_kubka
6,088 Views

Have a look at this:

http://now.netapp.com/NOW/knowledge/docs/ontap/rel731_vs/html/ontap/rnote/rel_notes/reference/r_oc_rn_feat73_aggr-size-max-drives.html

Bear in mind they are talking about data disks only, so parity & hot-spares are on top of these figures.

Using that table leads to the conclusion that with 7.2.x you can have no more than e.g. 10x 1TB + 10x 500GB data disks in one aggregate (assuming 1TB 'weighs' the same as 2x500GB).

Regards,
Radek

jmanzano07
6,088 Views

  Thanks Radek ,

         Well, forget the question about me adding to the existing Aggr.....  that is unless we slip in an upgrade to 7.3 prior

http://now.netapp.com/NOW/knowledge/docs/ontap/rel731_vs/html/ontap/rnote/rel_notes/concept/c_oc_rn_feat73_aggr-size.html

"In Data ONTAP 7.3 and later releases, the total size of an aggregate is computed using the usable size of its data disks rather than the raw size of all of its disks.

In previous releases of Data ONTAP, the total size of an aggregate was computed by adding together the raw size of all disks in the aggregate, including parity disks, and regardless of the amount of disk space available to be used in each data disk. This method of computing the aggregate size could result in an aggregate exceeding the maximum allowable size even though the amount of usable space in that aggregate was much smaller than the maximum allowable aggregate size."

amiller_1
6,088 Views

He he....I guess that does make it a moot point then.

And yes, while having 2 hot spares for every disk type is definitely best practice, I often won't do it when there's only one shelf of a particular disk type (usually a conversation with the customer so they understand). Or, if all the space isn't needed right away, I'll put 12 disks in an aggregate (leaving 2 hot spares) and note to them that when/if they do actually need the space of that 13th disk they can add it on the fly (kind of having your cake and eating it too....they only have to drop to 1 hot spare when they actually need the usable space).

madden
6,088 Views

The max aggregate size calculation can be confusing.  Let me walk though an example where we'll answer the question "can I have one aggregate consisting of one raiddp raidgroup of 13 x 500GB and one raiddp raidgroup of 13 x 1TB disks?"

First, here's some output found in "sysconfig -r" that includes 500GB and 1TB SATA disk we can use as reference:

      RAID Disk    Device    HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      ---------    ------    ------------- ---- ---- ---- ----- --------------    --------------
      dparity      0c.38     0c    2   6   FC:A   -  ATA   7200 423111/866531584  423889/868126304
      dparity      1b.48     1b    3   0   FC:B   -  ATA   7200 847555/1735794176 847827/1736350304

For releases prior to 7.3

From the docs we have this statement:

In previous releases of Data ONTAP, the total size of an aggregate was computed by adding together the raw size of all disks in the aggregate, including parity disks, and regardless of the amount of disk space available to be used in each data disk.

So to determine max disks in an aggregate we'd sum the "Phys" capacity of each data + parity disk we want in the aggregate, and this must be less than 16,777,216MB (16TB) to be permitted.

So with our question:

13 x 423889 =  5510557

13 x 847827 = 11021751

              16532308 MB

Since 16,532,308 < 16,777,216 it would ber permitted.


For releases 7.3 and later

From the docs we have this statement:

In Data ONTAP 7.3 and later releases, the total size of an aggregate is computed using the usable size of its data disks rather than the raw size of all of its disks.

So to determine max disks in an aggregate we'd sum the "Used" capacity of each data disk we want in the aggegate, and this must be less than 16777216MB (16TB) to be permitted.

So with our question:

11 x 423111 = 4654221

11 x 847555 = 9323105

             13977326 MB

Since 13,977,326 < 16,777,216 it would ber permitted.

So in summary, you could run with the aggr/raidgroup config proposed in the question above running in both the 7.2 and 7.3 release families.

You also had a question about adding 1TB disks (vs 750GB or 500GB) and from an reliability perspespective the hardware has similar failure rates.  When choosing a disk size I'd focus on your performance requirements (bigger disks support the same number of IOPs as smaller ones of same type/rpm) and availability requirements (a raid rebuild of a larger disk takes longer), and balance these with cost (using bigger disks your price/GB is less, support costs are less, rackspace is less, power/cooling is less).

Regarding spare disks, if you have a support contract and will get replacements within a reasonable timeframe (NBD or better) I'd say having one spare disk per disk type is usually sufficient.

Lastly, the idea to have two spare disks and potentially add one disk to your aggregate at the last minute (in case of emergency) is in most configs a bad idea.  If you have a full aggregate and add one disk the freespace in the aggregate will be most plentiful on that single disk you just added.  Incoming writes are going to have a preference to hit that disk since it has plentiful freespace, and reads of recently written data will also have to come from that disk.  Over time the system will become balanced again (you can run a reallocate to try and speed it up -- if on 7.3 best to use physical reallocation using a command like reallocate start -f -p /vol/VOLNAME) but in the meantime your aggr performance may be much lower due to waiting on this busy disk.  The only situation I might think about adding a single disk is if I really don't care at all about peformance (example: snapvault archive where WAN links are the bottleneck).

Hope that helps.

jmanzano07
6,088 Views

   It's good to know that failure rate for the 1tb drives are the same as the 500's ..... I guess that will  make me feel a little bit more at ease with having a single disk as a HS for that disk type.   Luckily we have the 4 hour Support Edge Contract so we receive replacements rather quickly  

  Either way, worse case you can make a Best Buy or Comp USA Run , right      ,,,,, ,  just kidding.... just kidding   

Public