Can't create flash pool

[HDD]

Hi Everyone,

I have a FAS2554 with 20x SATA and 4 x SSD. So, they're split between the two controllers.

My problem is that I am having a difficult time creating a Flash Pool, to leverage the SSDs.

fas2554-b> aggr status -v

           Aggr State           Status                Options

          aggr0 online          raid_dp, aggr         root, diskroot, nosnap=off, raidtype=raid_dp,

                                64-bit                raidsize=12, ignore_inconsistent=off,

                                hybrid_enabled        snapmirrored=off, resyncsnaptime=60,

                                                      fs_size_fixed=off, lost_write_protect=on,

                                                      ha_policy=cfo, hybrid_enabled=on,

                                                      percent_snapshot_space=0%,

                                                      free_space_realloc=off

                Volumes: vol0

                Plex /aggr0/plex0: online, normal, active

                    RAID group /aggr0/plex0/rg0: normal, block checksums

fas2554-b> aggr add aggr0 -T SSD 2

aggr add: Starting a new raid_dp RAID group;

The number of disks specified must be at least three

times the number of plexes in the target aggregate 'aggr0'.

fas2554-b>

I even tried this to no avail.

fas2554-b> sysconfig -r

Aggregate aggr0 (online, raid_dp, hybrid_enabled) (block checksums)

  Plex /aggr0/plex0 (online, normal, active)

    RAID group /aggr0/plex0/rg0 (normal, block checksums)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

      --------- ------          ------------- ---- ---- ---- ----- --------------    --------------

      dparity   0b.00.4         0b    0   4   SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      parity    0b.00.6         0b    0   6   SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      data      0b.00.8         0b    0   8   SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      data      0b.00.10        0b    0   10  SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      data      0b.00.12        0b    0   12  SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      data      0b.00.14        0b    0   14  SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      data      0b.00.16        0b    0   16  SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      data      0b.00.18        0b    0   18  SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

      data      0b.00.20        0b    0   20  SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

---------       ------          ------------- ---- ---- ---- ----- --------------    --------------

Spare disks for block checksum

spare           0b.00.22        0b    0   22  SA:A   0  FSAS  7200 5614621/11498743808 5625872/11521787400

spare           0b.00.0         0b    0   0   SA:A   0   SSD   N/A 381304/780910592  381554/781422768

spare           0b.00.2         0b    0   2   SA:A   0   SSD   N/A 381304/780910592  381554/781422768

Partner disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)

---------       ------          ------------- ---- ---- ---- ----- --------------    --------------

partner         0a.00.3         0a    0   3   SA:B   0   SSD   N/A 0/0               381554/781422768

partner         0a.00.1         0a    0   1   SA:B   0   SSD   N/A 0/0               381554/781422768

partner         0a.00.5         0a    0   5   SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.7         0a    0   7   SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.23        0a    0   23  SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.21        0a    0   21  SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.19        0a    0   19  SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.17        0a    0   17  SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.15        0a    0   15  SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.13        0a    0   13  SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.9         0a    0   9   SA:B   0  FSAS  7200 0/0               5625872/11521787400

partner         0a.00.11        0a    0   11  SA:B   0  FSAS  7200 0/0               5625872/11521787400

fas2554-b> aggr add aggr0 -g rg0 -d 0b.00.0

aggr add: Disk type does not match type of disks in the aggregate.

fas2554-b>

Not sure where I went wrong. I am running 7 mode. Any help would be appreciated it. Thanks!

[NAYABSK]

Hi,

May i know the version of ontap you are using and can you try changing the spare count to zero.

option raid.min_spare_count 0

Thanks,

Nayab

***If my reply helped you to fix the issue, can mark it as answer to help others***

[Sahana]

Hi,

The RAID option raid.min_spare_count does not apply to Flash Pool SSD RAID groups.

Try without raid group.  You can specify the SSDs by ID or by using the disk_size and ndisks parameters. You do not need to specify a new RAID group; Data ONTAP automatically puts the SSDs into their own RAID group.

Refer Creating a Flash Pool aggregate

[HDD]

Hi Sahana,

I tried without the raid group and this is what I am getting.

fas2554-b> aggr add aggr0 -d 0b.00.0
aggr add: Starting a new raid_dp RAID group;
The number of disks specified must be at least three
times the number of plexes in the target aggregate 'aggr0'.
fas2554-b>

I then tried the disk_size and ndisk parameter and this is the outcome:

fas2554-b> aggr add aggr0 -T SSD 2@400
aggr add: Starting a new raid_dp RAID group;
The number of disks specified must be at least three
times the number of plexes in the target aggregate 'aggr0'.
fas2554-b>

Please let me know if you need more information. Thanks!

[HDD]

Hi Nayabsk,

Ontap version is :

fas2554-b> version -v
NetApp Release 8.2.3P3 7-Mode: Tue Apr 28 14:48:22 PDT 2015  <1O>
fas2554-b>

As for changing spare count to zero:

fas2554-b> options raid.min_spare_count 0
option raid.min_spare_count: Cannot be set to 0 if there are more than 16 drives or any RAID4 groups configured.
fas2554-b>

Please let me know if you need any other information. Thanks!

[NAYABSK]

As i see you are trying to create using 2 disks which would be RAID 4 based on your version of Netapp, But if you are creating RAID4 least you need to have 1 disk as hot spare if i am not wrong.

But can you try as below and let me know.... As i guess your default Raid Group is RAID_DP and you just don't have enough disks to create ( RAID_DP needs 3 disks least )

aggr add aggr0 -t raid4 -g new -T SSD 2@400 

Thanks,

Nayab

[HDD]

Hi Nayab,

I believe your suggestion appears to work:

fas2554-b> aggr add aggr0 -t raid4 -g new -T SSD 2@400
Note: preparing to add 1 data disk and 1 parity disk.
Continue? ([y]es, [n]o, or [p]review RAID layout) p
The RAID group configuration will change as follows:
RAID Group                Current               NEW             RAID Type
----------                -------               ---             ---------
/aggr0/plex0/rg0          9 disks               9 disks         raid_dp
/aggr0/plex0/rg1                                2 disks         raid4

Continue? ([y]es, [n]o, or [p]review RAID layout) n
fas2554-b>

Will this pose any issues if I do the way you suggested? Also, I was reading up on this article: http://www.datacenterdude.com/netapp/swimming-deep-end-netapp-flash-pools/

The the article, it talks about a similar setup as my FAS2554. Muy question, would it be possible to do something similar? And thank you!

[NAYABSK]

Good to know it kicked in 🙂  Don't forget to mark it as solution to help out others 🙂

The only threat would be when there is SSD failure you will not have FLASH POOL working until you replace the failed disk as it is RAID_4 and you don't have any spare disk.

That's why Netapp made it a must to have hot spare for RAID4  

Thanks,

Nayab

***If my reply helped you to fix the issue, can mark it as answer to help others***

[NAYABSK · Accepted solution]

The way i suggested is the only possible way in ur condition as u have only 2 SSD, this is the best we can do out of 2 SSD

[HDD]

Thank you for your help!

[NAYABSK]

As my reply helped you, can help to mark it as accepted solution so that it would be helpful to others with same issue.

Thanks,
Nayab

[HDD]

Hi Nayab,

I believed I have already marked it as solved, by you. A couple of follow up questions, if you don't mind?

1) is there a way I can combine my 4 SSDs into some sort of Pool that can be used by both Controllers/Nodes?

2) When you say the FLASH POOL will not work unless I replace the failed SSD disk; other than performance

 loss, no data loss or such right?

Thanks!

[NAYABSK]

1) is there a way I can combine my 4 SSDs into some sort of Pool that can be used by both Controllers/Nodes?

At a given time only one controller can own or access a disk/SSD so you cannot share them

Ex:- You cannot have one aggr0 for both the controller's 

2) When you say the FLASH POOL will not work unless I replace the failed SSD disk; other than performance

 loss, no data loss or such right?

As you have RAID4 you will have  1 data SSD and 1 parity SSD

What if your parity SSD fails ?

You will still have ur flash pool working and once you replace the SSD it will start to build the parity ( During this time you will have performance hit ) and your flash pool is back to work

What if my Data SSD fails?

Your flash pool won't be working until you replace the SSD, once you replace the data is bulit on to the disk from the parity and your flash pool is back to work

Note:- I would suggest you to get 2 more  SSD and assign them as hot spare 1 to each controller to avoid any discrepencies. 

Follow Netapp best practices for better peace of mind 🙂