Flash Pool with a total of 4 SSD

[alessice]

Hi,

we buy a new FAS2720 with 4 SSD and 8 HDD and 3 DS212C with 12 HDD each, for a total of 4 SSD 960GB and 44 HDD 4TB.

Now I'm configuring Flash Pool, like the old FAS2554 (4 SSD + 20 HDD) that will be migrated on the new FAS.

So I use the same command use some years ago on the 2554:

storage aggregate modify -aggregate cluster4_01_NL_SAS_1 -hybrid-enabled true

storage pool create -storage-pool StoragePool_c4_Cache1 -disk-list 1.0.0,1.0.10,1.0.1

now

storage aggregate show-spare-disks -disk-type SSD

show one disk as spare on node02 (the same as 2554).

> storage aggregate show-spare-disks -disk-type SSD
                                                                      
Original Owner: cluster4-02
 Pool0
  Spare Pool

                                                             Usable Physical
 Disk             Type   Class          RPM Checksum           Size     Size Status
 ---------------- ------ ----------- ------ -------------- -------- -------- --------
 1.0.11           SSD    solid-state      - block           894.0GB  894.3GB zeroed

But when I try to add flash-pool to aggregate I have this error:

storage aggregate add cluster4_01_NL_SAS_1 -storage-pool StoragePool_c4_Cache1 -allocation-units 4 -raidtype raid4

Error: command failed: storage pool "StoragePool_c4_Cache1" does not have enough spare allocation units. Use "storage pool show-available-capacity" to get the spare allocation units assigned to node cluster4-01.

storage pool show-available-capacity 
                           Storage SyncMirror Allocation Unit  Total
Node       Storage Pool    Type    Pool       Unit size  Count Usable Size
---------- --------------- ------- ---------- ---------- ----- -----------
cluster4-01 
           StoragePool_c4_Cache1 
                           SSD     Pool0         670.5GB     2      1.31TB
cluster4-02 
           StoragePool_c4_Cache1 
                           SSD     Pool0         670.5GB     2      1.31TB
2 entries were displayed.

 So, what is changed on FAS2720 and/or ONTAP 9.9.1? I remember that in the past an SSD spare can be shared from node.

Now I'm in difficulty if I cannot create my flash pool.

Thanks

[TMACMD · Accepted solution]

By default, two allocation units are auto-assigned to each node. That is why when you try to specify 4 units, it is failing. It would probably work with 2 units

If you >>really<< want all four units to be part of the same aggregate, you will need to reassign the other two units:

STORAGE POOL REASSIGN

[alessice]

Ok, probably understand what is wrong, on my documentation I write "-allocation-units 4" but was "2", because I need to add flash pool to 2 aggregate and if reassign a total of 4 AU to only one node is impossible to do this.

Here is the status of flash-pool on current FAS 2554 that I want replicated on FAS 2720:

cluster2::> storage pool show-disks -storage-pool ClusterCache 

Storage Pool Name: ClusterCache

             Disk
    Disk     Type    Usable Size  Total Size
    -------- ------- ----------- -----------
    1.0.0    SSD         372.4GB     372.6GB
    1.0.1    SSD         372.4GB     372.6GB
    1.0.2    SSD         372.4GB     372.6GB
3 entries were displayed.

cluster2::> storage pool show-disks -storage-pool ClusterCache -instance 

          Name of Storage Pool: ClusterCache
              Name of the disk: 1.0.0
                     Disk Type: SSD
              Disk Usable Size: 372.4GB
                    Total Size: 372.6GB
                 List of Nodes: cluster2-node01, cluster2-node02

          Name of Storage Pool: ClusterCache
              Name of the disk: 1.0.1
                     Disk Type: SSD
              Disk Usable Size: 372.4GB
                    Total Size: 372.6GB
                 List of Nodes: cluster2-node01, cluster2-node02

          Name of Storage Pool: ClusterCache
              Name of the disk: 1.0.2
                     Disk Type: SSD
              Disk Usable Size: 372.4GB
                    Total Size: 372.6GB
                 List of Nodes: cluster2-node01, cluster2-node02
3 entries were displayed.

cluster2::> storage pool show-disks -storage-pool ClusterCache           

Storage Pool Name: ClusterCache

             Disk
    Disk     Type    Usable Size  Total Size
    -------- ------- ----------- -----------
    1.0.0    SSD         372.4GB     372.6GB
    1.0.1    SSD         372.4GB     372.6GB
    1.0.2    SSD         372.4GB     372.6GB
3 entries were displayed.

cluster2::> storage aggregate show-spare-disks -disk-type SSD
                                                                      
Original Owner: cluster2-node01
 Pool0
  Spare Pool

                                                             Usable Physical
 Disk             Type   Class          RPM Checksum           Size     Size Status
 ---------------- ------ ----------- ------ -------------- -------- -------- --------
 1.0.3            SSD    solid-state      - block           372.4GB  372.6GB zeroed

cluster2::> storage pool show -storage-pool ClusterCache -instance

                        Storage Pool Name: ClusterCache
                     UUID of Storage Pool: 3a072807-eec0-11e6-bd65-00a098c6ed93
           Nodes Sharing the Storage Pool: cluster2-node02, cluster2-node01
          Number of Disks in Storage Pool: 3
                     Allocation Unit Size: 279.2GB
      Allocation Unit Data Size for RAID4: 186.2GB
    Allocation Unit Data Size for RAID-DP: 93.08GB
   Allocation Unit Data Size for RAID-TEC: -
                             Storage Type: SSD
                 Storage Pool Usable Size: 0B
                  Storage Pool Total Size: 1.09TB
                         Is Pool Healthy?: true
                State of the Storage Pool: normal
  Reason for Storage Pool Being Unhealthy: -
Job ID of the Currently Running Operation: -

cluster2::> storage pool show-aggregate -storage-pool ClusterCache  -instance

                     Name of Storage Pool: ClusterCache
                                Aggregate: sata_data_1
                                 Capacity: 558.5GB
Number of AU's Assigned to This Aggregate: 2
                      Original Owner Name: cluster2-node01
                                     Node: cluster2-node01

                     Name of Storage Pool: ClusterCache
                                Aggregate: sata_data_2
                                 Capacity: 558.5GB
Number of AU's Assigned to This Aggregate: 2
                      Original Owner Name: cluster2-node02
                                     Node: cluster2-node02
2 entries were displayed.

So, I have two nodes, two aggregate set as hybrid, 4 SSD 960GB:

storage aggregate modify -aggregate cluster4_01_NL_SAS_1 -hybrid-enabled true
storage aggregate modify -aggregate cluster4_02_NL_SAS_1 -hybrid-enabled true

I will create a storage pool with 3 SSD, so one remains as spare, and assign 2 AU per aggregate with raid4 type:

storage aggregate add cluster4_01_NL_SAS_1 -storage-pool StoragePool_c4_Cache1 -allocation-units 2 -raidtype raid4
storage aggregate add cluster4_02_NL_SAS_1 -storage-pool StoragePool_c4_Cache1 -allocation-units 2 -raidtype raid4

I prefer raid4 instead of Raid-DP in order to have more space on flash-pool.

All should works fine, correct? Any suggestions?

Thanks

[TMACMD]

You are looking at two methods for FlashPool. The old method used WHOLE SSDs. The NEW method uses PARTITIONED SSDs.

All 4 SSDs should be modified to be "owned" by the same node.

Create the storage pool with 3 x SSDs. 

Modify the aggr to be hybrid enabled

Finally "add-disks" to each of your two AGGRs. Like you mention, use a storage-pool and specify raid4 to maximize usage.

You can only do one at a time. You need to wait for the first to finish before doing the second aggr. It will complain and not run if it is still adding-cache.

[alessice]

Ok, after set as 2 the number of AU per aggregate I have add successfully the flash-pool to the two aggregates.

Now ONTAP but also AIQ warning me about a low number of SSD spare on node01, since of 4 SSD there is only one spare assigned to node02:

cluster4::> storage pool show-disks 

Storage Pool Name: StoragePool_c4_Cache1

             Disk
    Disk     Type    Usable Size  Total Size
    -------- ------- ----------- -----------
    1.0.0    SSD         894.0GB     894.3GB
    1.0.1    SSD         894.0GB     894.3GB
    1.0.10   SSD         894.0GB     894.3GB
3 entries were displayed.

cluster4::> storage aggregate show-spare-disks -disk-type SSD
                                                                      
Original Owner: cluster4-02
 Pool0
  Spare Pool

                                                             Usable Physical
 Disk             Type   Class          RPM Checksum           Size     Size Status
 ---------------- ------ ----------- ------ -------------- -------- -------- --------
 1.0.11           SSD    solid-state      - block           894.0GB  894.3GB zeroed

Should I change the assignment of this SSD or in case of one SSD fail, indipendentment from node01 or node02, this spare will use?

I remember that in the past only one SSD for spare, in Flash Pool, is safe for all nodes in HA.

Thanks