I'm wondering if there is someone has tried to do it, and has some solution.
We had an flash pool aggregate that contained 8 400gb SSD drives. We recently got in 800gb SSD drives. In an attempt to increase the flash pool size we did a 'disk replace' on all the SSD drives of the flashpool and replaced them with the 800gb SSD drives. We were expecting the raidgroup to be upgraded after the last disk swap, but it doesn't appear to do that.
Has anyone ever tried this with success or have some other workaround for it?
You're running into a standard operation on aggregates. When a disk is "replaced" you are really simulating a failure and a standard swap in of a spare drive. This follows all the rules of the raidgroup rebuild including rapid rebuild via copy when available.
For your situation the key thing is that a disk, once in a raidgroup, can only be replaced with another disk of the same logical size. A physical disk of the same type but with larger capacity can be used when no other spares are available, but the larger disk will only ever be formatted for and used as if it were the same size as the smaller disk. This applies to all disk types.
To get a larger flash pool, you'd have to replace the flash pool part of the aggregate. Unfortunately, that means a complete aggregate rebuild from scratch, as you cannot remove the existing SSDs from a hybrid aggregate once created. You'd need to migrate all volumes off the aggregate, then rebuild the aggregate with the 800GB SSDs to use that capacity, or just use them as 400's.
Depending on platform and DoT version you might be able to use ADP on the 800's to split them up and not waste half of the disk, though I leave confirmation of whether you can do that with an existing non-ADP set of disks to others as I'm not as up to date on ADP at that level of detail. Although with the 800's now added you are likely in a complete rebuild situation to fully use those 800's in any case.
Hope this helps
Lead Storage Engineer | Consilio LLC
NCIE SAN Clustered, Data Protection
Kudos and accepted solutions are always appreciated.