ADP still limited to internal disks on FAS26XX with Ontap 9.2?

[nhwanderer]

Just trying to confirm that ADP is still only supported on the internal disks (those in the the shelf with the nodes) on the FAS26XX series running Ontap 9.2? I can't track it down in the docs.

Thanks!

[AlexDawson]

I can confirm that this is the case - up to 24 spinning drives only, and only in the internal shelves, and it is ADPv1 - RD (Root/Data), not ADPv2 - RDD (Root/Data/Data - that is SSD only).

Unfortunately I can't find a public resource that says as much. 

Hope this helps!

[aborzenkov]

---

@AlexDawson wrote:

only in the internal shelves

---

ONTAP 9.x release notes explicitly say: "Beginning with ONTAP 9.0, root-data partitioning is supported by the FAS2600 series on hard disk drives (HDDs) in external disk shelves." If what you say is true, I expect release notes to be fixed.

[AlexDawson]

I can confirm that the Sept 2017 9.2 release notes state that ADP on external shelves for the 2600 from 9.0 - interestingly this is not in the 9.0 release notes. I was earlier relying on an internal doc on ADP released July 2017.

Clearly only one is correct - I will find out and update.

[aborzenkov]

---

@AlexDawson wrote:

I was earlier relying on an internal doc on ADP released July 2017.

---

Do you mean SE presentation? Indeed, it says exactly opposite 🙂 Would be nice to align them.

---

@AlexDawson wrote:

Clearly only one is correct - I will find out and update.

---

Much appreciated!

[AlexDawson]

I can't confirm or deny if that document exists 😉 But yes, I've contacted the documentation and TME teams and have asked for them to fight it out and let me know which is correct.

[nhwanderer]

Thank you all! I got our systems in an am playing with them before we put them into production. I'll try to reinitialize one with ADP on the external shelf. 

[AlexDawson · Accepted solution]

Ok - I have clarified the situation with our TME team - Only the FAS2650 supports ADPv1 on an external shelf of NL-SAS drives if and only if the internal slots are empty. Additional shelves added to a system with internal drives are not partitioned.

The NL-SAS drives that we have available are all large form factor (LFF), while the internal shelf on a FAS2650 is small form factor (SFF), and we do not have a FAS2240-4/2554 type of system with controllers in a 24-slot LFF shelf in our most recent generation of hardware, therefore there may be a desire under some situations to deploy one with only "external" drives, and it is of course desirable to have them partitioned for the root aggregates.

The FAS2620 does not have this behavior, as it is a LFF 12-slot enclosure, but we also have SAS and SSD SFF drives in a LFF carrier to fit in it, so the internal slots can be used for the root partition under any of these configs.

Hope this helps!

[andris]

There's a non-public bug Feature Request that allows ADP to include external shelves on FAS26xx, starting with ONTAP 9.1.

If you're running ONTAP 9.2, the new Option 9 (ADP) boot menu option is the cat's meow.

[nhwanderer]

Thanks for all the information! We have a FAS2620, so it sounds like I'll have to stick with the internal shelf only ADP. That's fine with me 🙂

[aborzenkov]

---

@andris wrote:

There's a non-public bug Feature Request that allows ADP to include external shelves on FAS26xx, starting with ONTAP 9.1.

If you're running ONTAP 9.2, the new Option 9 (ADP) boot menu option is the cat's meow.

---

Oh! Now it became even more confusing. 😞 Do you refer to the same feature as Alex (ADP on external shelves withoout internal disks) or you mean that it is possible to force ADP on external disks even with internal disks present?

[AlexDawson]

Option 9 provides additional options detailed in this KB article.

The behavior I am referring to is the automatic partitioning performed when ONTAP 9.x tries booting on a system with only unpartitioned, un-owned drives, which is the equivalent of option 9b in the new sub-menu.

[aborzenkov]

---

@AlexDawson wrote:

I've contacted the documentation and TME teams and have asked for them to fight it out and let me know which is correct.

---

Any news? Topic came up again and now it appears ADP on external shelves does not work. It would be quite helpful if someone clarified it once and for all 🙂

How to create Root aggregate across external shelf during initialization

[andris]

To recap:

1. Entry-level FAS systems (with HDDs) will only perform ADP on the internal/embedded drives.
   • AFF systems will use ADP on up to 48 SSDs across multiple shelves. However, for AFF-A220/A220, you will typically see ADP on the 24 internal SSDs only because NetApp Mfg. initializes the system without any add-on storage.
2. 12-drive entry-level FAS systems (FAS2520, 2620, 2720) with HDD sizes that require RAID-TEC need to be explicitly set RAID-TEC to "false" for internal drive ADP (i.e. LOADER> setenv bootarg.raid.allow-raid-tec? false) to ensure that ADP will use 3d+2p+1s P1 partitions for each node. 
   Important: Once you have initialized the system, set this bootarg back to true so that any additional large-HDD RG's use TEC for optimal protection.
3. If the entry-level FAS system does not contain any internal drives, ADP will use external storage.

[aborzenkov]

---

@andris wrote:

To recap:

1. Entry-level FAS systems (with HDDs) will only perform ADP on the internal/embedded drives

What about entry level systems without internal disks? Will they use full disks for root during initialization?

Oops, sorry, missed 3.

[nhwanderer]

Here is a real world example.

I had to revert from 9.2 to 9.1p8 to be officially supported by VEEAM, and since the systems are new I took the chance to reinit just in case I fouled up the revert (there's no data yet).

Anyway, by default (standard boot menu 4 initialize), with the external shelf attached on my FAS2620 (internal is 12 4tb drives, external is 8 4tb drives and 4 SSDs) ADP did spread accross the spinning disks on both shelves:

redacted1::> version
NetApp Release 9.1P8: Wed Aug 30 13:33:41 UTC 2017

redacted1::> aggr status -r aggr0_redacted1_1_0

Owner Node:redacted1_1
Aggregate: aggr0_redacted1_1_0 (online, raid_dp) (block checksums)
Plex: /aggr0_redacted1_1_0/plex0 (online, normal, active, pool0)
RAID Group /aggr0_redacted1_1_0/plex0/rg0 (normal, block checksums)
Usable Physical
Position Disk Pool Type RPM Size Size Status
-------- --------------------------- ---- ----- ------ -------- -------- ----------
shared 1.0.1 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.3 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.5 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.7 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.9 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.11 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.1.1 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.1.3 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.1.5 0 FSAS 7200 61.58GB 3.64TB (normal)
9 entries were displayed.

redacted1::> aggr status -r aggr0_redacted1_2_0

Owner Node: redacted1_2
Aggregate: aggr0_lredacted1_2_0 (online, raid_dp) (block checksums)
Plex: /aggr0_redacted1_2_0/plex0 (online, normal, active, pool0)
RAID Group /aggr0_redacted1_2_0/plex0/rg0 (normal, block checksums)
Usable Physical
Position Disk Pool Type RPM Size Size Status
-------- --------------------------- ---- ----- ------ -------- -------- ----------
shared 1.0.0 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.2 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.4 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.6 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.8 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.0.10 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.1.0 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.1.2 0 FSAS 7200 61.58GB 3.64TB (normal)
shared 1.1.4 0 FSAS 7200 61.58GB 3.64TB (normal)
9 entries were displayed.

[nhwanderer]

An update to my last post. It seems when the FAS2620 did ADP accross our two shelves of NL-SAS, no root-data spares were created. In any case, I think I'm just going to go back to a single internal shelf of ADP, as I lose a little space, but on this system is feels a bit safer.