Sanitization time?

igor · ‎2010-04-30

Hello!

Does anyone have experioence with disk sanitization? How much time would it roughly take to sanitize 4 x 500 SATA disk?

Thanx,

Igor

BrendonHiggins · ‎2010-04-30

Have never used it but a 500Gb SATA disk rebuild time is about 12 hours and sanitization over writes the disk 7 times I think, so my guess is 84 hours or 3.5 days. I believe it will do all the disks together, rather than one at a time.

Hope it helps

Bren

igor · ‎2010-04-30

According to a NetApp document I received, for a 500GB SATA disk respective times are:

- 2 hours for Zeroing,

- 4 hours for Rapid RAID Recovery, and

- 6 hours for Disk Reconstruction

So, by your calculation it would be more like... 6x7=42 hours?

Igor

BrendonHiggins · ‎2010-04-30

Sounds good to me. Sorry I was thinking the 1 Tb drives which we have.

Beer o'clock here in the UK on a holiday weekend. Later

Bren

RichardSopp · ‎2010-04-30

Disk reconstruction is not the same as disk sanitization.

Disk reconstruction is a rebuild of data or parity (depending on the disk that failed) by performing complex parity calculations, as opposed to RAPID RAID recovery which is more akin to a physical disk copy out.

Remember disk sanitization is also a low priority task so depending on system load your sanitization time may vary.

You can sanitize multiple disks at the same time so the time taken to sanitize one disk is roughly the same as the time taken to do many.

In tests I’ve done with 500GB SATA drives its about 9 hours on a 7 cycle run (-c 7).

Richard

igor · ‎2010-04-30

Do I need 7 cycles? Here's the info on disk anitize command:

disk sanitize start [-p pattern1|-r [-p pattern2|-r [-p pattern3|-r]]] [-c cycle_count] disk_li

-p pattern1 -p pattern2 -p pattern3 specifies a cycle of one to three user-defined hex byte overwrite patterns that can be applied in succession to the disks being sanitized. The default pattern is three passes, using 0x55 for the first pass, 0xaa for the second pass, and 0x3c for the third pass.

-r replaces a patterned overwrite with a random overwrite for any or all of the passes.

-c cycle_count specifies the number of times the specified overwrite patterns will be applied. The default value is one cycle. The maximum value is seven cycles.

disk_list specifies a space-separated list of the IDs of the spare disks to be sanitized.

I'd like it to be as fast as possible, as long as it gets the job done. I'm trying to see whether it will remove the disk RAID label - there's no sensitive data on it. Is the default 1 cycle enough for that?

Igor

igor · ‎2010-04-30

UGH!

Sorry to have bothered you. I missed the sign saying "spares only!"... the disks I'm trying to sanitize are from a filer running on a higher version of DataONTAP, which means they have a different disk RAID label.

netapp> sysconfig -r
Aggregate aggr0 (online, raid_dp) (block checksums)
   Plex /aggr0/plex0 (online, normal, active)
     RAID group /aggr0/plex0/rg0 (normal)

       RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks)    Phys (MB/blks)
       --------- ------ ------------- ---- ---- ---- ----- --------------    --------------
       dparity   0c.00.0 0c    0   0   SA:2   - SATA 7200 423111/866531584 423946/868242816
       parity    0c.00.1 0c    0   1   SA:2   - SATA 7200 423111/866531584 423946/868242816
       data      0c.00.2 0c    0   2   SA:2   - SATA 7200 423111/866531584 423946/868242816

Spare disks (empty)

Broken disks

RAID Disk       Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks)    Phys (MB/blks)
---------       ------ ------------- ---- ---- ---- ----- --------------    --------------
label version   0c.00.3 0c    0   3   SA:2   - SATA 7200 423111/866531584 423946/868242816
label version   0c.00.4 0c    0   4   SA:2   - SATA 7200 423111/866531584 423946/868242816
label version   0c.00.5 0c    0   5   SA:2   - SATA 7200 423111/866531584 423946/868242816
label version   0c.00.6 0c    0   6   SA:2   - SATA 7200 423111/866531584 423946/868242816
netapp>
netapp> disk sanitize start 0c.00.3 0c.00.4 0c.00.5 0c.00.6
Warning : The sanitization process involves a disk format and writing repeated patterns. This process can take a long time. Are you sure? yes
disk start: 0c.00.3 isn't a spare disk.

Being of a different RAID label, the system can't touch them (Broken disks) let alone carry out disk unfail, label makespare or somethig else that would make them accessible. Dang it!

Thanx your efforts so late on a Friday, though. 😉

Cheers!

Igor

agirard2k · ‎2010-10-18

Hi Guys,

due to the lack of information about sanitization time, I decided to post some experience here, because I had the same kind of question for my current project.

I have the "honor" to sanitize 4 systems in the next weeks and will report here my results.

1. System is a FAS6080A with 2 x 224 x 450 GB 15k FC Drives (ONTAP 7.3.1.1L1P4).

I'm sanitizing with 3 random patterns and only 1 cycle ("disk sanitize start -r -r -r <disk name>"), but can already say that each pattern takes 55 minutes, means almost 3 hours per drive.

I can also say that the filer was only able to sanitize 100 drives at a time.

All sanitized drives are marked as "sanitized" and after a new fresh initialization marked as "Not Owned".

A "disk assign" results in a "bad label" after a "no valid labels" error messages.

In order to get the drive in system again, I had to make first a "disk unfail -s <disk name>" for each drive and then zeroing them again.

That's all for the moment, I will come back soon with SATA drives on FAS3050...

EDIT1:

2. System: FAS3050 and 1TB 7,2k SATA Drives.

1 Cycle with 3 random patterns: 4 hours and 12 minutes per pattern, 12,5 hours total.

3. System: FAS3050 with 300GB 5,4k and 7,2k SATA Drives

5400rpm: 1 Cycle with 3 random patterns: 2 hours and 30 minutes per pattern, 7,5 hours total.

7200rpm: 1 Cycle with 3 random patterns: 1 hours and 40 minutes per pattern, 5 hours total.

4. System: FAS6080 with 750GB 7,2k SATA Drives

1 Cycle with 3 random patterns: 3 hours and 33 minutes per pattern, about 10,5 hours total.

To answer the original question, assuming that a 500GB SATA Drive would take half of the time of a TB drive, we would need, only for the sanitization part, about 2 Hours x 6 cycle x 3 pattern = 36 hours to sanitize it and be DoD compliant.

PS: All time information here are only about the sanitization times, as written in the sanitization log file, do not include any zeroing, unfailing, initializing times...

Hope this will help...

Cya,

Anthony.

GLENDONLOWDER · ‎2014-08-01

I wish I'd found this thread before I started my little adventure as there are some great little treasures here. I found this while I was in the middle of mine and while I wait for the proverbial paint to dry I'll add my experience. I had to find out the hard way that you can only sanitize 100 Drives at at time. I found that it had issues with specifying more that 60 disks in the disk sanitize command so I had to construct my commands accordingly.

Configuration

FAS6080

Data OnTap 7.3.6 (not on 8.2.x because of compatibility constraints...)

48 DS14 ESH2/ESH4 shelves attached with 600GB 15K FC HDD's so 672 HDD's in all.

I kicked off ~60 drives at first (thought that this was all I could do at first, later I added more to equal 100)

I used the following sanitization pattern:disk sanitize start -p 0x55 -p 0xAA -r -c 6 disk list; which means using the syntax... disk sanitize start [-p pattern1|-r [-p pattern2|-r [-p pattern3|-r]]] [-c cycle_count] disk_list I'm getting three patterns. Two of which I specified and one random with 6 passes total.

So far I've found that it's taking ~8 hours to get ~50% doing some rough math you'll find it's going to take me 16 hours per 200 drives ( 100 drives X 2 controllers). I have 672 drives so I'm guessing about 64 hours.

Check my math Vlad

672HDD's / 200HDD's per iteration = 3.36 iterations but you round up to 4 as the last iteration is likely to take a full time allotment meaning 4 iterations *16hours = 64 hours for the whole burrito...

Performance impact

It doesn't seem to matter how many drives are being done at at time, and it seems to not really tax that CPU that much on a 6080. I'm not sure that having more CPU power in any case would speed up the process as the bottle neck here is writes to the spindle (I'm pretty sure anyway). I'm decommissioning the entire array and so there is nothing else running and the CPU's are taking a little nap but there are 16 CPU's performing here in my case, so if you have a 2020 with nearline SAS don't expect the same results.

~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=

For those who care to see the sysstat... (I couldn't get the nice CPU parallelization output *>sysstat -M as that command isn't available in 7.x )

This is with 0 load on the array, so the only thing running is the disk sanitization. Remember though some of the CPU load is just to run Data OnTap.

~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=

sysstat -m -c 10 1

ANY AVG CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7

9% 2% 1% 1% 0% 0% 0% 1% 3% 6%

8% 1% 1% 1% 0% 0% 0% 1% 3% 6%

9% 1% 1% 1% 0% 0% 0% 1% 3% 6%

8% 1% 1% 1% 0% 0% 0% 1% 3% 6%

10% 2% 1% 1% 0% 0% 0% 1% 3% 6%

9% 1% 1% 1% 0% 0% 0% 1% 3% 6%

18% 3% 1% 1% 0% 0% 0% 5% 4% 11%

10% 2% 1% 1% 0% 0% 0% 3% 3% 6%

9% 1% 1% 1% 0% 0% 0% 1% 3% 6%

9% 2% 1% 1% 0% 0% 0% 1% 3% 6%

~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=

sysstat -xs -c 10 1

CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s iSCSI kB/s

in out read write read write age hit time ty util in out in out

5% 0 0 0 0 0 0 24 24 0 0 >60 100% 0% - 2% 0 0 0 0 0 0

5% 0 0 0 0 0 0 16 1084 0 0 >60 100% 5% T 6% 0 0 0 0 0 0

9% 0 0 0 0 0 0 8 0 0 0 >60 100% 0% - 0% 0 0 0 0 0 0

5% 0 0 0 0 0 0 24 32 0 0 >60 100% 0% - 2% 0 0 0 0 0 0

14% 0 0 0 0 0 0 5532 0 0 0 >60 100% 0% - 9% 0 0 0 0 0 0

4% 0 0 0 0 0 0 0 0 0 0 >60 100% 0% - 0% 0 0 0 0 0 0

5% 0 0 0 0 0 0 24 24 0 0 >60 100% 0% - 2% 0 0 0 0 0 0

4% 0 0 0 0 0 0 8 0 0 0 >60 100% 0% - 0% 0 0 0 0 0 0

5% 0 0 0 0 0 0 0 8 0 0 >60 100% 0% - 1% 0 0 0 0 0 0

5% 0 0 0 0 0 0 24 24 0 0 >60 - 0% - 2% 0 0 0 0 0 0

--

Summary Statistics ( 10 samples 1.0 secs/sample)

CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s iSCSI kB/s

in out read write read write age hit time ty util in out in out

Min

4% 0 0 0 0 0 0 0 0 0 0 >60 100% 0% * 0% 0 0 0 0 0 0

Avg

6% 0 0 0 0 0 0 566 119 0 0 >60 100% 0% * 2% 0 0 0 0 0 0

Max

14% 0 0 0 0 0 0 5532 1084 0 0 >60 100% 5% * 9% 0 0 0 0 0 0

~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=

sysstat -m -c 10 1

ANY AVG CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7

12% 2% 1% 1% 0% 0% 0% 1% 3% 9%

15% 2% 1% 1% 0% 0% 0% 6% 3% 7%

8% 1% 1% 1% 0% 0% 0% 1% 2% 6%

8% 1% 1% 1% 0% 0% 0% 1% 2% 5%

8% 1% 1% 1% 0% 0% 1% 2% 3% 5%

8% 1% 1% 1% 0% 0% 0% 1% 2% 5%

8% 1% 1% 1% 0% 0% 0% 1% 3% 5%

8% 1% 1% 1% 0% 0% 0% 1% 2% 5%

8% 1% 1% 1% 0% 0% 0% 1% 2% 6%

8% 1% 1% 1% 0% 0% 0% 1% 2% 5%

~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=

sysstat -xs -c 10 1

CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s iSCSI kB/s

in out read write read write age hit time ty util in out in out

5% 0 0 0 0 0 0 0 8 0 0 >60 100% 0% - 1% 0 0 0 0 0 0

5% 0 0 0 0 0 0 8 24 0 0 >60 100% 0% - 2% 0 0 0 0 0 0

5% 0 0 0 0 0 0 0 0 0 0 >60 100% 0% - 0% 0 0 0 0 0 0

5% 0 0 0 0 0 0 0 0 0 0 >60 - 0% - 0% 0 0 0 0 0 0

5% 0 0 0 0 0 0 8 24 0 0 >60 100% 0% - 2% 0 0 0 0 0 0

5% 0 0 0 0 0 0 0 8 0 0 >60 100% 0% - 1% 0 0 0 0 0 0

5% 0 0 0 0 0 0 12 880 0 0 >60 100% 8% Tv 6% 0 0 0 0 0 0

5% 0 0 0 0 0 0 12 36 0 0 >60 100% 0% - 3% 0 0 0 0 0 0

8% 0 0 0 0 0 0 0 0 0 0 >60 100% 0% - 0% 0 0 0 0 0 0

5% 0 0 0 0 0 0 0 0 0 0 >60 100% 0% - 0% 0 0 0 0 0 0

--

Summary Statistics ( 10 samples 1.0 secs/sample)

CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s iSCSI kB/s

in out read write read write age hit time ty util in out in out

Min

5% 0 0 0 0 0 0 0 0 0 0 >60 100% 0% * 0% 0 0 0 0 0 0

Avg

5% 0 0 0 0 0 0 4 98 0 0 >60 100% 0% * 1% 0 0 0 0 0 0

Max

8% 0 0 0 0 0 0 12 880 0 0 >60 100% 8% * 6% 0 0 0 0 0 0

~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=

Ahmed · ‎2014-10-30

Just a quick question, will be need a sanitization license to do disk sanitization on NetApp Release 7.3.1 and NetApp Release 8.1.2 7-Mode?