reallocate TR?

[danpancamo]

After fighting with our DBAs for months now about IO performance, we finally narrowed down a growing performance issue to disk fragmentation.   We accidentally discovered that a copy of a database was 2x faster to backup than the original copy which lead us to the conclusion that we did actually have an issue with IO. 

We ran a reallocate measure which resulted in a threshold of 3.   I'm not sure exactly how this is calculated, but obviously the result is misleading. The results should have been 10 IMHO.

We ran reallocate start -f -p on the volume and it immediately reduced the backup time in half.   Disk util, disk caching, latency all were significantly better after the reallocate completed.

It appears that a the Sybase full reindex basically tries to optimize by re-writing data...   This process somehow causes the disk fragmentation.

I've only been able to find limited information on the reallocate command, however with this significant performance increase, there should be a whitepaper on the subject that includes the effects of reallocate on Cache, PAM, Dedup, Vmware, Databases, Exchange, etc...

Is there a TR document in the works on reallocate?  If not, someone should start one.

[radek_kubka]

It's available for NetApp partners via Field Portal:

https://fieldportal.netapp.com/Core/DownloadDoc.aspx?documentID=76389&contentID=68936

If you are an end-user, you have to work with your NetApp partner or rep to get hold of it.

[danpancamo]

We still run reallocation on large Database/Vmware/VDI volumes to help with "fragmentation"...

I hear that a new option in Ontap 8.1.2 called Read-Ahead in WAFL and just-in-time CSC (continuous segment cleaning) that may help with fragmentation issues...  

All I'm able to find on the subject is the Netapp Patent information and some engineers that worked on the project...

Can anyone shed some light on on CSC?

[merick]

CSC has been renamed to FSR (Free Space Reallocate). This setting is an always on process that will optimize free space in an aggregate to allow WAFL to keep writing fast and efficiently even as the filesystem ages. This process is very similar to what reallocate -A does, only it does it in real-time by removing existing blocks in a soon to be written location (the allocation area) in order to allow our write allocator to lay the data down in a more efficient method. The existing data blocks are then written to disk alongside user data in a future CP operation.

If you want to enable this on an existing aggregate, please check with NetApp support first or your local SE to make sure you have the necessary headroom. If your existing on disk structure is not optimal there are performance considerations as the FSR process tries to optimize the existing data.

Thanks,

Erick

[korso]

Free space reallocation of aggregates

Starting in Data ONTAP 8.1.1, you can enable free space reallocation on aggregates to improve write performance. Free space reallocation improves write performance by optimizing the free space within an aggregate.

Free space reallocation works best on workloads that perform a mixture of small random overwrites and sequential or random reads. You can expect additional CPU utilization when you enable free space reallocation. You should not enable free space reallocation if your storage system has sustained, high CPU utilization.

Note: You can use the statistics show-periodic command to monitor CPU utilization.

For best results, you should enable free space reallocation when you create a new aggregate. If you enable free space reallocation on an existing aggregate, there might be a period where Data ONTAP performs additional work to optimize free space. This additional work can temporarily impact system performance.

Always check with NetApp Support before enable this feature...

[vims]

This is just right on time. Thanks. Is there an article or document how to translate measure log file?

Vic

[merick]

Although 8.1.1 was the first release to support this capability it is not recommended to enable FSR on releases earlier then 8.1.1P1.

[vladimirzhigulin]

Having "free_space_realloc on" for an aggregate helps me not running  "reallocate start -f /vol/foobar" or do I still need to run "reallocate start -f .. " every time I add more disks to the aggregate?

Vladimir

[merick]

No, unfortunately. Volume level reallocate will "level" the volume across all new disks, and would still be recommended under certain circumstances. What FSR and aggregate level reallocate do is optimize the white space in the aggregate to keep writes snappy.

[vladimirzhigulin]

That's a pity, I find it being a tedious task to run "reallocate start -f .."  on a big aggregate with a lot of volumes.

Vladimir

[vims]

reallocate schedule ......blah blah blah should work as a charm, just set it and forget it.  that is what i'm doing.

I personally prefer to run it on volume

[vladimirzhigulin]

Just curious, how many volumes you've got per an aggr in average?

Usually "reallocate" brings some CPU load as well and my filers are already busy enough serving user's requests ..

[vims]

not that many ~40-50, depends on size