ONTAP Discussions

FlashPool performance testing ?

ousturali
14,127 Views

          hi to all ,

we had installed  a new fas 2240-4 with a flash pool with 6 ssd disks and 42X1tb sata disks  ,and i am doing performance tests with sqlio software to test the storage.

but i did not get any i/o benefits in my tests.And i knew that i succesfully created the Hybrid aggregate and using the ssd disks.

am i doing the wrong test or can someone lead me the correct way to test the storage system correctly.

thanks.

13 REPLIES 13

CCOLEMAN_
14,055 Views

The “stats” command is used to show Flash Pool performance and diagnostics counters in Data ONTAP operating in 7-Mode. The “stats” command is used in the node CLI for Data ONTAP operating in Cluster-Mode. A stats preset “stats show -p hybrid_aggr” shows interactive statistics from the command line.

Can you tell us more about the workload you're running? How long are you running sqlio?

ousturali
14,055 Views

hi coleman ,

i knew that "stats" command and i am using it , According to the " stats show -p hybrid_aggr" output there are some work on the SSD disk for write and read operations .But my main concern is as follows.

1-I firtsly created a normal aggregate with 20 disks and tested with sqlio with the given parameters ,and i had collected the output .

2-Then i had converted the aggregate to a hybird aggr ,and then added the flash pool with default settings for the write and read operations as it is described in the tr-4070.Then run the same sqlio test with the same paramaters and noticed that there is no iops gain in the tests.

that confuses my mind.

Note:i had attached the sqlio parameters as below.

Thanks.

CCOLEMAN_
14,055 Views

Couple of things,

I noticed these tests are only running for 60 seconds, if a steady amount of random I/O is occurring on the aggregate, the Flash Pool should warm in a matter of 10–15 minutes as a general estimate.

Remember that writes should be acknowledged at memory speeds if your system is properly architected. Part of that architecture is making sure you have sufficient drives in the subsystem to avoid the disk drives becoming the bottleneck for the system. Flash Pool is not about making writes get acknowledged at faster than memory speeds (there isn’t such a thing) but rather about the number of drives needed in the storage system to satisfy the workload requirements (avoid the disk from becoming the bottleneck). With Flash Pool you can use fewer spindles to address those random workloads that generally are the most susceptible to becoming disk bound.

You can tune the specific volume you're worried about with the "priority" command which requires advanced privileges (priv set advanced)  and see if that helps, but I wouldn't do this unless you have a good reason to do so.

But as far as I can tell, Flash Pool is doing it's job.

Hope this helps, let me know if you have any other questions.

ousturali
14,055 Views

Hi again,

I followed your advise ,and made the test running for 15minutes with all the rest of the parameters were the same.

Now i am sending you the sqlio performance tables .

The first  chart is with only sata disks, without flashPool created

The second one is ,with flashpool added to aggregate and run the test for 1 minute

the last one is also with flashpool and the test run for 15 minutes.

The system is test system by now,and there is no other load on the storage system .                   

accoring to those i could not see any performance gain in any of the fields ,

Only sata disks

Netapp 2240 20 Disk RAID DP SQLIO 10G NFS
1t  SATA driveDB SizeI/O SecMBs/secMin GecikmeOrt GecikmeMaks Gecikme
RandomRead10 GB323320211712571785
RandomWrite10 GB4039252210121224
SequentalRead10 GB7336458355459565
SequentalWrite10 GB428626729541102

sata+ssd with flash pool for 1 minute

5 ssd diskNetapp 2240 20 + 5ssd FlashPool Disk RAID DP SQLIO 10G NFS
1t  SATA driveDB SizeI/O SecMBs/secMin GecikmeOrt GecikmeMaks Gecikme
RandomRead10 GB37242320108160917
RandomWrite10 GB32522031124416982
SequentalRead10 GB5884367068757669
SequentalWrite10 GB3957247210331369

sata+ssd with flash pool for 15 minute

5 ssd diskNetapp 2240 20 + 5ssd FlashPool Disk RAID DP SQLIO 10G NFS
1t  SATA driveDB SizeI/O SecMBs/secMin GecikmeOrt GecikmeMaks Gecikme
RandomRead10 GB477729811685611388
RandomWrite10 GB3435214211911736
SequentalRead10 GB45732851238942424
SequentalWrite10 GB337821111211364219

danpancamo
14,055 Views

not sure what  ryanbeaty is talking about, but it also looks to me that your flash pools are not gaining you very much, and even hurting your SequentalReads and writes.

Have you run any other benchmarks like vdbench on linux/solaris?

And are you using NFS on a windows server to connect to a SQL database?   What are you using for your client server?     458MB/s is 3.6Gbit/s which isn't bad... Are you sure your not maxing our your server?

alanv
14,055 Views

Couple of things.. Flashpool accelerates writes that are random only. Also, your random write block size is too large for the algorithm to pickup. Change your block size to 16k or less and retest. do a test for at least 15mins. The IOPs should grow the longer the test runs.

use the command "stats show -p hybrid_aggr -i 1" to show the read and/or writes replaced...

CLOCATEL75
14,055 Views

Exactly, I think  anormal blocks (due to size) are discarded from flashpool acceleration. Random writes/reads often use 8K block size. It should be useful to show the CPU usage during workloads. Flash pool should use more CPU and it could explain why sequential writes/reads are worse with flashpool... not sure.

Ousturali, although it's an old post, give attention to Flash Pool policies, be careful with SQLIO and system cache (-BN to remove system cache // because i find that random writes on SATA are high).

ryanbeaty
14,055 Views

IOPS are only a small portion of what you want to look at. The same benchmark software can show little IOPS or an unreasonably high IOPS count with the right tweaking. Your main concern should be latency. I'd rather have extremely low IOPS and extremely low ms response time, than super high IOPS and just mediocre ms response time.

jnsolutions
14,056 Views

I have been using flashpool for over a year now with production workload. Spec: 2x 3220 7-mode, 8.1.3P3, 12x100GB SSD, 24x 450GB 10k, 24x 2TB 7.2k.

It seems good a keeping data hot that is required by VM OSs. But we get next to nothing out of the SSD for writes, our average write is about 80 bytes (not kb). We do 7-8k IOPS during the day. The workload is perfect for SSDs (we use to use SSD), but less than 10% of writes go to the SSDs with flashpool.

We have latency in the 100s of milliseconds, which substantially effects production workloads, and disk utilisation of 100%.

We lodged a ticket when we first purchased the product due to the low SSD write hit rate, then we were told that it would not go to SSD if there is capacity to go to SAS or SATA. Which made perfect sense, but there is certainly no IOPS capacity any more. During the sales process NetApps estimate was that it could cover 46k IOPS per controller with our setup.

Price wise it is comparable to other vendors pure SSD options. I have lodged a ticket with NetApp, I will hold my judgement till they have had a proper chance to resolve the situation. But at this stage it looks like just great marketing.

oweinmann
7,904 Views

Hi,

we also see very poor performance with Flashpool (FAS 2240 12.5TB Flashpool). Has anyone ever been able to get more performance out of it?

Best Regards,

Oliver

mital_shah
7,618 Views

Was doing some research on Flash Pool performance and came across this thread.

The NetApp Flash Pool SE Presentation on FieldPortal (Partners Only), states this for writes:

"Flash Pool™ technology does not Accelerate write operations – Data ONTAP® is already write optimized" - perhaps why you don't seem to be seeing many writes on the SSDs? 

upadhayay1122
5,751 Views

As per the WAFL fuctionality, the write operations already converted into the sequential stripe before commiting onto the raid layer, and fleshpool only omtimized the overwrites which are random not sequential means it does not optimize for the random new writes as well.

 

So here i have confusion, which data will be eligible for flashpool optimization, i.e how to define the overwrite for a block as WAFL always write to new blocks so when and how overwrite to a block happens.

 

 

Thanks in advance.

 

 

aborzenkov
5,735 Views

As it is not possible to add "me too" to followup - yes, that is what I ask myself as well. In all the buzz about flash pool nowhere is defined what "overwrite" actually is.

Public