Network Storage Protocols Discussions

NFS performance on 10Gb link

BLACKRABBIT

Hello all!

I have an interesting task: write to array a dump of network traffic from 10Gbit/s channel.

Topology is quiet simple: network tap, capture server and NetApp array.

NetApp array directly connected to the server via another 10Gbps port. Also I am using NFSv3.

Now I tested this solution with one FAS6250 in cluster mode together with 20HDD of SAS drives or 20HDD of SATA drives.

Performance is the same (SATA/SAS) - near 450MB per second. This is 2 times slower than I expected... 😃

We planned to use HA pair of FAS8040 in the future and expected 3000MB speed (2x10Gb links).

Now we have a lot of questions. Is it possible? May be we missed some configuration? Could you help me with performance tuning or it is impossible?

On the capture server (NFS client) side I have mount options:

type nfs (rw,bg,hard,intr,rsize=1048576,wsize=1048576,vers=3,proto=tcp,timeo=600,retrans=2)

Writing has 32 threads inside.

Also I tuned the sysctl.conf parameters:

net.core.rmem_default=524288

net.core.wmem_default=524288

net.core.rmem_max=16777216

net.core.wmem_max=16777216

net.ipv4.tcp_rmem=4096 524288 16777216

net.ipv4.tcp_wmem=4096 524288 16777216

net.ipv4.ipfrag_high_thresh=524288

net.ipv4.ipfrag_low_thresh=393216

net.ipv4.tcp_timestamps=0

net.ipv4.tcp_window_scaling=1

net.core.optmem_max=524287

net.core.netdev_max_backlog=2500

sunrpc.tcp_slot_table_entries=128

sunrpc.udp_slot_table_entries=128

I will wait your feedback.

Thank you in advance!

5 REPLIES 5

shane_bradley

We'd really need to see a perfstat or at the very least a statit

BLACKRABBIT

Of course! This is perfstat of the best result that we have: [link deleted]

Write speed is near 900MB via two 10Gb links. Aggregate -max-write-alloc-blocks was set to 256K.

shane_bradley

So this is 100% writes? IF so i wouldnt expect an 6250 to have much more in it.

We would get about ~1.3GB/s max write onto our FAS6280's the best i've seen was a FAS80x0 with SSD doing about 3.5GB/s (but that was disk bound on 20 data spindles.)

BLACKRABBIT

Thank you for the answer!

Yes. My result for 100% write.

Your results only for one node? Can I expect double result for the HA-pair?

shane_bradley

Yeah thats one node.

Announcements
NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner
Public