I'm having performance issues with Snapmirror XDP relationships between two clusters (primary 4-node 3250, secondary 2-node 3220, all running cDOT 8.2P5).
The replication LIFs on both sides are pure 10G (private replication VLAN), flat network (no routing), using jumbo frames (however also tested without jumbo and problem persist).
The vaulting works in general, but average throughput for a single node/relationship never goes beyond 1Gbit/s - most of the time it is even much slower (300-500MBit/s or less).
I verified that neither the source node(s) nor the destination node(s) are CPU or disk bound during the transfers (at least not all of the source nodes).
I also verfied the SnapMirror traffic is definitely going through the 10g interfaces.
There is no compressed volume involved, only dedupe (on source).
The dedupe schedules are not within the same time window.
Also checked the physical network interface counters on switch and netapps, no errors / drops, clean.
However, the replication is SLOW, no matter what i try.
Customer impression is that it got even slower over time, e.g. throughput of a source node was ~1GBit/s when relationship(s) were initialized (not as high as expected), and dropped to ~ 500MBit/s after some months of operation / regular updates...
Meanwhile the daily update (of all relationships) sums up to ~ 1,4 TB per day, and it takes almost the whole night to finish 😞
So the question is: how to tune that?
Is anyone having similar issues regarding Snapmirror XDP throughput over 10Gbit Ethernet?
Are there any configurable parameters (network compression? TCP win size? TCP delayed ack's? Anything I don't think of?) on source/destination side?