Ideally gather a perfstat and send to NetApp support, but for a quick check in the meantime...
During the period of high latency, run a sysstat -x 1 on the console. Firstly look at the 'CP ty' column, watch out for any 'B' or (worse) 'b'. This suggests your NVRAM is not big enough, or you're not flushing data to disk fast enough (busy disks). Also for a rough disk utilisation figure, check the 'Disk Util' column to see how busy the busiest disk on the system is.
If the problem is NVRAM being full ('B' or 'b') you would normally expect ALL write workloads on the filer to be affected, not just those on the affected aggr.
To see how busy the disks are, you can use statit.