ZAPI:file-write-file intermittent failure

[mneil]

IHAC who is using file-write-file API to create /etc/exports and /etc/quotas files.  It was working well for quite some time (possibly years) but recently they have noticed intermittent failures.  I noticed many ZAPI related queries in this community so starting here as I don't know if this falls into the traditional support case bucket.  Looking for guidance on how to proceed:  this forum? a different forum? create a burt? create a support case?

Customer has done extensive testing and discovered a scenario which reproduced the problem reliably, and also a workaround (see below for details).

Thanks in advance!

 

Details from customer:

I created a script to write file to filer using file-write-file API with the same hardcoded content repeatedly.

From the tcpdump result, it shows the size of max pkg segments sent out via http got changed from 1432 to 1460 and then repeated.

And every time when the segment size is 1460, the pkg segment is lost on filer and the write request fails.

If I add “sleep 5” between each file-write-file action, I could see max pkg segment size is always 1432 and all file writes succeed. (sleep 1 is the same as not using sleep)

My questions are as below:

1. Is it expected behavior for the max segment size to change from 1432 to 1460?
2. If 1st question it’s expected, is it expected that max pkg segment size remains 1432 if sending write file request with Zapi every 5 seconds instead of continuously?
3. Is there any pkg size bug/limitation for ontapi or Zapi if the max segment size is 1460?

Here’re some console output and tcpdump result during the test.

Attached files are full logs from script console and client side tcpdump result.

Thanks!

################### file write repeatedly with no sleep between each write ##################################

info: 24/10/2014 09:30:46 GMT: wrfile /etc/quotas.bak.wuyua.1

info: 24/10/2014 09:30:46 GMT: content:

/vol/lnn63f2v1 -root=nas,root=nbu,rw,anon=60001

/vol/lnn63f2v1/l_chsvcs_oas -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_ec_storage_lnn63f2v1_test -root=nas,rw,anon=60001

…

info: 24/10/2014 09:30:47 GMT: wrfile /etc/quotas.bak.wuyua.1 succeeded.

10/24/2014 09:30:47.103853 vin105f2.ms.com.shell > np858c14n1.46130: . ack 150 win 8759 <nop,nop,timestamp 7342076 1640359681> (DF)

10/24/2014 09:30:47.108913 vin105f2.ms.com.http > np858c14n1.46131: S 105655904:105655904(0) ack 855142305 win 5840 <mss 1432,nop,nop,sackOK,nop,wscale 8>

10/24/2014 09:30:47.108922 np858c14n1.46131 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:30:47.109424 np858c14n1.46131 > vin105f2.ms.com.http: P 1:64(63) ack 1 win 12 (DF)

10/24/2014 09:30:47.109717 np858c14n1.46131 > vin105f2.ms.com.http: . 64:1496(1432) ack 1 win 12 (DF)

10/24/2014 09:30:47.110159 vin105f2.ms.com.http > np858c14n1.46131: . ack 64 win 23

10/24/2014 09:30:47.110175 np858c14n1.46131 > vin105f2.ms.com.http: . 1496:2928(1432) ack 1 win 12 (DF)

10/24/2014 09:30:47.110179 np858c14n1.46131 > vin105f2.ms.com.http: P 2928:4360(1432) ack 1 win 12 (DF)

10/24/2014 09:30:47.110480 vin105f2.ms.com.http > np858c14n1.46131: . ack 1496 win 34

10/24/2014 09:30:47.110493 np858c14n1.46131 > vin105f2.ms.com.http: . 4360:5792(1432) ack 1 win 12 (DF)

10/24/2014 09:30:47.110497 np858c14n1.46131 > vin105f2.ms.com.http: P 5792:6324(532) ack 1 win 12 (DF)

10/24/2014 09:30:47.110942 vin105f2.ms.com.http > np858c14n1.46131: . ack 2928 win 46

10/24/2014 09:30:47.110947 vin105f2.ms.com.http > np858c14n1.46131: . ack 4360 win 57

10/24/2014 09:30:47.111428 vin105f2.ms.com.http > np858c14n1.46131: . ack 5792 win 68

10/24/2014 09:30:47.111432 vin105f2.ms.com.http > np858c14n1.46131: . ack 6324 win 79

10/24/2014 09:30:47.125173 vin105f2.ms.com.http > np858c14n1.46131: P 1:125(124) ack 6324 win 79

10/24/2014 09:30:47.125200 np858c14n1.46131 > vin105f2.ms.com.http: . ack 125 win 12 (DF)

10/24/2014 09:30:47.128334 vin105f2.ms.com.http > np858c14n1.46131: P 125:349(224) ack 6324 win 79

10/24/2014 09:30:47.128369 np858c14n1.46131 > vin105f2.ms.com.http: . ack 349 win 14 (DF)

10/24/2014 09:30:47.130406 vin105f2.ms.com.http > np858c14n1.46131: F 349:349(0) ack 6324 win 79

10/24/2014 09:30:47.130512 np858c14n1.46131 > vin105f2.ms.com.http: F 6324:6324(0) ack 350 win 14 (DF)

10/24/2014 09:30:47.131242 vin105f2.ms.com.http > np858c14n1.46131: . ack 6325 win 79

10/24/2014 09:30:47.136032 np858c14n1.46132 > vin105f2.ms.com.shell: S 846418832:846418832(0) win 5840 <mss 1460,sackOK,timestamp 1640359721[|tcp]> (DF)

10/24/2014 09:30:47.142894 vin105f2.ms.com.shell > np858c14n1.46132: S 1136172468:1136172468(0) ack 846418833 win 8760 <mss 1460,nop,nop,sackOK,nop,wscale 0,[|tcp]> (DF)

10/24/2014 09:30:47.142909 np858c14n1.46132 > vin105f2.ms.com.shell: . ack 1 win 12 <nop,nop,timestamp 1640359727 7342116> (DF)

info: 24/10/2014 09:30:47 GMT: rdfile /etc/quotas.bak.wuyua.1 succeeded.

info: 24/10/2014 09:30:47 GMT: content:

/vol/lnn63f2v1 -root=nas,root=nbu,rw,anon=60001

/vol/lnn63f2v1/l_chsvcs_oas -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_ec_storage_lnn63f2v1_test -root=nas,rw,anon=60001

…

info: 24/10/2014 09:30:47 GMT: wrfile /etc/quotas.bak.wuyua.2

info: 24/10/2014 09:30:47 GMT: content:

/vol/lnn63f2v1 -root=nas,root=nbu,rw,anon=60001

/vol/lnn63f2v1/l_chsvcs_oas -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_ec_storage_lnn63f2v1_test -root=nas,rw,anon=60001

…

error: 24/10/2014 09:31:47 GMT: wrfile /etc/quotas.bak.wuyua.2 failed: 'node_wrfile: failed to write file '/etc/quotas.bak.wuyua.2.1414143047''

10/24/2014 09:30:47.714893 np858c14n1.46135 > vin105f2.ms.com.shell: F 149:149(0) ack 31 win 12 <nop,nop,timestamp 1640360300 7342687> (DF)

10/24/2014 09:30:47.718405 np858c14n1.46136 > vin105f2.ms.com.http: S 839518386:839518386(0) win 5840 <mss 1460,sackOK,timestamp 1640360303[|tcp]> (DF)

10/24/2014 09:30:47.719675 vin105f2.ms.com.http > np858c14n1.46136: S 104525957:104525957(0) ack 839518387 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 8>

10/24/2014 09:30:47.719686 np858c14n1.46136 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:30:47.720153 np858c14n1.46136 > vin105f2.ms.com.http: P 1:64(63) ack 1 win 12 (DF)

10/24/2014 09:30:47.720455 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:30:47.720904 vin105f2.ms.com.http > np858c14n1.46136: . ack 64 win 23

10/24/2014 09:30:47.720917 np858c14n1.46136 > vin105f2.ms.com.http: . 1524:2984(1460) ack 1 win 12 (DF)

10/24/2014 09:30:47.720922 np858c14n1.46136 > vin105f2.ms.com.http: P 2984:4444(1460) ack 1 win 12 (DF)

10/24/2014 09:30:47.721693 vin105f2.ms.com.1016 > np858c14n1.65534: . ack 2 win 8759 <nop,nop,timestamp 7342695 1640360300> (DF)

10/24/2014 09:30:47.721935 vin105f2.ms.com.shell > np858c14n1.46135: . ack 150 win 8759 <nop,nop,timestamp 7342694 1640360300> (DF)

10/24/2014 09:30:47.922779 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:30:48.323793 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:30:49.127858 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:30:50.736049 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:30:53.951464 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:00.382213 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:13.244404 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:17.720013 vin105f2.ms.com.http > np858c14n1.46136: . ack 64 win 23

10/24/2014 09:31:17.720026 np858c14n1.46136 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:31:38.969007 np858c14n1.46136 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:47.720307 vin105f2.ms.com.http > np858c14n1.46136: . ack 64 win 23

10/24/2014 09:31:47.720328 np858c14n1.46136 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:31:47.727101 np858c14n1.46304 > vin105f2.ms.com.shell: S 920007074:920007074(0) win 5840 <mss 1460,sackOK,timestamp 1640420321[|tcp]> (DF)

10/24/2014 09:31:47.734167 vin105f2.ms.com.shell > np858c14n1.46304: S 174929190:174929190(0) ack 920007075 win 8760 <mss 1460,nop,nop,sackOK,nop,wscale 0,[|tcp]> (DF)

10/24/2014 09:31:47.734182 np858c14n1.46304 > vin105f2.ms.com.shell: . ack 1 win 12 <nop,nop,timestamp 1640420328 7402706> (DF)

10/24/2014 09:31:47.734495 np858c14n1.46304 > vin105f2.ms.com.shell: P 1:7(6) ack 1 win 12 <nop,nop,timestamp 1640420328 7402706> (DF)

10/24/2014 09:31:47.743523 vin105f2.ms.com.1023 > np858c14n1.65534: S 586950619:586950619(0) win 8192 <mss 1460,nop,nop,sackOK,nop,wscale 0,[|tcp]> (DF)

info: 24/10/2014 09:31:47 GMT: wrfile /etc/quotas.bak.wuyua.3

info: 24/10/2014 09:31:47 GMT: content:

/vol/lnn63f2v1 -root=nas,root=nbu,rw,anon=60001

/vol/lnn63f2v1/l_chsvcs_oas -root=nas,rw,anon=60001

…

info: 24/10/2014 09:31:48 GMT: wrfile /etc/quotas.bak.wuyua.3 succeeded.

10/24/2014 09:31:47.999982 np858c14n1.46306 > vin105f2.ms.com.http: S 918542907:918542907(0) win 5840 <mss 1460,sackOK,timestamp 1640420594[|tcp]> (DF)

10/24/2014 09:31:48.003054 vin105f2.ms.com.shell > np858c14n1.46305: . ack 150 win 8759 <nop,nop,timestamp 7402975 1640420590> (DF)

10/24/2014 09:31:48.003324 vin105f2.ms.com.1022 > np858c14n1.65534: . ack 2 win 8759 <nop,nop,timestamp 7402975 1640420590> (DF)

10/24/2014 09:31:48.008042 vin105f2.ms.com.http > np858c14n1.46306: S 1044419239:1044419239(0) ack 918542908 win 5840 <mss 1432,nop,nop,sackOK,nop,wscale 8>

10/24/2014 09:31:48.008052 np858c14n1.46306 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:31:48.008537 np858c14n1.46306 > vin105f2.ms.com.http: P 1:64(63) ack 1 win 12 (DF)

10/24/2014 09:31:48.008830 np858c14n1.46306 > vin105f2.ms.com.http: . 64:1496(1432) ack 1 win 12 (DF)

10/24/2014 09:31:48.009429 vin105f2.ms.com.http > np858c14n1.46306: . ack 64 win 23

10/24/2014 09:31:48.009464 np858c14n1.46306 > vin105f2.ms.com.http: . 1496:2928(1432) ack 1 win 12 (DF)

10/24/2014 09:31:48.009469 np858c14n1.46306 > vin105f2.ms.com.http: P 2928:4360(1432) ack 1 win 12 (DF)

10/24/2014 09:31:48.009762 vin105f2.ms.com.http > np858c14n1.46306: . ack 1496 win 34

10/24/2014 09:31:48.009789 np858c14n1.46306 > vin105f2.ms.com.http: . 4360:5792(1432) ack 1 win 12 (DF)

10/24/2014 09:31:48.009796 np858c14n1.46306 > vin105f2.ms.com.http: P 5792:6324(532) ack 1 win 12 (DF)

10/24/2014 09:31:48.010398 vin105f2.ms.com.http > np858c14n1.46306: . ack 2928 win 46

10/24/2014 09:31:48.010641 vin105f2.ms.com.http > np858c14n1.46306: . ack 4360 win 57

10/24/2014 09:31:48.010884 vin105f2.ms.com.http > np858c14n1.46306: . ack 5792 win 68

10/24/2014 09:31:48.010888 vin105f2.ms.com.http > np858c14n1.46306: . ack 6324 win 79

10/24/2014 09:31:48.037208 vin105f2.ms.com.http > np858c14n1.46306: P 1:125(124) ack 6324 win 79

10/24/2014 09:31:48.037236 np858c14n1.46306 > vin105f2.ms.com.http: . ack 125 win 12 (DF)

10/24/2014 09:31:48.044277 vin105f2.ms.com.http > np858c14n1.46306: P 125:349(224) ack 6324 win 79

10/24/2014 09:31:48.044312 np858c14n1.46306 > vin105f2.ms.com.http: . ack 349 win 14 (DF)

10/24/2014 09:31:48.044519 vin105f2.ms.com.http > np858c14n1.46306: F 349:349(0) ack 6324 win 79

10/24/2014 09:31:48.044705 np858c14n1.46306 > vin105f2.ms.com.http: F 6324:6324(0) ack 350 win 14 (DF)

10/24/2014 09:31:48.045588 vin105f2.ms.com.http > np858c14n1.46306: . ack 6325 win 79

info: 24/10/2014 09:31:48 GMT: wrfile /etc/quotas.bak.wuyua.4

info: 24/10/2014 09:31:48 GMT: content:

/vol/lnn63f2v1 -root=nas,root=nbu,rw,anon=60001

/vol/lnn63f2v1/l_chsvcs_oas -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_ec_storage_lnn63f2v1_test -root=nas,rw,anon=60001

…

error: 24/10/2014 09:32:49 GMT: wrfile /etc/quotas.bak.wuyua.4 failed: 'node_wrfile: failed to write file '/etc/quotas.bak.wuyua.4.1414143108''

10/24/2014 09:31:48.603106 np858c14n1.46315 > vin105f2.ms.com.http: S 921507581:921507581(0) win 5840 <mss 1460,sackOK,timestamp 1640421197[|tcp]> (DF)

10/24/2014 09:31:48.604182 vin105f2.ms.com.http > np858c14n1.46315: S 1065930085:1065930085(0) ack 921507582 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 8>

10/24/2014 09:31:48.604193 np858c14n1.46315 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:31:48.604644 np858c14n1.46315 > vin105f2.ms.com.http: P 1:64(63) ack 1 win 12 (DF)

10/24/2014 09:31:48.604924 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:48.605392 vin105f2.ms.com.http > np858c14n1.46315: . ack 64 win 23

10/24/2014 09:31:48.605405 np858c14n1.46315 > vin105f2.ms.com.http: . 1524:2984(1460) ack 1 win 12 (DF)

10/24/2014 09:31:48.605408 np858c14n1.46315 > vin105f2.ms.com.http: P 2984:4444(1460) ack 1 win 12 (DF)

10/24/2014 09:31:48.606410 vin105f2.ms.com.1018 > np858c14n1.65534: . ack 2 win 8759 <nop,nop,timestamp 7403578 1640421193> (DF)

10/24/2014 09:31:48.606651 vin105f2.ms.com.shell > np858c14n1.46312: . ack 150 win 8759 <nop,nop,timestamp 7403578 1640421193> (DF)

10/24/2014 09:31:48.806222 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:49.208263 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:50.012325 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:51.619424 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:31:54.835082 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:32:01.265732 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:32:14.128055 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:32:17.720599 vin105f2.ms.com.http > np858c14n1.46136: . ack 64 win 23

10/24/2014 09:32:17.720611 np858c14n1.46136 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:32:18.603851 vin105f2.ms.com.http > np858c14n1.46315: . ack 64 win 23

10/24/2014 09:32:18.604744 np858c14n1.46315 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:32:39.852083 np858c14n1.46315 > vin105f2.ms.com.http: . 64:1524(1460) ack 1 win 12 (DF)

10/24/2014 09:32:47.720892 vin105f2.ms.com.http > np858c14n1.46136: . ack 64 win 23

10/24/2014 09:32:47.720905 np858c14n1.46136 > vin105f2.ms.com.http: R 839518450:839518450(0) win 0 (DF)

10/24/2014 09:32:48.604146 vin105f2.ms.com.http > np858c14n1.46315: . ack 64 win 23

10/24/2014 09:32:48.604157 np858c14n1.46315 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:32:48.612174 np858c14n1.46471 > vin105f2.ms.com.shell: S 982329833:982329833(0) win 5840 <mss 1460,sackOK,timestamp 1640481215[|tcp]> (DF)

#################################sleep 5 after each file write ###################################

info: 24/10/2014 09:44:01 GMT: wrfile /etc/quotas.bak.wuyua.1

info: 24/10/2014 09:44:01 GMT: content:

/vol/lnn63f2v1 -root=nas,root=nbu,rw,anon=60001

/vol/lnn63f2v1/l_chsvcs_oas -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_ec_storage_lnn63f2v1_test -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_it_sp2010externalstorage102 -root=nas,rw,anon=60001

…

info: 24/10/2014 09:44:01 GMT: wrfile /etc/quotas.bak.wuyua.1 succeeded.

10/24/2014 09:44:01.452292 np858c14n1.47945 > vin105f2.ms.com.http: S 1684877269:1684877269(0) win 5840 <mss 1460,sackOK,timestamp 1641154157[|tcp]> (DF)

10/24/2014 09:44:01.454800 vin105f2.ms.com.1020 > np858c14n1.65533: . ack 2 win 8759 <nop,nop,timestamp 8136410 1641154153> (DF)

10/24/2014 09:44:01.455347 vin105f2.ms.com.shell > np858c14n1.47944: . ack 150 win 8759 <nop,nop,timestamp 8136410 1641154153> (DF)

10/24/2014 09:44:01.460169 vin105f2.ms.com.http > np858c14n1.47945: S 3961966448:3961966448(0) ack 1684877270 win 5840 <mss 1432,nop,nop,sackOK,nop,wscale 8>

10/24/2014 09:44:01.460176 np858c14n1.47945 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:44:01.460645 np858c14n1.47945 > vin105f2.ms.com.http: P 1:64(63) ack 1 win 12 (DF)

10/24/2014 09:44:01.460927 np858c14n1.47945 > vin105f2.ms.com.http: . 64:1496(1432) ack 1 win 12 (DF)

10/24/2014 09:44:01.461455 vin105f2.ms.com.http > np858c14n1.47945: . ack 64 win 23

10/24/2014 09:44:01.461462 np858c14n1.47945 > vin105f2.ms.com.http: . 1496:2928(1432) ack 1 win 12 (DF)

10/24/2014 09:44:01.461465 np858c14n1.47945 > vin105f2.ms.com.http: P 2928:4360(1432) ack 1 win 12 (DF)

10/24/2014 09:44:01.461804 vin105f2.ms.com.http > np858c14n1.47945: . ack 1496 win 34

10/24/2014 09:44:01.461809 np858c14n1.47945 > vin105f2.ms.com.http: . 4360:5792(1432) ack 1 win 12 (DF)

10/24/2014 09:44:01.461811 np858c14n1.47945 > vin105f2.ms.com.http: P 5792:6324(532) ack 1 win 12 (DF)

10/24/2014 09:44:01.462343 vin105f2.ms.com.http > np858c14n1.47945: . ack 2928 win 46

10/24/2014 09:44:01.462466 vin105f2.ms.com.http > np858c14n1.47945: . ack 4360 win 57

10/24/2014 09:44:01.462682 vin105f2.ms.com.http > np858c14n1.47945: . ack 5792 win 68

10/24/2014 09:44:01.462811 vin105f2.ms.com.http > np858c14n1.47945: . ack 6324 win 79

10/24/2014 09:44:01.476223 vin105f2.ms.com.http > np858c14n1.47945: P 1:125(124) ack 6324 win 79

10/24/2014 09:44:01.476228 np858c14n1.47945 > vin105f2.ms.com.http: . ack 125 win 12 (DF)

10/24/2014 09:44:01.481191 vin105f2.ms.com.http > np858c14n1.47945: P 125:349(224) ack 6324 win 79

10/24/2014 09:44:01.481197 np858c14n1.47945 > vin105f2.ms.com.http: . ack 349 win 14 (DF)

10/24/2014 09:44:01.481345 vin105f2.ms.com.http > np858c14n1.47945: F 349:349(0) ack 6324 win 79

10/24/2014 09:44:01.481552 np858c14n1.47945 > vin105f2.ms.com.http: F 6324:6324(0) ack 350 win 14 (DF)

10/24/2014 09:44:01.482286 vin105f2.ms.com.http > np858c14n1.47945: . ack 6325 win 79

info: 24/10/2014 09:44:06 GMT: sleep 5 seconds

info: 24/10/2014 09:44:06 GMT: wrfile /etc/quotas.bak.wuyua.2

info: 24/10/2014 09:44:06 GMT: content:

/vol/lnn63f2v1 -root=nas,root=nbu,rw,anon=60001

/vol/lnn63f2v1/l_chsvcs_oas -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_ec_storage_lnn63f2v1_test -root=nas,rw,anon=60001

/vol/lnn63f2v1/l_it_sp2010externalstorage102 -root=nas,rw,anon=60001

…

info: 24/10/2014 09:44:07 GMT: wrfile /etc/quotas.bak.wuyua.2 succeeded.

10/24/2014 09:44:07.042132 np858c14n1.47994 > vin105f2.ms.com.http: S 1698853103:1698853103(0) win 5840 <mss 1460,sackOK,timestamp 1641159747[|tcp]> (DF)

10/24/2014 09:44:07.044481 vin105f2.ms.com.1016 > np858c14n1.65533: . ack 2 win 8759 <nop,nop,timestamp 8142000 1641159743> (DF)

10/24/2014 09:44:07.044528 vin105f2.ms.com.shell > np858c14n1.47993: . ack 150 win 8759 <nop,nop,timestamp 8142000 1641159743> (DF)

10/24/2014 09:44:07.050614 vin105f2.ms.com.http > np858c14n1.47994: S 4037633397:4037633397(0) ack 1698853104 win 5840 <mss 1432,nop,nop,sackOK,nop,wscale 8>

10/24/2014 09:44:07.050623 np858c14n1.47994 > vin105f2.ms.com.http: . ack 1 win 12 (DF)

10/24/2014 09:44:07.051082 np858c14n1.47994 > vin105f2.ms.com.http: P 1:64(63) ack 1 win 12 (DF)

10/24/2014 09:44:07.051375 np858c14n1.47994 > vin105f2.ms.com.http: . 64:1496(1432) ack 1 win 12 (DF)

10/24/2014 09:44:07.051977 vin105f2.ms.com.http > np858c14n1.47994: . ack 64 win 23

10/24/2014 09:44:07.051988 np858c14n1.47994 > vin105f2.ms.com.http: . 1496:2928(1432) ack 1 win 12 (DF)

10/24/2014 09:44:07.051991 np858c14n1.47994 > vin105f2.ms.com.http: P 2928:4360(1432) ack 1 win 12 (DF)

10/24/2014 09:44:07.052407 vin105f2.ms.com.http > np858c14n1.47994: . ack 1496 win 34

10/24/2014 09:44:07.052414 np858c14n1.47994 > vin105f2.ms.com.http: . 4360:5792(1432) ack 1 win 12 (DF)

10/24/2014 09:44:07.052417 np858c14n1.47994 > vin105f2.ms.com.http: P 5792:6324(532) ack 1 win 12 (DF)

10/24/2014 09:44:07.053013 vin105f2.ms.com.http > np858c14n1.47994: . ack 2928 win 46

10/24/2014 09:44:07.053133 vin105f2.ms.com.http > np858c14n1.47994: . ack 4360 win 57

10/24/2014 09:44:07.053439 vin105f2.ms.com.http > np858c14n1.47994: . ack 5792 win 68

10/24/2014 09:44:07.053565 vin105f2.ms.com.http > np858c14n1.47994: . ack 6324 win 79

10/24/2014 09:44:07.066947 vin105f2.ms.com.http > np858c14n1.47994: P 1:125(124) ack 6324 win 79

10/24/2014 09:44:07.066983 np858c14n1.47994 > vin105f2.ms.com.http: . ack 125 win 12 (DF)

10/24/2014 09:44:07.071701 vin105f2.ms.com.http > np858c14n1.47994: P 125:349(224) ack 6324 win 79

10/24/2014 09:44:07.071709 np858c14n1.47994 > vin105f2.ms.com.http: . ack 349 win 14 (DF)

10/24/2014 09:44:07.071838 vin105f2.ms.com.http > np858c14n1.47994: F 349:349(0) ack 6324 win 79

10/24/2014 09:44:07.072066 np858c14n1.47994 > vin105f2.ms.com.http: F 6324:6324(0) ack 350 win 14 (DF)

10/24/2014 09:44:07.072961 vin105f2.ms.com.http > np858c14n1.47994: . ack 6325 win 79