=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2022.04.06 10:18:41 =~=~=~=~=~=~=~=~=~=~=~= login as: angelo angelo@10.40.5.54's password: MSFASP02> MSFASP02> MSFASP02> sysstat -c 60 -u -s CPU Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk ops/s in out read write read write age hit time ty util 35% 9173 77813 126424 110127 64 0 0 2 91% 86% : 100% 49% 5927 78830 65067 201773 106001 0 0 2 93% 55% F 100% 49% 4784 105574 50910 236112 197262 0 0 2 94% 100% : 100% 52% 7684 142502 72262 229714 182131 0 0 2 92% 100% : 100% 29% 5461 88516 55126 94007 81498 0 0 2 94% 100% # 100% 33% 7040 31647 96947 150070 1542 0 0 2 94% 100% B 100% 58% 5392 92380 22627 214567 243340 0 0 2 94% 100% : 100% 45% 5443 46514 55828 251827 162444 0 0 2 95% 100% : 100% 37% 5945 38309 55271 152434 148029 0 0 2 94% 100% : 100% 36% 8957 45751 110207 125490 35 0 0 2 94% 100% : 100% 36% 9401 38034 124003 125632 29 0 0 2 93% 0% - 100% 37% 9824 44957 130116 146314 0 0 0 2 94% 0% - 100% 48% 8201 37406 103334 189073 71830 0 0 0s 95% 51% F 100% 44% 5616 46004 42470 197635 128853 0 0 0s 95% 100% : 100% Wed Apr 6 10:19:17 MYT [MSFASP02:sshd_1:info]: Failed password for administrator from 10.40.9.11 port 55544ssh2 Wed Apr 6 10:19:17 MYT [MSFASP02:sshd_2:info]: Failed password for root from 10.40.9.11 port 55545ssh2 48% 5832 37444 65260 274221 147125 0 0 2 96% 100% : 100% 41% 6180 42188 84004 225936 125443 0 0 2 96% 100% : 100% 35% 6064 44299 54652 207160 31875 0 0 2 94% 55% : 100% 34% 12133 45878 194933 121573 5 0 0 2 94% 0% - 100% 35% 10906 73188 190622 114304 16 0 0 2 92% 0% - 100% CPU Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk ops/s in out read write read write age hit time ty util 35% 11748 56222 172966 114972 0 0 0 2 93% 0% - 100% 46% 6373 34685 100620 207174 91196 0 0 2 93% 48% F 100% 49% 5436 68040 75175 197688 147317 0 0 2 95% 100% : 100% 40% 4730 49202 53596 239296 148968 0 0 2 93% 100% : 100% 41% 8034 54881 107261 155182 127872 0 0 2 93% 100% : 100% 32% 7588 41350 126102 130669 3997 0 0 2 93% 71% : 100% 31% 7434 47248 134820 134253 0 0 0 2 93% 0% - 100% 30% 8081 50932 115386 114515 16 0 0 2 92% 0% - 100% 31% 8808 48855 126494 121943 5 0 0 2 93% 0% - 100% 49% 3948 31610 57132 240152 144597 0 0 2 94% 93% F 100% 35% 5117 49270 67801 180549 119126 0 0 2 94% 100% : 100% 39% 5746 49470 87035 190873 127712 0 0 2 95% 100% : 100% 34% 4948 32285 74430 161277 139757 0 0 2 95% 100% : 100% 27% 6383 44270 91928 94893 23992 0 0 2 93% 100% : 100% 33% 8032 53382 122215 145696 43 0 0 2 93% 72% : 100% 35% 9156 54727 172157 140915 101 0 0 2 94% 0% - 100% 30% 8089 34108 137485 132472 0 0 0 2 94% 0% - 100% 45% 6453 44327 94160 225552 73709 0 0 2 95% 48% F 100% 42% 6672 48963 80114 192677 126060 0 0 2 95% 100% : 100% 36% 5068 28693 78584 181123 126523 0 0 0s 95% 100% : 100% CPU Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk ops/s in out read write read write age hit time ty util 38% 5762 43717 88741 201776 139731 0 0 2 93% 100% : 100% 33% 6537 45319 95951 140749 75171 0 0 2 93% 100% : 100% 33% 8616 36932 153689 152507 43 0 0 2 95% 71% : 100% 35% 8698 62567 151693 151979 16 0 0 2 94% 0% - 100% 34% 8684 61144 149437 156675 0 0 0 2 94% 0% - 100% 41% 7207 54040 126603 174708 1518 0 0 2 94% 15% F 100% 51% 6466 39964 81144 224361 173532 0 0 2 96% 100% : 100% 47% 7290 77261 67517 201095 133873 0 0 2 95% 100% : 100% 48% 7304 59273 92661 224496 145533 0 0 2 94% 100% : 100% 43% 8334 191428 78738 97675 99080 0 0 2 93% 100% # 100% 32% 6603 37176 148885 163094 34 0 0 0s 93% 100% B 100% 62% 5985 65101 41507 289751 204915 0 0 2 93% 100% : 100% 57% 7160 141798 83914 227063 205981 0 0 2 94% 100% : 100% 36% 4380 35190 30248 187963 124866 0 0 2 92% 100% : 100% 38% 9556 64222 136696 116712 53 0 0 2 92% 69% : 100% 55% 6853 50390 88584 215881 100058 0 0 2 93% 61% F 100% 42% 6120 50944 63011 169894 143791 0 0 2 94% 100% : 100% 33% 5053 80225 40015 152999 105779 0 0 2 92% 100% : 100% 31% 4539 70151 46494 122408 113144 0 0 2 88% 100% : 100% 35% 6307 76287 102995 127763 70932 0 0 2 90% 100% : 100% CPU Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk ops/s in out read write read write age hit time ty util 53% 6773 85224 125070 209517 87315 0 0 2 90% 66% F 100% -- Summary Statistics ( 60 samples 1 secs/sample) CPU Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk ops/s in out read write read write age hit time ty util Min 27% 3948 28693 22627 94007 0 0 0 0s 88% 0% * 100% Avg 39% 7033 58468 96151 172981 82564 0 0 2 94% 71% * 100% Max 62% 12133 191428 194933 289751 243340 0 0 2 96% 100% * 100% MSFASP02> MSFASP02> MSFASP02> statit -b statit not found. Type '?' for a list of commands MSFASP02> priv set ad      diag Warning: These diagnostic commands are for use by NetApp personnel only. MSFASP02*> MSFASP02*> MSFASP02*> statit -b MSFASP02*> statit -b   e Hostname: MSFASP02 ID: 1575139470 Memory: 16800 MB NetApp Release 8.2.4P6 7-Mode: Wed Jan 11 01:07:08 PST 2017 <1O> Start time: Wed Apr 6 10:20:44 MYT 2022 CPU Statistics 22.235269 time (seconds) 100 % 70.362160 system time 316 % 2.255994 rupt time 10 % (732805 rupts x 3 usec/rupt) 68.106166 non-rupt system time 306 % 107.519984 idle time 484 % 14.983371 time in CP 67 % 100 % 1.398395 rupt time in CP 9 % (472332 rupts x 3 usec/rupt) Multiprocessor Statistics (per second) cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7 total sk switches 6952.56 6688.47 5330.14 6197.77 13168.09 20222.51 19407.86 20120.83 98088.22 hard switches 6775.99 6524.59 5193.42 6050.66 12943.36 19955.55 19130.28 19845.00 96418.85 domain switches 110.23 110.41 91.70 94.44 129.93 140.54 138.88 145.04 961.18 CP rupts 5553.52 3585.43 1359.82 3508.21 1347.99 1348.04 3191.55 1347.90 21242.47 nonCP rupts 3763.80 1702.39 657.96 1663.80 652.25 652.30 1969.62 652.30 11714.41 IPI rupts 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 grab kahuna 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 grab kahuna usec 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 suspend domain 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 suspend domain usec 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 CP rupt usec 33429.05 4005.93 1280.94 4246.05 1378.44 1226.70 15120.80 2202.81 62890.85 nonCP rupt usec 22263.82 1791.61 525.52 1903.10 560.19 524.57 9993.49 1006.73 38569.27 idle 667336.92 726767.73 768661.89 743675.91 598097.73 439150.84 442881.13 448988.18 4835560.43 kahuna 6873.13 6540.73 4975.07 5844.86 10322.61 13615.17 13420.39 13563.77 75156.01 storage 15614.88 19909.86 14184.49 19013.71 26871.36 39934.03 37439.35 38317.28 211285.10 exempt 50030.65 46819.36 37483.87 43225.79 70510.19 110017.74 103966.54 107503.31 569557.62 raid 581.33 575.17 531.32 476.72 714.14 924.88 914.13 918.50 5636.36 raid_exempt 40727.86 43944.06 36064.78 38879.04 62497.51 100072.55 93159.66 92256.86 507602.40 target 331.10 208.86 198.38 249.60 380.03 709.68 549.53 683.78 3311.05 dnscache 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 cifs 50.33 44.88 36.20 34.81 44.70 73.22 48.21 54.60 387.13 wafl_exempt 95949.37 97207.68 88668.41 93464.94 145539.37 184418.68 180451.20 186861.87 1072561.75 wafl_xcleaner 5624.80 6869.99 5783.29 5303.56 5898.47 7755.47 7238.46 9779.91 54254.08 sm_exempt 16.55 24.15 18.84 18.26 14.08 22.40 20.73 18.57 153.72 cluster 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 protocol 272.63 315.49 253.20 290.80 344.45 400.85 482.75 431.79 2792.10 nwk_exclusive 1490.34 1284.36 1029.67 1059.44 1772.09 1967.68 2085.52 2211.49 12900.72 nwk_exempt 56228.82 40649.02 36079.80 38948.03 70097.02 95369.66 87261.77 90999.12 515633.38 nwk_legacy 1031.69 1009.21 911.39 939.00 1368.86 1573.54 1631.82 1584.33 10050.02 hostOS 2146.36 2031.55 3312.53 2425.97 3588.31 2241.98 3334.16 2616.65 21697.65 ssan_exempt 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 FreeBSD CPU state Statistics (per second) user 0.00 0.04 0.00 0.00 0.00 0.00 0.00 0.00 0.04 nice 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sys 33.78 40.07 33.60 37.42 58.60 73.17 73.71 72.45 422.80 intr 0.00 0.00 0.04 0.00 0.00 0.49 0.58 0.40 1.53 idle 99.21 92.87 99.35 95.57 74.34 59.32 58.69 60.13 639.48 nonrt-pf-cnt 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.09 0.09 nonrt-pf-usec 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.90 0.90 rt-pf-cnt 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 rt-pf-usec 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 kern-pf-cnt 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 kern-pf-usec 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 19.285369 seconds with one or more CPUs active ( 87%) 15.613708 seconds with 2 or more CPUs active ( 70%) 11.953241 seconds with 3 or more CPUs active ( 54%) 8.601492 seconds with 4 or more CPUs active ( 39%) 5.878068 seconds with 5 or more CPUs active ( 26%) 3.853652 seconds with 6 or more CPUs active ( 17%) 2.430217 seconds with 7 or more CPUs active ( 11%) 3.671660 seconds with one CPU active ( 17%) 3.660467 seconds with 2 CPUs active ( 16%) 3.351749 seconds with 3 CPUs active ( 15%) 2.723423 seconds with 4 CPUs active ( 12%) 2.024416 seconds with 5 CPUs active ( 9%) 1.423435 seconds with 6 CPUs active ( 6%) 0.968415 seconds with 7 CPUs active ( 4%) 1.461801 seconds with all CPUs active ( 7%) Domain Utilization of Shared Domains (per second) 0.00 idle 552805.90 kahuna 0.00 storage 0.00 exempt 409169.28 raid 0.00 raid_exempt 0.00 target 0.00 dnscache 0.00 cifs 0.00 wafl_exempt 0.00 wafl_xcleaner 0.00 sm_exempt 0.00 cluster 0.00 protocol 417830.29 nwk_exclusive 0.00 nwk_exempt 0.00 nwk_legacy 0.00 hostOS 0.00 ssan_exempt switch domain to domain (per second) 0.00 idle 144.86 kahuna 15.83 storage 92.06 exempt 16.55 raid 0.00 raid_exempt 0.45 target 0.00 dnscache 1.08 cifs 0.99 wafl_exempt 0.00 wafl_xcleaner 0.00 sm_exempt 0.00 cluster 0.36 protocol 13.72 nwk_exclusive 324.30 nwk_exempt 350.97 nwk_legacy 0.00 hostOS 0.00 ssan_exempt Exempt Domain Suspension Stats (per second) Miscellaneous Statistics (per second) 96418.85 hard context switches 6703.40 NFS operations 0.00 CIFS operations 0.00 HTTP operations 99453.04 network KB received 224765.30 network KB transmitted 181877.36 disk KB read 76315.69 disk KB written 50040.32 NVRAM KB written 0.00 nolog KB written 26509.42 WAFL bufs given to clients 0.00 checksum cache hits ( 0%) 26410.97 no checksum - partial buffer 40.30 FCP operations 0.00 iSCSI operations WAFL Statistics (per second) 8.41 name cache hits ( 67%) 4.23 name cache misses ( 33%) 151803.92 buf hash hits ( 65%) 81591.10 buf hash misses ( 35%) 9463.66 inode cache hits ( 100%) 0.00 inode cache misses ( 0%) 36463.24 buf cache hits ( 94%) 2420.12 buf cache misses ( 6%) 1111.70 blocks read 23764.09 blocks read-ahead 5810.23 chains read-ahead 2994.88 dummy reads 16998.09 blocks speculative read-ahead 13816.47 blocks written 43.62 stripes written 159.79 blocks page flipped 0.00 blocks over-written 0.00 wafl_timer generated CP 0.00 snapshot generated CP 0.00 wafl_avail_bufs generated CP 0.00 dirty_blk_cnt generated CP 0.09 full NV-log generated CP 0.00 back-to-back CP 0.00 flush generated CP 0.00 sync generated CP 0.00 deferred back-to-back CP 0.00 low mbufs generated CP 0.00 low datavecs generated CP 0.00 nvlog replay takeover time limit CP 19711.25 non-restart messages 455.13 IOWAIT suspends 108.16 next nvlog nearly full msecs 0.00 dirty buffer susp msecs 0.00 nvlog full susp msecs 0.00 nvlh susp msecs 3098412 buffers RAID Statistics (per second) 5216.44 xors 0.00 long dispatches [0] 0.00 long consumed [0] 0.00 long consumed hipri [0] 0.00 long low priority [0] 0.00 long high priority [0] 99.98 long monitor tics [0] 0.00 long monitor clears [0] 0.00 long dispatches [1] 0.00 long consumed [1] 0.00 long consumed hipri [1] 0.00 long low priority [1] 99.98 long high priority [1] 99.98 long monitor tics [1] 0.00 long monitor clears [1] 18 max batch 15.52 blocked mode xor 1344.89 timed mode xor 4.99 fast adjustments 4.72 slow adjustments 0 avg batch start 0 avg stripe/msec 0.00 checksum dispatches 0.00 checksum consumed 43.98 tetrises written 0.00 master tetrises 0.00 slave tetrises 2628.30 stripes written 2588.14 partial stripes 40.16 full stripes 13816.83 blocks written 14695.08 blocks read 8.23 1 blocks per stripe size 1 159.61 1 blocks per stripe size 7 192.22 2 blocks per stripe size 7 189.02 3 blocks per stripe size 7 171.03 4 blocks per stripe size 7 128.98 5 blocks per stripe size 7 68.90 6 blocks per stripe size 7 23.84 7 blocks per stripe size 7 71.24 1 blocks per stripe size 14 108.21 2 blocks per stripe size 14 134.43 3 blocks per stripe size 14 178.19 4 blocks per stripe size 14 193.66 5 blocks per stripe size 14 199.28 6 blocks per stripe size 14 187.72 7 blocks per stripe size 14 179.04 8 blocks per stripe size 14 154.26 9 blocks per stripe size 14 117.70 10 blocks per stripe size 14 83.02 11 blocks per stripe size 14 49.02 12 blocks per stripe size 14 22.62 13 blocks per stripe size 14 8.10 14 blocks per stripe size 14 Network Interface Statistics (per second) iface side bytes packets multicasts errors collisions pkt drops e0a recv 8.63 0.13 0.13 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 e0b recv 28.24 0.22 0.22 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 e1a recv 24346088.91 9230.43 0.00 0.00 0.00 xmit 106034612.80 9051.52 0.04 0.00 0.00 e1b recv 18.98 0.09 0.09 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 e4a recv 25769387.54 6215.13 0.00 0.00 0.00 xmit 2965746.90 903.83 0.04 0.00 0.00 e4b recv 0.00 0.00 0.00 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 c0a recv 385.87 13.85 2.07 0.00 0.00 xmit 368.06 1.98 2.07 0.00 0.00 c0b recv 385.87 1.98 2.07 0.00 0.00 xmit 368.06 1.98 2.07 0.00 0.00 e0M recv 101.28 0.76 0.13 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 e0P recv 6.48 0.04 0.04 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 ifgrp0 recv 50115476.45 15445.55 0.22 0.00 0.00 xmit 109000359.70 9955.36 0.09 0.00 0.00 ifgrp0-406 recv 51722963.55 15760.05 0.49 0.00 0.00 xmit 121158252.68 19054.68 0.00 0.00 0.00 ifgrp0-408 recv 174.68 2.88 2.88 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 ifgrp0-1408 recv 31.66 0.49 0.49 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 ifgrp0-405 recv 328.40 2.97 1.80 0.00 0.00 xmit 354.21 1.66 0.00 0.00 0.00 Disk Statistics (per second) ut% is the percent of time the disk was busy. xfers is the number of data-transfer commands issued per second. xfers = ureads + writes + cpreads + greads + gwrites chain is the average number of 4K blocks per command. usecs is the average disk round-trip time per 4K block. disk ut% xfers ureads--chain-usecs writes--chain-usecs cpreads-chain-usecs greads--chain-usecs gwrites-chain-usecs /aggr00_root/plex0/rg0: 3b.12.10 0 1.89 0.31 1.00 5714 1.57 5.97 134 0.00 .... . 0.00 .... . 0.00 .... . 3b.12.21 1 2.07 0.31 1.00 143 1.75 5.56 392 0.00 .... . 0.00 .... . 0.00 .... . 0a.11.0 1 1.17 0.54 1.00 6417 0.40 20.33 295 0.22 8.20 732 0.00 .... . 0.00 .... . /Aggr02_data/plex0/rg0: 3b.12.0 16 30.00 0.00 .... . 13.99 61.53 127 16.01 54.83 114 0.00 .... . 0.00 .... . 0a.11.3 16 30.50 0.00 .... . 14.57 59.14 132 15.92 54.78 117 0.00 .... . 0.00 .... . 3b.12.1 96 254.82 197.61 3.91 6915 13.49 29.40 1118 43.72 8.08 1391 0.00 .... . 0.00 .... . 3b.12.5 100 257.61 201.07 3.94 5928 13.31 29.78 6545 43.23 7.89 1750 0.00 .... . 0.00 .... . 3b.12.2 95 250.28 193.78 3.98 6555 13.40 27.83 1104 43.09 8.10 1446 0.00 .... . 0.00 .... . 0a.11.5 95 252.58 197.79 3.86 6797 13.22 30.25 1116 41.56 8.62 1647 0.00 .... . 0.00 .... . 3b.12.3 96 266.07 211.42 3.78 7054 13.27 28.33 1091 41.38 7.96 1526 0.00 .... . 0.00 .... . 0a.11.6 97 255.23 198.51 3.84 7164 13.36 28.54 1101 43.36 8.47 1502 0.00 .... . 0.00 .... . 3b.12.4 96 257.61 202.91 3.96 6732 13.09 28.37 979 41.61 8.53 1433 0.00 .... . 0.00 .... . 0a.11.7 97 264.00 206.38 3.99 6739 13.45 28.43 1180 44.17 8.17 1528 0.00 .... . 0.00 .... . 0a.11.4 100 263.55 205.97 4.02 5796 13.27 28.80 6299 44.31 7.46 1932 0.00 .... . 0.00 .... . 0a.11.8 97 259.37 206.02 3.94 6950 13.18 30.06 1082 40.17 8.90 1364 0.00 .... . 0.00 .... . 3b.12.6 96 258.78 202.38 3.91 6974 13.27 27.94 1083 43.14 8.05 1640 0.00 .... . 0.00 .... . 0a.11.9 96 258.15 201.84 3.83 6807 13.13 30.33 1218 43.18 8.02 1579 0.00 .... . 0.00 .... . 3b.12.7 96 254.24 198.51 3.84 6989 13.31 30.00 1065 42.42 8.96 1267 0.00 .... . 0.00 .... . 0a.11.10 94 248.12 192.39 3.89 6723 13.27 29.20 947 42.46 8.24 1406 0.00 .... . 0.00 .... . /Aggr02_data/plex0/rg1: 3b.12.8 13 27.75 0.00 .... . 13.22 62.46 121 14.53 57.60 93 0.00 .... . 0.00 .... . 0a.11.11 13 28.11 0.00 .... . 13.58 60.86 117 14.53 57.60 90 0.00 .... . 0.00 .... . 3b.12.9 96 261.39 206.11 3.99 6472 12.77 31.33 1081 42.51 8.18 1532 0.00 .... . 0.00 .... . 0a.11.12 96 258.74 204.49 3.89 7128 12.64 30.22 1030 41.61 8.06 1536 0.00 .... . 0.00 .... . 3b.12.20 95 252.04 197.92 3.85 7058 12.24 31.29 986 41.88 8.35 1397 0.00 .... . 0.00 .... . 0a.11.13 96 256.67 202.91 3.87 7237 12.82 30.64 1017 40.93 8.70 1305 0.00 .... . 0.00 .... . 3b.12.11 94 245.74 191.76 4.07 6454 12.73 29.44 1061 41.25 8.51 1362 0.00 .... . 0.00 .... . 0a.11.14 95 258.92 204.94 3.89 6669 12.60 30.01 1101 41.38 8.26 1373 0.00 .... . 0.00 .... . 3b.12.12 96 258.24 203.41 3.88 6813 12.77 29.88 1050 42.06 8.12 1396 0.00 .... . 0.00 .... . 0a.11.15 96 257.30 203.09 3.92 6834 12.64 32.05 990 41.56 8.17 1447 0.00 .... . 0.00 .... . 3b.12.13 94 248.71 194.64 3.89 6819 12.64 30.16 1050 41.43 8.55 1254 0.00 .... . 0.00 .... . 0a.11.16 95 249.07 196.30 4.00 7003 12.10 28.01 1124 40.66 8.36 1358 0.00 .... . 0.00 .... . 3b.12.14 96 264.68 208.99 3.87 6888 12.60 30.14 1100 43.09 7.75 1386 0.00 .... . 0.00 .... . 0a.11.17 95 255.63 201.21 3.78 7066 12.68 30.10 885 41.74 7.70 1268 0.00 .... . 0.00 .... . 3b.12.15 95 258.96 204.44 3.84 6952 12.73 32.10 920 41.79 8.08 1253 0.00 .... . 0.00 .... . 0a.11.18 95 259.59 204.94 3.73 7184 12.55 31.15 914 42.10 8.50 1352 0.00 .... . 0.00 .... . /Aggr02_data/plex0/rg2: 0a.11.19 19 41.02 0.00 .... . 16.37 57.03 183 24.65 39.13 158 0.00 .... . 0.00 .... . 3b.12.16 19 41.38 0.00 .... . 16.82 55.56 181 24.56 39.04 179 0.00 .... . 0.00 .... . 0a.11.20 99 285.50 218.43 3.74 7358 15.47 27.87 1320 51.59 7.28 1622 0.00 .... . 0.00 .... . 3b.12.17 99 283.21 216.90 3.92 7172 15.92 28.06 1262 50.38 7.55 1628 0.00 .... . 0.00 .... . 0a.11.21 99 283.70 215.87 3.77 7254 15.43 27.89 1346 52.40 7.02 1715 0.00 .... . 0.00 .... . 3b.12.18 99 288.38 220.28 3.75 7284 15.56 27.87 1319 52.54 6.94 1759 0.00 .... . 0.00 .... . 0a.11.22 100 288.25 221.27 3.72 7570 15.70 26.19 1323 51.28 7.20 1764 0.00 .... . 0.00 .... . 3b.12.19 99 288.70 220.95 3.82 7278 15.52 27.39 1329 52.22 6.78 1816 0.00 .... . 0.00 .... . 0a.11.23 99 285.05 218.57 3.91 7061 15.29 28.94 1275 51.19 7.16 1768 0.00 .... . 0.00 .... . Aggregate statistics: Minimum 0 1.17 0.00 0.40 0.00 0.00 0.00 Mean 79 213.44 163.06 12.77 37.56 0.00 0.00 Maximum 100 288.70 221.27 16.82 52.54 0.00 0.00 Spares and other disks: 3a.21.3 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.2 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.1 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.0 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.5 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.10 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.4 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.8 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.7 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.9 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.11 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.13 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.6 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.12 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.14 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.15 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.17 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.18 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.16 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.19 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.21 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.20 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3d.21.22 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3a.21.23 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3b.12.23 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 3b.12.22 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0a.11.1 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0a.11.2 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . FCP Statistics (per second) 202610.01 FCP Bytes recv 299273.60 FCP Bytes sent 40.30 FCP ops iSCSI Statistics (per second) 0.00 iSCSI Bytes recv 0.00 iSCSI Bytes xmit 0.00 iSCSI ops Interrupt Statistics (per second) 11.69 int_1 7289.73 int_2 0.09 int_3 0.54 int_4 29.46 int_5 0.54 int_6 0.36 int_7 17.36 int_8 3172.71 int_9 3116.09 int_10 0.54 int_12 33.55 int_13 3287.43 int_16 0.13 Gigabit Ethernet (IRQ 17) 0.22 Gigabit Ethernet (IRQ 18) 0.76 int_19 0.04 int_20 0.00 RTC 0.00 IPI 999.85 Msec Clock 17961.10 total MSFASP02*> MSFASP02*> MSFASP02*> wafltop start MSFASP02*> wafltop start         op CPU Utilization Percent Application Total STRIPE VOL_LOG VOL_VBN_RANGE VBN_RANGE VOL_VBN VBN VOL AGGR_VBN_RANGE AGGR_VBN AGGR SERIAL XCleaner ----------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- ------- Aggr02_data:msvfasp02_datastore09_vol:other:SHARING_MSG_PREFETCH: 12287 0 0 0 0 0 0 12287 0 0 0 0 0 Aggr02_data:msvfasp02_datastore05_vol:other:SHARING_MSG_PREFETCH: 6041 0 0 0 0 0 0 6041 0 0 0 0 0 Aggr02_data:msvfasp02_datastore02_vol:other:SHARING_MSG_PREFETCH: 4967 0 0 0 0 0 0 4967 0 0 0 0 0 Aggr02_data:msvfasp02_datastore11_vol:other:SHARING_MSG_PREFETCH: 2864 0 0 0 0 0 0 2864 0 0 0 0 0 Aggr02_data:msvfasp02_datastore08_vol:other:SHARING_MSG_PREFETCH: 2505 0 0 0 0 0 0 2505 0 0 0 0 0 Aggr02_data:msvfasp02_datastore10_vol:other:SHARING_MSG_PREFETCH: 2288 0 0 0 0 0 0 2288 0 0 0 0 0 Aggr02_data:msvfasp02_datastore07_vol:other:SHARING_MSG_PREFETCH: 2102 0 0 0 0 0 0 2102 0 0 0 0 0 Aggr02_data:msvfasp02_datastore01_vol:other:SHARING_MSG_PREFETCH: 1899 0 0 0 0 0 0 1899 0 0 0 0 0 Aggr02_data:msvfasp02_datastore06_vol:other:SHARING_MSG_PREFETCH: 1348 0 0 0 0 0 0 1348 0 0 0 0 0 Aggr02_data:msvfasp02_datastore04_vol:other:SHARING_MSG_PREFETCH: 618 0 0 0 0 0 0 618 0 0 0 0 0 Aggr02_data:msvfasp02_datastore03_vol:other:SHARING_MSG_PREFETCH: 182 0 0 0 0 0 0 182 0 0 0 0 0 other:other:other:WAFL_CP_AWAIT_WRITE_: 171 0 0 0 0 0 0 0 0 0 0 171 0 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_READ: 17 17 0 0 0 0 0 0 0 0 0 0 0 other:other:other:RAIDIO_READ_DONE: 9 9 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore10_vol:nfsv3:WAFL_WRITE: 9 9 0 0 0 0 0 0 0 0 0 0 0 other:other:other:FREE_CACHE_REFILL: 7 7 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore09_vol:nfsv3:WAFL_WRITE: 5 5 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore06_vol:nfsv3:WAFL_READ: 4 4 0 0 0 0 0 0 0 0 0 0 0 other:other:other:WAFL_WRITE_DONE: 4 4 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore06_vol:other:WAFL_CPWF_WALLOC_PRE: 2 0 0 0 0 2 0 0 0 0 0 0 0 CPU Time us Application Total STRIPE VOL_LOG VOL_VBN_RANGE VBN_RANGE VOL_VBN VBN VOL AGGR_VBN_RANGE AGGR_VBN AGGR SERIAL XCleaner ----------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- ------- Aggr02_data:msvfasp02_datastore07_vol:nfsv3:WAFL_READ: 5161 5074 87 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore06_vol:nfsv3:WAFL_READ: 2471 2361 110 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore10_vol:nfsv3:WAFL_READ: 2163 2055 108 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore11_vol:nfsv3:WAFL_READ: 1961 1925 36 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore09_vol:nfsv3:WAFL_READ: 1568 1533 35 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvxesxp02_boot_vol:fcp:WAFL_READ: 1563 1563 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore08_vol:nfsv3:WAFL_READ: 1460 1408 52 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore04_vol:nfsv3:WAFL_READ: 1407 1376 31 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore02_vol:nfsv3:WAFL_READ: 1381 1374 7 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvesxt02_boot_vol:fcp:WAFL_READ: 1235 1235 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvesxp01_boot_vol:fcp:WAFL_READ: 966 915 51 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_swap_vol:nfsv3:WAFL_READ: 952 377 575 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_READ: 949 940 9 0 0 0 0 0 0 0 0 0 0 aggr00_root:vol0:spinvfs:WAFL_READ: 866 866 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvesxp06_boot_vol:fcp:WAFL_READ: 663 663 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvesxp05_boot_vol:fcp:WAFL_READ: 626 587 39 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvxesxp01_boot_vol:fcp:WAFL_READ: 624 624 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_WRITE: 619 593 18 0 0 8 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_swap_vol:nfsv3:WAFL_LOOKUP: 541 541 0 0 0 0 0 0 0 0 0 0 0 Aggr02_data:msvesxp03_boot_vol:fcp:WAFL_READ: 507 507 0 0 0 0 0 0 0 0 0 0 0 Latency Application Latency ms ----------- ---------- Aggr02_data:msvfasp02_datastore10_vol:nfsv3:WAFL_SETATTR: 677.500 Aggr02_data:msvfasp02_datastore09_vol:nfsv3:WAFL_SETATTR: 336.400 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_SETATTR: 276.000 Aggr02_data:msvfasp02_datastore07_vol:nfsv3:WAFL_READ: 145.375 Aggr02_data:msvesxp04_boot_vol:fcp:WAFL_WRITE: 111.800 Aggr02_data:msvfasp02_datastore06_vol:nfsv3:WAFL_READ: 106.888 Aggr02_data:msvfasp02_datastore11_vol:nfsv3:WAFL_READ: 97.215 Aggr02_data:msvfasp02_datastore02_vol:nfsv3:WAFL_READ: 79.761 Aggr02_data:msvfasp02_datastore04_vol:nfsv3:WAFL_READ: 64.090 Aggr02_data:msvfasp02_datastore10_vol:nfsv3:WAFL_READ: 53.666 Aggr02_data:msvfasp02_datastore08_vol:nfsv3:WAFL_READ: 44.137 Aggr02_data:msvfasp02_datastore07_vol:nfsv3:WAFL_WRITE: 31.873 Aggr02_data:msvfasp02_datastore11_vol:nfsv3:WAFL_WRITE: 27.073 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_READ: 20.302 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_WRITE: 18.505 Aggr02_data:msvfasp02_datastore09_vol:nfsv3:WAFL_READ: 16.409 Aggr02_data:msvfasp02_iso_vol:nfsv3:WAFL_WRITE: 15.176 Aggr02_data:msvesxp06_boot_vol:fcp:WAFL_WRITE: 10.368 Aggr02_data:msvfasp02_datastore03_vol:nfsv3:WAFL_WRITE: 9.631 Aggr02_data:msvfasp02_datastore09_vol:nfsv3:WAFL_WRITE: 9.297 Application System Latency ms ----------- ----------------- Aggr02_data:msvfasp02_datastore04_vol:file i/o:WAFL_WRITE: 38.333 Aggr02_data:msvfasp02_datastore11_vol:file i/o:WAFL_WRITE: 3.500 Aggr02_data:msvfasp02_datastore08_vol:file i/o:WAFL_WRITE: 1.875 Aggr02_data:msvfasp02_datastore10_vol:file i/o:WAFL_WRITE: 1.829 Aggr02_data:msvfasp02_datastore01_vol:file i/o:WAFL_WRITE: 1.800 Aggr02_data:msvfasp02_datastore02_vol:file i/o:WAFL_WRITE: 1.750 Aggr02_data:msvfasp02_datastore09_vol:file i/o:WAFL_WRITE: 1.714 Aggr02_data:msvfasp02_datastore05_vol:file i/o:WAFL_WRITE: 1.600 aggr00_root:vol0:file i/o:WAFL_CREATE: 0.500 aggr00_root:vol0:file i/o:WAFL_WRITE: 0.111 I/O utilization ---------MB Read---------- ---------MB Write--------- --------IOs Read---------- --------IOs Write--------- Application MB Total Standard ExtCache Hybrid Standard ExtCache Hybrid Standard ExtCache Hybrid Standard ExtCache Hybrid ----------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- Aggr02_data:msvfasp02_datastore09_vol:other:SHARING_MSG_PREFETCH: 3189101 3189101 0 0 0 0 0 816409787 0 0 0 0 0 Aggr02_data:msvfasp02_datastore05_vol:other:SHARING_MSG_PREFETCH: 1568000 1568000 0 0 0 0 0 401408058 0 0 0 0 0 Aggr02_data:msvfasp02_datastore02_vol:other:SHARING_MSG_PREFETCH: 1380027 1380027 0 0 0 0 0 353286921 0 0 0 0 0 Aggr02_data:msvfasp02_datastore11_vol:other:SHARING_MSG_PREFETCH: 865491 865491 0 0 0 0 0 221565905 0 0 0 0 0 Aggr02_data:msvfasp02_datastore08_vol:other:SHARING_MSG_PREFETCH: 600320 600320 0 0 0 0 0 153681855 0 0 0 0 0 Aggr02_data:msvfasp02_datastore10_vol:other:SHARING_MSG_PREFETCH: 581744 581744 0 0 0 0 0 148926573 0 0 0 0 0 Aggr02_data:msvfasp02_datastore07_vol:other:SHARING_MSG_PREFETCH: 536489 536489 0 0 0 0 0 137341010 0 0 0 0 0 Aggr02_data:msvfasp02_datastore01_vol:other:SHARING_MSG_PREFETCH: 406920 406920 0 0 0 0 0 104171657 0 0 0 0 0 Aggr02_data:msvfasp02_datastore06_vol:other:SHARING_MSG_PREFETCH: 339688 339688 0 0 0 0 0 86960156 0 0 0 0 0 Aggr02_data:msvfasp02_datastore04_vol:other:SHARING_MSG_PREFETCH: 158508 158508 0 0 0 0 0 40578013 0 0 0 0 0 Aggr02_data:msvfasp02_datastore03_vol:other:SHARING_MSG_PREFETCH: 48855 48855 0 0 0 0 0 12506988 0 0 0 0 0 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_READ: 1615 1615 0 0 0 0 0 111653 0 0 0 0 0 Aggr02_data:msvesxt02_boot_vol:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 Aggr02_data:msvesxp04_boot_vol:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 Aggr02_data:msvesxp06_boot_vol:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_vmtemplate_vol:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore11_vol:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 Aggr02_data:msvesxp08_boot_vol:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 Aggr02_data:msvfasp02_datastore04_vol:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 aggr00_root:vol0:other:WAFL_CPWF_WALLOC_PRE: 824 0 0 0 824 0 0 0 0 0 0 0 0 NVLog Utilization Application NVLog in KB/s ----------- ------------------ Aggr02_data:msvfasp02_datastore09_vol:nfsv3:WAFL_WRITE: 18218 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_WRITE: 10569 Aggr02_data:msvfasp02_datastore02_vol:nfsv3:WAFL_WRITE: 6144 Aggr02_data:msvfasp02_datastore08_vol:nfsv3:WAFL_WRITE: 5840 Aggr02_data:msvfasp02_datastore04_vol:nfsv3:WAFL_WRITE: 3051 Aggr02_data:msvfasp02_datastore10_vol:nfsv3:WAFL_WRITE: 1844 Aggr02_data:msvfasp02_datastore01_vol:nfsv3:WAFL_WRITE: 1675 Aggr02_data:msvfasp02_datastore11_vol:nfsv3:WAFL_WRITE: 1163 Aggr02_data:msvfasp02_datastore06_vol:nfsv3:WAFL_WRITE: 986 Aggr02_data:msvfasp02_datastore10_vol:file i/o:WAFL_WRITE: 387 Aggr02_data:msvfasp02_datastore07_vol:nfsv3:WAFL_WRITE: 301 Aggr02_data:msvesxp06_boot_vol:fcp:WAFL_WRITE: 126 Aggr02_data:msvesxp05_boot_vol:fcp:WAFL_WRITE: 112 Aggr02_data:msvesxp07_boot_vol:fcp:WAFL_WRITE: 70 Aggr02_data:msvesxp08_boot_vol:fcp:WAFL_WRITE: 68 Aggr02_data:msvfasp02_datastore03_vol:nfsv3:WAFL_WRITE: 68 Aggr02_data:msvxesxp02_boot_vol:fcp:WAFL_WRITE: 63 Aggr02_data:msvesxp01_boot_vol:fcp:WAFL_WRITE: 63 Aggr02_data:msvfasp02_datastore09_vol:file i/o:WAFL_WRITE: 50 Aggr02_data:msvfasp02_datastore08_vol:file i/o:WAFL_WRITE: 22 Application NVLog_b2b in KB/s ----------- ---------------------- Aggr02_data:msvfasp02_datastore10_vol:nfsv3:WAFL_WRITE: 49461 Aggr02_data:msvfasp02_datastore09_vol:nfsv3:WAFL_WRITE: 12310 Aggr02_data:msvfasp02_datastore08_vol:nfsv3:WAFL_WRITE: 4734 Aggr02_data:msvfasp02_datastore02_vol:nfsv3:WAFL_WRITE: 4235 Aggr02_data:msvfasp02_datastore05_vol:nfsv3:WAFL_WRITE: 2919 Aggr02_data:msvfasp02_datastore04_vol:nfsv3:WAFL_WRITE: 2150 Aggr02_data:msvfasp02_datastore01_vol:nfsv3:WAFL_WRITE: 1060 Aggr02_data:msvfasp02_datastore06_vol:nfsv3:WAFL_WRITE: 1037 Aggr02_data:msvfasp02_datastore07_vol:nfsv3:WAFL_WRITE: 569 Aggr02_data:msvfasp02_datastore11_vol:nfsv3:WAFL_WRITE: 491 Aggr02_data:msvesxp03_boot_vol:fcp:WAFL_WRITE: 247 Aggr02_data:msvfasp02_datastore09_vol:file i/o:WAFL_WRITE: 216 Aggr02_data:msvfasp02_datastore10_vol:file i/o:WAFL_WRITE: 163 Aggr02_data:msvfasp02_datastore05_vol:file i/o:WAFL_WRITE: 75 Aggr02_data:msvesxp02_boot_vol:fcp:WAFL_WRITE: 67 Aggr02_data:msvesxt02_boot_vol:fcp:WAFL_WRITE: 61 Aggr02_data:msvfasp02_datastore08_vol:file i/o:WAFL_WRITE: 52 Aggr02_data:msvfasp02_datastore03_vol:nfsv3:WAFL_WRITE: 51 Aggr02_data:msvfasp02_datastore02_vol:file i/o:WAFL_WRITE: 48 Aggr02_data:msvfasp02_datastore06_vol:file i/o:WAFL_WRITE: 39 MSFASP02*> MSFASP02*> MSFASP02*> MSFASP02*>