Curious if anybody has seen this before and if it's something to be concerned with. Have a V3140 that is currently seeing a lot of H type CPs which are defined as a CP caused by high water mark; the amount of modified data in the filer's memory cache is high enough that it is ideal to start a CP to force it out to disk. The H CP is caused when the amount of modified data in the filer's cache exceeds a threshold called the high-water mark. It differs from the F CP, which is caused by consumption of the NVRAM. For example, a file delete operation requires almost no room in the NVRAM (the file handle of the file and the containing directory is logged) but may involve changing many files: the directory changes when we remove the entry; the inode file changes as we free the inode; metadata changes as we free the blocks in the deleted file. After performing a lot of these operations, the filer could easily hit the modified-data high water mark before the NVRAM is filled and need a CP from log full.
This is the first time I've encountered this many back to back. I'm going to attempt to see if they correlate in time with some performance issues being seen on several servers. Here's a a chunk of the sysstat from during that time with many more H CPs occuring after as well:
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s iSCSI kB/s
in out read write read write age hit time ty util in out in out
31% 335 1 0 1170 727 9158 34752 57972 0 0 16s 76% 94% : 80% 830 0 50702 2881 0 0
36% 44 0 0 870 346 258 50030 60360 0 0 2s 74% 94% Hs 98% 807 15 13972 4015 101 0
27% 94 4 0 944 947 182 31568 78592 0 0 2s 96% 100% :f 99% 837 6 14033 2052 83 4
42% 118 2 0 1378 1414 229 43444 12700 0 0 18s 63% 54% Hn 64% 1249 6 45895 3161 77 0
35% 88 1 0 993 671 147 49008 97612 0 0 2s 97% 100% :s 96% 901 0 18266 4321 0 0
20% 44 1 0 1073 562 83 20596 33596 0 0 2s 90% 100% :f 74% 1019 6 39317 5925 25 0
39% 19 0 0 1182 398 42 46708 69628 0 0 2s 77% 84% Hf 85% 1152 8 31939 5424 82 0
24% 89 0 0 1030 551 284 22400 47772 0 0 20s 91% 100% :f 100% 930 7 42178 3229 42 0
35% 93 2 0 937 525 134 50512 42060 0 0 18s 70% 99% Hn 97% 827 11 40673 3862 113 0
33% 79 17 0 970 516 381 43872 79988 0 0 2s 95% 100% :f 98% 862 8 15997 3931 56 0
22% 67 1 0 1170 467 251 40884 58144 0 0 2s 84% 100% :f 97% 1089 9 16591 6260 140 0
43% 197 54 0 935 570 4744 57332 30876 0 0 2s 63% 61% Hs 78% 677 3 36088 2617 98 0
25% 73 0 0 782 461 55 38608 62212 0 0 2s 93% 100% :f 87% 704 1 24660 2426 5 0
15% 21 8 0 904 196 31 15116 39432 0 0 20s 91% 100% :f 63% 870 2 35131 3919 4 4
41% 65 15 0 923 210 185 55140 58780 0 0 18s 70% 81% Hf 84% 839 1 32961 4134 5 0
27% 17 4 0 896 90 174 28812 65048 0 0 18s 92% 100% :f 78% 863 9 18964 4103 35 0
16% 24 0 0 827 190 189 20816 30108 0 0 2s 82% 100% :v 62% 793 7 25824 4366 76 0
