Subscribe

Seeing a lot of CP types of H

Curious if anybody has seen this before and if it's something to be concerned with. Have a V3140 that is currently seeing a lot of H type CPs which are defined as a CP caused by high water mark; the amount of modified data in the filer's memory cache is high enough that it is ideal to start a CP to force it out to disk. The H CP is caused when the amount of modified data in the filer's cache exceeds a threshold called the high-water mark. It differs from the F CP, which is caused by consumption of the NVRAM. For example, a file delete operation requires almost no room in the NVRAM (the file handle of the file and the containing directory is logged) but may involve changing many files: the directory changes when we remove the entry; the inode file changes as we free the inode; metadata changes as we free the blocks in the deleted file. After performing a lot of these operations, the filer could easily hit the modified-data high water mark before the NVRAM is filled and need a CP from log full.

This is the first time I've encountered this many back to back. I'm going to attempt to see if they correlate in time with some performance issues being seen on several servers. Here's a a chunk of the sysstat from during that time with many more H CPs occuring after as well:

CPU   NFS  CIFS  HTTP   Total    Net kB/s   Disk kB/s     Tape kB/s Cache Cache  CP   CP Disk    FCP iSCSI   FCP  kB/s iSCSI  kB/s
                                  in   out   read  write  read write   age   hit time  ty util                 in   out    in   out
31%   335     1     0    1170   727  9158  34752  57972     0     0    16s  76%  94%  :   80%    830     0 50702  2881     0     0
36%    44     0     0     870   346   258  50030  60360     0     0     2s  74%  94%  Hs  98%    807    15 13972  4015   101     0
27%    94     4     0     944   947   182  31568  78592     0     0     2s  96% 100%  :f  99%    837     6 14033  2052    83     4
42%   118     2     0    1378  1414   229  43444  12700     0     0    18s  63%  54%  Hn  64%   1249     6 45895  3161    77     0
35%    88     1     0     993   671   147  49008  97612     0     0     2s  97% 100%  :s  96%    901     0 18266  4321     0     0
20%    44     1     0    1073   562    83  20596  33596     0     0     2s  90% 100%  :f  74%   1019     6 39317  5925    25     0
39%    19     0     0    1182   398    42  46708  69628     0     0     2s  77%  84%  Hf  85%   1152     8 31939  5424    82     0
24%    89     0     0    1030   551   284  22400  47772     0     0    20s  91% 100%  :f 100%    930     7 42178  3229    42     0
35%    93     2     0     937   525   134  50512  42060     0     0    18s  70%  99%  Hn  97%    827    11 40673  3862   113     0
33%    79    17     0     970   516   381  43872  79988     0     0     2s  95% 100%  :f  98%    862     8 15997  3931    56     0
22%    67     1     0    1170   467   251  40884  58144     0     0     2s  84% 100%  :f  97%   1089     9 16591  6260   140     0
43%   197    54     0     935   570  4744  57332  30876     0     0     2s  63%  61%  Hs  78%    677     3 36088  2617    98     0
25%    73     0     0     782   461    55  38608  62212     0     0     2s  93% 100%  :f  87%    704     1 24660  2426     5     0
15%    21     8     0     904   196    31  15116  39432     0     0    20s  91% 100%  :f  63%    870     2 35131  3919     4     4
41%    65    15     0     923   210   185  55140  58780     0     0    18s  70%  81%  Hf  84%    839     1 32961  4134     5     0
27%    17     4     0     896    90   174  28812  65048     0     0    18s  92% 100%  :f  78%    863     9 18964  4103    35     0
16%    24     0     0     827   190   189  20816  30108     0     0     2s  82% 100%  :v  62%    793     7 25824  4366    76     0

Re: Seeing a lot of CP types of H

Back to back CP can be identified by B/b. H is because of higher watermark reached for dirty blocks.

Not all writes and written to NVRAM, centain workloads  examples like snapmirror destination wites are not logged in NVRAM so it will not cause F (NVRAM log full)  type CPs

Re: Seeing a lot of CP types of H

Chris,

Hello.

H type CPs are completely normal and not an indication of any type of problem or the potential for any type of problem in the future. All H means here is "the platform and OS version that you are running has decided to take a CP to ensure memory is freed for further operations". That is a highly summarized version of the internals.

Otherwise, what Mr. Baijulal wrote is correct even if not completely relevant to your question. There are indeed some operations that are not logged in NVRAM, but none of those are from a client perspective or in any way impact the stability of a write (regardless of source) to the underlying disks.

Thanks,

Jason Ledbetter

NGS Staff Engineer, Performance