Subscribe
Accepted Solution

FAS3270 8.1.3 7 mode High CPU

HA cluster

one of them has high PCU for most of the day

low disk utilization

only 1 volume

no snapmirror running

 

Can you help me diagnose this?

 

sysstats:

 

CPU    NFS   CIFS   HTTP   Total     Net   kB/s    Disk   kB/s    Tape   kB/s  Cache  Cache    CP  CP  Disk   OTHER    FCP  iSCSI     FCP   kB/s   iSCSI   kB/s
                                       in    out    read  write    read  write    age    hit  time  ty  util                            in    out      in    out
 97%   4174      0      0    4175   37510  17281   42470  65257       0      0     0s    89%   72%  Ff   30%       1      0      0       0      0       0      0
 96%   9616      0      0    9617   43921  29773   40534  73538       0      0     0s    87%  100%  :f   29%       1      0      0       0      0       0      0
 97%   8037      0      0    8038   80496  18436   57822 105111       0      0     0s    89%   69%  Ff   38%       1      0      0       0      0       0      0
 94%   6078      0      0    6079   81909  27050   75516 132370       0      0     2s    91%   60%  Ff   53%       1      0      0       0      0       0      0
 88%   9009      0      0    9011   53004  30718  114955  37070       0      0     7     94%   28%  :    25%       2      0      0       0      0       0      0
 92%   8093      0      0    8094   39846  29767  135351 113644       0      0     7     95%   99%  Ff   39%       1      0      0       0      0       0      0
 86%   3386      0      0    3387   29391  21747  174987  18285       0      0     7     94%   18%  :    24%       1      0      0       0      0       0      0
 93%   8631      0      0    8632   51926  23599  122516  58558       0      0     7     95%   51%  Ff   33%       1      0      0       0      0       0      0
 92%   6296      0      0    6297   31305  18636  143023  58304       0      0     7     95%  100%  :f   27%       1      0      0       0      0       0      0
 91%   3872      0      0    3921   33872  20821  122773  22259       0      0     7     95%   35%  :    23%      49      0      0       0      0       0      0
 96%   9363      0      0    9364  106888  35007  233705 162940       0      0     2s    96%   75%  Fn   46%       1      0      0       0      0       0      0
 97%  11531      0      0   11549  133391  33845  198742 199055       0      0     1s    96%   94%  Ff   46%      18      0      0       0      0       0      0
 94%   5216      0      0    5217   82463  37715  191201 113337       0      0     2s    95%   54%  Ff   38%       1      0      0       0      0       0      0
 93%   8415      0      0    8416   58934  27807  155552  44060       0      0     2s    95%   36%  :    29%       1      0      0       0      0       0      0
 91%   6129      0      0    6130   41378  13127  116706 125874       0      0     6s    95%   79%  Ff   32%       1      0      0       0      0       0      0
 89%   4437      0      0    4439   32507  30104  232480   6582       0      0     6s    95%    6%  :    35%       2      0      0       0      0       0      0
 93%   8641      0      0    8644   32094  39364  135072     11       0      0    19s    95%    0%  -    21%       3      0      0       0      0       0      0
 89%   9602      0      0    9603   40307  49023  131162  73103       0      0     1s    96%   94%  Tf   32%       1      0      0       0      0       0      0
 92%   3680      0      0    3681   29037  35443  152199  73802       0      0     1s    96%   92%  :    27%       1      0      0       0      0       0      0
ANY1+ ANY2+ ANY3+ ANY4+  AVG CPU0 CPU1 CPU2 CPU3 Network Protocol Cluster Storage Raid Target Kahuna WAFL_Ex(Kahu) WAFL_XClean SM_Exempt Cifs Exempt Intr Host  Ops/s   CP
  99%   87%   65%   35%  72%  70%  76%  78%  64%     32%       0%      0%     21%  27%     0%     3%    139%( 93%)          0%        0%   0%    58%   8%   1%   9305 100%
  99%   94%   84%   60%  85%  84%  89%  91%  75%     31%       0%      0%     23%  36%     0%     3%    159%( 94%)          0%        0%   0%    80%   7%   1%  10018 100%
  96%   82%   64%   37%  71%  67%  77%  80%  59%     20%       0%      0%     29%  36%     0%     4%    119%( 80%)          0%        0%   0%    64%   8%   3%   3773 100%
  99%   89%   75%   47%  78%  77%  83%  85%  66%     18%       0%      0%     26%  36%     0%     3%    144%( 90%)          0%        0%   0%    77%   7%   1%   3285 100%
  94%   80%   60%   31%  67%  64%  72%  76%  55%     18%       0%      0%     25%  33%     0%     3%    121%( 81%)          0%        0%   0%    61%   7%   1%   3079 100%
  96%   85%   69%   43%  74%  72%  79%  80%  63%     18%       0%      0%     25%  35%     0%     3%    138%( 84%)          0%        0%   0%    67%   7%   1%   3151 100%
  93%   75%   56%   31%  65%  62%  69%  72%  56%     27%       0%      0%     21%  26%     0%     5%    127%( 80%)          0%        0%   0%    43%   8%   2%   4700  44%
  98%   84%   64%   37%  71%  69%  75%  77%  64%     31%       0%      0%     22%  27%     0%     4%    144%( 90%)          9%        0%   0%    40%   8%   1%   5306  27%
 100%   89%   75%   52%  79%  77%  81%  84%  75%     21%       0%      0%     22%  34%     0%    11%    147%( 86%)         30%        0%   0%    45%   7%   1%   4459 100%
  95%   82%   64%   36%  70%  67%  76%  79%  59%     26%       0%      0%     29%  36%     0%     3%    126%( 83%)          0%        0%   0%    52%   8%   1%   7247 100%
  94%   78%   57%   31%  66%  62%  70%  73%  58%     31%       0%      0%     21%  27%     0%     4%    126%( 82%)          0%        0%   0%    45%   8%   1%  10036 100%
  95%   79%   60%   37%  68%  65%  72%  76%  60%     30%       0%      0%     23%  31%     0%     4%    131%( 82%)          0%        0%   0%    46%   8%   1%   9865 100%
  99%   91%   72%   37%  75%  73%  80%  83%  64%     32%       0%      0%     22%  29%     0%     3%    139%( 91%)          0%        0%   0%    66%   8%   1%  10653 100%
 100%   94%   76%   35%  76%  76%  79%  83%  67%     21%       0%      0%     14%  17%     0%     2%    148%( 97%)          0%        0%   0%    96%   6%   1%   3739 100%
 100%   92%   73%   34%  75%  74%  75%  79%  72%     29%       0%      0%      7%  11%     0%     1%    154%( 98%)          0%        0%   0%    91%   5%   1%   9558 100%
  99%   86%   66%   30%  71%  68%  71%  75%  68%     25%       0%      0%      8%  11%     0%     2%    146%( 95%)          0%        0%   0%    85%   5%   1%   6398 100%
 100%   91%   72%   34%  75%  74%  76%  81%  68%     32%       0%      0%      9%  10%     0%     2%    154%( 97%)          0%        0%   0%    86%   6%   1%   9660  74%

Re: FAS3270 8.1.3 7 mode High CPU

Hi,

 

it could be nfs getattr traffic.

 

 

option nfs.per_client_stats.enable on

nfsstat -z

 

wait 10s

 

nfsstat -l and pick the ip of the host

nfstat -h <ip>

 

You see the NFS statistics.

 

 

option nfs.per_client_stats.enable off

 

Marcus

Re: FAS3270 8.1.3 7 mode High CPU

I'll run it next time tjhis happens.

 

Seems ok for the last few days.

 

Thanks

Re: FAS3270 8.1.3 7 mode High CPU

its fairly normal to have a single core running at 99% and looking at the other cores they are not all running at 99%.