Grafana - alerting issue

[bkamil]

Hi,

I'm trying to define alerts for some of the graphs in Grafana. I get:

error tsdb.HandleRequest() error Request failed status: 500 Internal Server Error

every time when I press "Test Rule" button.

My queries are the most basic ones - like "WHEN avg () OF query (A, 5m, now) IS ABOVE 10", etc.

/opt/graphite/storage/log/webapp/exception.log tells me the following when that happens:

2017-06-13,09:27:37.044 :: Exception encountered in <POST http://localhost:80/render>
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/django/core/handlers/base.py", line 147, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/graphite/webapp/graphite/render/views.py", line 124, in renderView
    seriesList = evaluateTarget(requestContext, target)
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 8, in evaluateTarget
    result = evaluateTokens(requestContext, tokens)
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 29, in evaluateTokens
    return evaluateTokens(requestContext, tokens.expression, replacements)
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 54, in evaluateTokens
    args = [evaluateTokens(requestContext, arg, replacements) for arg in tokens.call.args]
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 29, in evaluateTokens
    return evaluateTokens(requestContext, tokens.expression, replacements)
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 54, in evaluateTokens
    args = [evaluateTokens(requestContext, arg, replacements) for arg in tokens.call.args]
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 29, in evaluateTokens
    return evaluateTokens(requestContext, tokens.expression, replacements)
  File "/opt/graphite/webapp/graphite/render/evaluator.py", line 59, in evaluateTokens
    return func(requestContext, *args, **kwargs)
  File "/opt/graphite/webapp/graphite/render/functions.py", line 2198, in highestAverage
    return sorted( seriesList, key=lambda s: safeDiv(safeSum(s),safeLen(s)) )[-n:]
TypeError: bad operand type for unary -: 'list'

I've had this problem when using the software versions suggested in Chris doc and I still have it now after upgrading most of the components.

Currently I'm at: Grafana 4.3.2, Graphite (whisper, carbon, graphite-web) 1.0.1, Django 1.9.

Any ideas?

[madden · Accepted solution]

Hi @bkamil

According to http://docs.grafana.org/alerting/rules/  "The query used in an alert rule cannot contain any template variables."  All of the default dashboard panels use template variables so to configure alerting you would need to create new dashboard panels without templates and then configure alerting on them.  Could that be your issue?

Cheers,
Chris Madden

Solution Architect - 3rd Platform - Systems Engineering NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

[bkamil]

That was it, Chris!

Thanks a lot.

Kamil