Data Infrastructure Management Software Discussions

Highlighted

Graphite db not getting any data even though access log shows the metrics

I have setup everything according to the documentation "Quick Start: Installing Graphite and Grafana Christopher Madden, NetApp 7 September, 2015". 

Harvest version 1.6. Graphite version latest available on GitHub. 

I see that the access log shows the below information:

10.23.107.20 - - [30/Jun/2020:12:35:00 +0200] "netapp.perf.C-mode.netapp01.node.netapp01-01.cifs.cifs_op_count.GetAttr 11.9666666666667 1593513300\n" 400 226

10.23.107.20 - - [30/Jun/2020:12:35:01 +0200] "netapp.perf.C-mode.labnetapp01.node.labnetapp01-01.aggr.l01_01_aggr0_STD.plex0.rg0.cp_read_chain 1.53571428571429 1593513301\n" 400 226

10.23.107.20 - - [30/Jun/2020:12:35:08 +0200] "netapp.perf.C-mode.netapp03.svm.bn_backup.vol.bn_fsw.qos_ops 0.241379310344828 1593513306\n" 400 226

 

I cant see the graphite.db getting modified. It shows a modified date from one week ago. The grafana is connected to the graphite. It doesnt show any data either which confirms there is nothing coming into the DB.

I also checked the "metrics received" graph in graphite dashboard and it is empty too.

 

Error log show.

 

[Thu Jun 25 16:28:26.289427 2020] [:error] [pid 26814]   warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')

[Fri Jun 26 11:44:35.873521 2020] [:error] [pid 18431] mod_wsgi (pid=18431): Target WSGI script '/opt/graphite/conf/graphite.wsgi' cannot be loaded as Python module.

[Fri Jun 26 11:44:35.873566 2020] [:error] [pid 18431] mod_wsgi (pid=18431): Exception occurred processing WSGI script '/opt/graphite/conf/graphite.wsgi'.

[Fri Jun 26 11:44:35.873622 2020] [:error] [pid 18431] Traceback (most recent call last):

[Fri Jun 26 11:44:35.873650 2020] [:error] [pid 18431]   File "/opt/graphite/conf/graphite.wsgi", line 25, in <module>

[Fri Jun 26 11:44:35.873739 2020] [:error] [pid 18431]     import graphite.metrics.search

[Fri Jun 26 11:44:35.873757 2020] [:error] [pid 18431]   File "/usr/lib/python2.7/site-packages/graphite/metrics/search.py", line 6, in <module>

[Fri Jun 26 11:44:35.873838 2020] [:error] [pid 18431]     from graphite.storage import is_pattern, match_entries

[Fri Jun 26 11:44:35.873871 2020] [:error] [pid 18431]   File "/usr/lib/python2.7/site-packages/graphite/storage.py", line 9, in <module>

[Fri Jun 26 11:44:35.874029 2020] [:error] [pid 18431]     from graphite.remote_storage import RemoteStore

[Fri Jun 26 11:44:35.874048 2020] [:error] [pid 18431]   File "/usr/lib/python2.7/site-packages/graphite/remote_storage.py", line 8, in <module>

[Fri Jun 26 11:44:35.874351 2020] [:error] [pid 18430] mod_wsgi (pid=18430): Target WSGI script '/opt/graphite/conf/graphite.wsgi' cannot be loaded as Python module.

[Fri Jun 26 11:44:35.874381 2020] [:error] [pid 18430] mod_wsgi (pid=18430): Exception occurred processing WSGI script '/opt/graphite/conf/graphite.wsgi'.

 

 

 

Please advice which is the best place to look at to find out the root cause of this issue.

@vachagan_gratian @yannb 

1 REPLY 1
Highlighted

Graphite db not getting any data even though access log show it is getting performance logs

The graphite db is not getting updated. The metrics seem to be flowing in but the db is not getting updated. I don't understand what the error log is trying to tell. kindly help with a what should we look at to solve this issue. 

 

[root@netappharvestlab-graphite webapp]# pwd 

/opt/graphite/storage/log/webapp

[root@netappharvestlab-graphite webapp]# ll

total 31744

-rw-r--r-- 1 apache apache 15631568 Jun 30 14:37 access.log

-rw-r--r-- 1 apache apache   129654 Jun 26 11:44 error.log

 

[root@netappharvestlab-graphite storage]# ll

total 68

drwxr-xr-x 3 apache apache    23 Jun 15 16:00 ceres

-rw-rw-rw- 1 apache apache 69632 Jun 26 11:44 graphite.db

drwxr-xr-x 2 apache apache     6 Jun 15 16:00 lists

drwxr-xr-x 4 apache apache    37 Jun 15 16:01 log

drwxr-xr-x 3 apache apache    23 Jun 15 16:00 rrd

 

Acces log:

10.23.107.20 - - [30/Jun/2020:14:34:04 +0200] "netapp.capacity.C-mode.netapp02.svm.abc.vol.unb_root.afs_used 11.58984375 1593520440\n" 400 226

10.23.107.20 - - [30/Jun/2020:14:34:04 +0200] "netapp.perf.C-mode.netapp03.svm.netapp06-02.vol.vol0.other_latency 0.00604095159190437 1593520442\n" 400 226

10.23.107.20 - - [30/Jun/2020:14:34:09 +0200] "netapp.perf.C-mode.netapp02.svm.bac_backup.vol.bn_fsw.qos_ops 0.225806451612903 1593520447\n" 400 226

 

Error log: 

[Thu Jun 25 16:28:26.280416 2020] [:error] [pid 26813]   warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')

[Thu Jun 25 16:28:26.289408 2020] [:error] [pid 26814] /usr/lib/python2.7/site-packages/graphite/settings.py:249: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security

[Thu Jun 25 16:28:26.289427 2020] [:error] [pid 26814]   warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')

[Fri Jun 26 11:44:35.873521 2020] [:error] [pid 18431] mod_wsgi (pid=18431): Target WSGI script '/opt/graphite/conf/graphite.wsgi' cannot be loaded as Python module.

[Fri Jun 26 11:44:35.873566 2020] [:error] [pid 18431] mod_wsgi (pid=18431): Exception occurred processing WSGI script '/opt/graphite/conf/graphite.wsgi'.

[Fri Jun 26 11:44:35.873622 2020] [:error] [pid 18431] Traceback (most recent call last):

[Fri Jun 26 11:44:35.873650 2020] [:error] [pid 18431]   File "/opt/graphite/conf/graphite.wsgi", line 25, in <module>

[Fri Jun 26 11:44:35.873739 2020] [:error] [pid 18431]     import graphite.metrics.search

[Fri Jun 26 11:44:35.873757 2020] [:error] [pid 18431]   File "/usr/lib/python2.7/site-packages/graphite/metrics/search.py", line 6, in <module>

[Fri Jun 26 11:44:35.873838 2020] [:error] [pid 18431]     from graphite.storage import is_pattern, match_entries

[Fri Jun 26 11:44:35.873871 2020] [:error] [pid 18431]   File "/usr/lib/python2.7/site-packages/graphite/storage.py", line 9, in <module>

[Fri Jun 26 11:44:35.874029 2020] [:error] [pid 18431]     from graphite.remote_storage import RemoteStore

[Fri Jun 26 11:44:35.874048 2020] [:error] [pid 18431]   File "/usr/lib/python2.7/site-packages/graphite/remote_storage.py", line 8, in <module>

[Fri Jun 26 11:44:35.874351 2020] [:error] [pid 18430] mod_wsgi (pid=18430): Target WSGI script '/opt/graphite/conf/graphite.wsgi' cannot be loaded as Python module.

[Fri Jun 26 11:44:35.874381 2020] [:error] [pid 18430] mod_wsgi (pid=18430): Exception occurred processing WSGI script '/opt/graphite/conf/graphite.wsgi'.

Check out the KB!
NetApp Insights To Action
All Community Forums