Active IQ Unified Manager Discussions

NetApp Harvest Graphite Carbon GAIResolver issue

shashii82
9,010 Views

All, we recently upgraded our NABox OVA to beta version 5.2b and suddenly data collection stopped working. After some analysis we came to know that the GAIResolver module is missing. Any help would be appreciated! 

 

root@harvestbox01:/opt/graphite# systemctl status carbon-relay.service

* carbon-relay.service - graphite carbon relay

   Loaded: loaded (/etc/systemd/system/carbon-relay.service; enabled)

   Active: failed (Result: exit-code) since Wed 2018-09-19 13:42:13 CEST; 5min ago

  Process: 6003 ExecStart=/opt/graphite/bin/carbon-relay.py --pidfile=/opt/graphite/storage/carbon-relay-a.pid start (code=exited, status=1/FAILURE)

  Process: 6001 ExecStartPre=/bin/rm -f /opt/graphite/storage/carbon-relay-a.pid (code=exited, status=0/SUCCESS)

 

Sep 19 13:42:13  harvestbox01 carbon-relay.py[6003]: File "/opt/graphite/lib/carbon/service.py", line 183, in setupRelayProcessor

Sep 19 13:42:13 harvestbox01 carbon-relay.py[6003]: from carbon.client import CarbonClientManager

Sep 19 13:42:13 harvestbox01 carbon-relay.py[6003]: File "/opt/graphite/lib/carbon/client.py", line 15, in <module>

Sep 19 13:42:13 harvestbox01 carbon-relay.py[6003]: from carbon.resolver import setUpRandomResolver

Sep 19 13:42:13 harvestbox01 carbon-relay.py[6003]: File "/opt/graphite/lib/carbon/resolver.py", line 5, in <module>

Sep 19 13:42:13 harvestbox01 carbon-relay.py[6003]: from twisted.internet._resolver import GAIResolver

Sep 19 13:42:13 harvestbox01 carbon-relay.py[6003]: ImportError: No module named _resolver

Sep 19 13:42:13 harvestbox01 systemd[1]: carbon-relay.service: control process exited, code=exited status=1

Sep 19 13:42:13 harvestbox01 systemd[1]: Failed to start graphite carbon relay.

Sep 19 13:42:13 harvestbox01 systemd[1]: Unit carbon-relay.service entered failed state.

 

 

1 ACCEPTED SOLUTION

Raviere
8,669 Views

connect with root account on nabox

edit

/opt/graphite/lib/carbon/client.py

comment line 15

"from carbon.resolver import setUpRandomResolver"

 

and add just after this code just below

 

try:
    from carbon.resolver import setUpRandomResolver
except ImportError:
    setUpRandomResolver = None

 

save

restart cache-relay

systemctl start graphite_carbon_relay.service

View solution in original post

11 REPLIES 11

vachagan_gratian
8,914 Views

Hi, you probably mean v 2.5b, right? Did you verify md5 after download?

 

@yannb, maybe you can look at this? I only worked with 2.3 until now.

yannb
8,859 Views

That's indeed a known bug with a specific version of carbon.

 

For some reason it seems sometimes NAbox updates itself when it's connected to internet.

 

I would try re-applying the most recent 2.5b update available on https://nabox.org/downloads/

 

Le me know how it goes

shashii82
8,759 Views

Thanks, we haven't get chance to do re-apply the update but looks like there is no alternative. 

shashii82
8,756 Views

yes, previous beta version was indeed working correctly. do you think any other alternative or solution?

yannb
8,693 Views

Did you re-apply the update? It's not breaking anything, just upload the tgz in the UI

shashii82
8,629 Views

I haven't tried this for almost a week now.

Raviere
8,670 Views

connect with root account on nabox

edit

/opt/graphite/lib/carbon/client.py

comment line 15

"from carbon.resolver import setUpRandomResolver"

 

and add just after this code just below

 

try:
    from carbon.resolver import setUpRandomResolver
except ImportError:
    setUpRandomResolver = None

 

save

restart cache-relay

systemctl start graphite_carbon_relay.service

shashii82
8,599 Views

@Raviere Great, thank you Raviere! it did worked for me like a charm. My grafana was not working for almost two weeks.

 

root@harvestbox01:~# vi /opt/graphite/lib/carbon/client.py

root@harvestbox01:/opt/graphite/lib/carbon# systemctl status carbon-relay.service
* carbon-relay.service - graphite carbon relay
   Loaded: loaded (/etc/systemd/system/carbon-relay.service; enabled)
   Active: failed (Result: exit-code) since Mon 2018-09-24 17:23:14 CEST; 1 weeks 3 days ago
  Process: 646 ExecStart=/opt/graphite/bin/carbon-relay.py --pidfile=/opt/graphite/storage/carbon-relay-a.pid start (code=exited, status=1/FAILURE)
  Process: 605 ExecStartPre=/bin/rm -f /opt/graphite/storage/carbon-relay-a.pid (code=exited, status=0/SUCCESS)

Sep 24 17:23:13 harvestbox01 carbon-relay.py[646]: File "/opt/graphite/lib/carbon/service.py", line 183, in setupRelayProcessor
Sep 24 17:23:13 harvestbox01 carbon-relay.py[646]: from carbon.client import CarbonClientManager
Sep 24 17:23:13 harvestbox01 carbon-relay.py[646]: File "/opt/graphite/lib/carbon/client.py", line 15, in <module>
Sep 24 17:23:13 harvestbox01 carbon-relay.py[646]: from carbon.resolver import setUpRandomResolver
Sep 24 17:23:13 harvestbox01 carbon-relay.py[646]: File "/opt/graphite/lib/carbon/resolver.py", line 5, in <module>
Sep 24 17:23:13 harvestbox01 carbon-relay.py[646]: from twisted.internet._resolver import GAIResolver
Sep 24 17:23:13 harvestbox01 carbon-relay.py[646]: ImportError: No module named _resolver
Sep 24 17:23:14 harvestbox01 systemd[1]: carbon-relay.service: control process exited, code=exited status=1
Sep 24 17:23:14 harvestbox01 systemd[1]: Failed to start graphite carbon relay.
Sep 24 17:23:14 harvestbox01 systemd[1]: Unit carbon-relay.service entered failed state.

 

root@harvestbox01:/opt/graphite/lib/carbon# systemctl start carbon-relay.service

 

root@harvestbox01:/opt/graphite/lib/carbon# systemctl status carbon-relay.service
* carbon-relay.service - graphite carbon relay
   Loaded: loaded (/etc/systemd/system/carbon-relay.service; enabled)
   Active: active (running) since Fri 2018-10-05 10:57:56 CEST; 3s ago
  Process: 18145 ExecStart=/opt/graphite/bin/carbon-relay.py --pidfile=/opt/graphite/storage/carbon-relay-a.pid start (code=exited, status=0/SUCCESS)
  Process: 18143 ExecStartPre=/bin/rm -f /opt/graphite/storage/carbon-relay-a.pid (code=exited, status=0/SUCCESS)
 Main PID: 18150 (carbon-relay.py)
   CGroup: /system.slice/carbon-relay.service
           `-18150 /usr/bin/python /opt/graphite/bin/carbon-relay.py --pidfile=/opt/graphite/storage/carbon-relay-a.pid start

Oct 05 10:57:55 harvestbox01 systemd[1]: Starting graphite carbon relay...
Oct 05 10:57:56 harvestbox01 carbon-relay.py[18145]: /usr/local/lib/python2.7/dist-packages/cffi/model.py:531: UserWarning: 'point_conversion_form_t' has no valu...gned int'
Oct 05 10:57:56 harvestbox01 carbon-relay.py[18145]: % self._get_c_name())
Oct 05 10:57:56 harvestbox01 systemd[1]: Started graphite carbon relay.
Hint: Some lines were ellipsized, use -l to show in full.

 

 

yannb
7,595 Views

@shashii82, were you running the beta version as well?

shashii82
7,593 Views

@yannb Yes, we did updated it with the recent beta version and it stopped working. But with the help of Raviere I did modified the /opt/graphite/lib/carbon/client.py and it started. However, I'm getting the another error while starting the carbon-relay service i.e.

 

carbon-relay.py[18145]: /usr/local/lib/python2.7/dist-packages/cffi/model.py:531: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'

 

carbon-relay.py[18145]: % self._get_c_name())

 

I hope this won't affect the reporting or anything.

 

 

 

shashii82
8,588 Views

Does this have any impact to the reporting? Please find below error while starting the carbon-relay service:

----------------------------------------------------------------------------------------

Oct 05 10:57:56 harvestbox01 carbon-relay.py[18145]: /usr/local/lib/python2.7/dist-packages/cffi/model.py:531: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'
Oct 05 10:57:56 harvestbox01 carbon-relay.py[18145]: % self._get_c_name())

 

----------------------------------------------------------------------------------------

 

root@harvestbox01:/opt/graphite/lib/carbon# systemctl status carbon-relay.service -l
* carbon-relay.service - graphite carbon relay
   Loaded: loaded (/etc/systemd/system/carbon-relay.service; enabled)
   Active: active (running) since Fri 2018-10-05 10:57:56 CEST; 24min ago
  Process: 18145 ExecStart=/opt/graphite/bin/carbon-relay.py --pidfile=/opt/graphite/storage/carbon-relay-a.pid start (code=exited, status=0/SUCCESS)
  Process: 18143 ExecStartPre=/bin/rm -f /opt/graphite/storage/carbon-relay-a.pid (code=exited, status=0/SUCCESS)
 Main PID: 18150 (carbon-relay.py)
   CGroup: /system.slice/carbon-relay.service
           `-18150 /usr/bin/python /opt/graphite/bin/carbon-relay.py --pidfile=/opt/graphite/storage/carbon-relay-a.pid start

Oct 05 10:57:55 harvestbox01 systemd[1]: Starting graphite carbon relay...
Oct 05 10:57:56 harvestbox01 carbon-relay.py[18145]: /usr/local/lib/python2.7/dist-packages/cffi/model.py:531: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'
Oct 05 10:57:56 harvestbox01 carbon-relay.py[18145]: % self._get_c_name())
Oct 05 10:57:56 harvestbox01 systemd[1]: Started graphite carbon relay.

 

 

Public