That was considered some time ago, the main issue is that Harvest doesn't maintain de databases and path, it only talks to Graphite.
Maybe the best option would be to have a tool alongside Harvest distribution to put in a cron that would recreate aggregate/volume hierarchy. We just need to find someone to do it 🙂
... View more
Hi @hashiya1112,
Harvest 1.2 is pretty old actually, can you upgrade to 1.4 and see if that happens again.
It looks like a glitch in metrics returned by the system, not something we can fix in Harvest but tell us if it happens again with a recent version.
... View more
This is something I recently tested for a customer :
# THIS MIGHT BE DIFFERENT FOR AN ACTUAL RHEL
# See http://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F
yum install -y http://epel.mirror.constant.com/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
# This should work :
yum install -y graphite-web python-carbon
Edit /etc/carbon/storage-schemas.conf, put the following content :
#==================================================================
# Schema definitions for Whisper files. Entries are scanned in order,
# and first match wins. This file is scanned for changes every 60 seconds.
#
# [name]
# pattern = regex
# retentions = timePerPoint:timeToStore, timePerPoint:timeToStore, ...
# Carbon's internal metrics. This entry should match what is specified in
# CARBON_METRIC_PREFIX and CARBON_METRIC_INTERVAL settings
[carbon]
pattern = ^carbon\.
retentions = 60:90d
##
## Dev defaults
##
[netapp.perf.dev]
pattern = ^netapp\.perf\.dev\..*
retentions = 10s:24h, 1m:100d, 15m:395d, 1h:5y
[netapp.poller.perf.dev]
pattern = ^netapp\.poller\.perf\.dev\..*
retentions = 10s:24h, 1m:100d, 15m:395d, 1h:5y
[netapp.perf7.dev]
pattern = ^netapp\.perf7\.dev\..*
retentions = 10s:24h, 1m:100d, 15m:395d, 1h:5y
[netapp.poller.perf7.dev]
pattern = ^netapp\.poller\.perf7\.dev\..*
retentions = 10s:24h, 1m:100d, 15m:395d, 1h:5y
##
## Prod defaults
##
[OPM]
pattern = ^netapp-performance\..*
retentions = 5m:100d, 15m:395d, 1h:5y
[OPM Capacity]
pattern = ^netapp-capacity\..*
retentions = 15m:100d, 1d:5y
[OPM Capacity Poller]
pattern = ^netapp-poller\..*
retentions = 15m:100d, 1d:5y
[netapp.capacity]
pattern = ^netapp\.capacity\.*
retentions = 15m:100d, 1d:5y
[netapp.poller.capacity]
pattern = ^netapp\.poller\.capacity\.*
retentions = 15m:100d, 1d:5y
[netapp.perf]
pattern = ^netapp\.perf\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y
[netapp.poller.perf]
pattern = ^netapp\.poller\.perf\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y
[netapp.perf7]
pattern = ^netapp\.perf7\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y
[netapp.poller.perf7]
pattern = ^netapp\.poller\.perf7\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y
##
## Catch all
##
[defaults]
pattern = .*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y
#==================================================================
Then :
systemctl enable carbon-cache
systemctl start carbon-cache
firewall-cmd --permanent --add-service=httpd
firewall-cmd --reload
Edit /etc/httpd/conf.d/graphite-web.conf , and make sure that section matches (it defaults to "local" instead of "all granted")
<IfModule mod_authz_core.c>
# Apache 2.4
Require all granted
</IfModule>
Run the following command :
PYTHONPATH=/usr/share/graphite/webapp django-admin syncdb --settings=graphite.settings
You will be asked to create a super user, just accept the defaults, and give it a password
Now if you go on http://yourserver/ you should see Graphite web interface
Install Grafana
yum install -y initscripts fontconfig https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-5.0.4-1.x86_64.rpm
systemctl enable grafana-server
systemctl start grafana-server
firewall-cmd --permanent --add-port=3000/tcp
firewall-cmd --reload
Now if you go on http://yourserver:3000/ you should see Grafana web interface (admin/admin to log in)
Add a Datasource
Go in Configurations > Datasources and add a Datasource for Graphite :
Name : Graphite Default : YES (important)
Type : Graphite
URL : http://localhost/
Version 1.0.x (Important)
Leave defaults for the rest
Create an API key for Harvest
Go to Settings > API Keys
Key Name : Harvest
Role : Editor
Click Add and save the key somewhere (i.e. eyJrIjoiOHJKaVNYMzVmT3FERkxuOE1KY3NqeXdXM3JMQlBxTWIiLCJuIjoiSGFydmVzdCIsImlkIjoxfQ==)
You cannot get it after you leave that page, you will use it in Harvest install guide.
Install dependencies for Harvest
yum install -y perl-LWP-Protocol-https perl-XML-Parser perl-JSON
... View more
How did you install Graphite? This guide is for Debian based distributions.
It looks like graphite isn't fully installed, or the system init files aren't installed.
An alternative is to use NAbox, that contains everything
... View more
There is currently no way to do that.
You can, for a given node, go to "vol_summary", but that gives you an average for all the volumes in that node, which is probably not what you're looking for.
We started to think about options to give you an idea of what volumes are impacting a given node the most, but it's early talk so far, there are still loose end needed to tie.
another alternative would be to dynamically create sylbolic links into the whisper structure of graphite to establish aggregate/volume relationship, and regularly update it. It's kind of hacky so not really sure that's a good approach but it would probably work. More something you would find in NAbox than in Harvest itself I'm afraid, but it's being considered.
... View more
Yes, Graphite integration is still available in OCUM 7.3 (that includes OPM) :
Configuring a connection from a Unified Manager server to an external data provider
From there you just have to use Grafana/Graphite integration that is not dependant on NetApp options. Keep in mind that you will have to build your own Dashboards, as there is no pre-built option that I know of.
That's the other great thnig about Harvest, it comes with a plethora of pre-built dashboard but they are only compatible with the set of metrics collected by Harvest (which are much more than the set provided by just OPM)
Hope it helps.
... View more
Hi Alex,
Where does that error come from?
It looks like a problem with OCUM more than Harvest, is OCUM correctly collecting data about that cluster/aggregate?
... View more
No it is not tested with 7.3, but it doesn't mean it doesn't work.
If you are using NAbox, you can just force compatibility and see if there is any problem, if not, you can just go in the templates directory and copy the 7.2 template as 7.3 and see if it works.
... View more
When you say all the hosts, you mean the storage clients ? I don't think this is what fixed the issue, but rebooting a node is more likely. Does it happens often?
... View more
What does the following command returns? vserver services web show -name sysmgr -instance You can also try to restart sysmgr with vserver services web modify
... View more
NAbox is the virtual appliance I was talking about yes, I’m glad you found it. Alerting is a different challenge because grafana can’t really do it (alerting in grafana doesn’t work with template variables, which harvest dashboards are using a lot) and it would be a lot of work to try to implement it manually.
... View more
Yes that would be fairly simple indeed. I think usually you would find the panel that shows IOPS read and write. You can either copy it (Click on title > left menu > Panel JSON, then paste it in a new one) or modify the one you see by "hiding" the read and write (and other...) series (click on the eye) and add another series that uses the "asPercent()" graphite function that's available in the menu. The idea would be to mask what you don't need, and add the "total_ops" metric. Once you have that, you add two series using the asPercent() graphite method, you end up with something like this, after adding metrics E, D and F
... View more
For the forseable future, there is a plan to maintain it, but just to be clear, it is not a "NetApp" plan to maintain it, it still relies on good will of individuals, like Chris before. Short answer is "yes", same as before 🙂
... View more
Releasing a new version of Harvest or anything on toolchest is extremely painful, but we are working on it. This will be solved in Harvest 1.4 when it's available. In the meantime, renaming the template with "7.2" instead of "7.1" does the job. Latest beta of NAbox will take care of it for you as well.
... View more
As you can imagine there is a lot of support for Harvest and NAbox out there, but here is a bit more informations. Chris properly trasnferred all the assets to a group of people, and was kind enought to support us a little bit more when we had questions. We are releasing Harvest 1.4 as we speak, that should get us to 9.3 without issues, and I'm still personally supporting NAbox. Please continue using the communities for your Harvest questions, it's the best way to get support, NAbox is still supported by email from the web site but I should probably monitor these communities as well. Thank you for your support
... View more