Object Storage

St.node util on some nodes

sanbre
941 Views

Hi,

 

we recently extended our st.grid env. with 4 nodes per site (8 new nodes total, whole grid now 16 nodes).

What we see in our monitoring that it fills up until 99.88% on 3 nodes, now that we can't handle in our alerting/monitoring.

 

Is there any way we can spread the load of these full nodes to the other nodes in the cluster or is there a way to set a threshold on the nodes that it fills up to (let's say)90% and no more?

 

Nodes are virtual (ESXi), not appliances.

3 REPLIES 3

Ontapforrum
922 Views

Some pointers - Planning a StorageGRID expansion in the following pdf.  In addition, it may be worth raising a ticket and get more help around this.

 

Following section talks about rebalancing: (Page-9) 
https://library.netapp.com/ecm/ecm_download_file/ECMLP2873547

 

What is EC rebalancing?
EC rebalancing is a StorageGRID procedure that might be required after a Storage Node
expansion. The procedure is run as a command-line script from the primary Admin Node. When
you run the EC rebalance procedure, StorageGRID redistributes erasure-coded fragments among
the existing and the newly expanded Storage Nodes at a site

 

https://kb.netapp.com/Advice_and_Troubleshooting/Hybrid_Cloud_Infrastructure/StorageGRID/How_to_read_the_StorageGRID_EC_rebalance_output

 

Docs: 
https://mysupport.netapp.com/documentation/docweb/index.html?productID=63606&language=en-US

 

Chris_Jones
824 Views

I'd definitely recommend upgrading to StorageGRID 11.6.0.4 and then running the EC Rebalance. You get some nice graphs in the Support > Metric > EC Overview section that give you progress status and estimated completion time details.

aronk
822 Views

If you are using Erasure coding then you can make use of the EC rebalancing as mentioned above. If you are only storing replicas, then it is a bit more difficult to rebalance the stored objects.  You can override the read-only watermarks under Configuration/Storage Options. you can also adjust the alert thresholds in Alert/Rules, but as long as there are sufficient nodes with free capacity to to meet the ILM rules, there is no need to do anything about the older full nodes.

Public