topic Re: St.node util on some nodes in Object Storage

St.node util on some nodes

sanbre — Wed, 04 Jun 2025 09:56:07 GMT

Hi,

we recently extended our st.grid env. with 4 nodes per site (8 new nodes total, whole grid now 16 nodes).

What we see in our monitoring that it fills up until 99.88% on 3 nodes, now that we can't handle in our alerting/monitoring.

Is there any way we can spread the load of these full nodes to the other nodes in the cluster or is there a way to set a threshold on the nodes that it fills up to (let's say)90% and no more?

Nodes are virtual (ESXi), not appliances.

Re: St.node util on some nodes

Ontapforrum — Fri, 21 Oct 2022 15:45:55 GMT

Some pointers - Planning a StorageGRID expansion in the following pdf. In addition, it may be worth raising a ticket and get more help around this.

Following section talks about rebalancing: (Page-9)
https://library.netapp.com/ecm/ecm_download_file/ECMLP2873547

What is EC rebalancing?
EC rebalancing is a StorageGRID procedure that might be required after a Storage Node
expansion. The procedure is run as a command-line script from the primary Admin Node. When
you run the EC rebalance procedure, StorageGRID redistributes erasure-coded fragments among
the existing and the newly expanded Storage Nodes at a site

https://kb.netapp.com/Advice_and_Troubleshooting/Hybrid_Cloud_Infrastructure/StorageGRID/How_to_read_the_StorageGRID_EC_rebalance_output

Docs:
https://mysupport.netapp.com/documentation/docweb/index.html?productID=63606&language=en-US

Re: St.node util on some nodes

Chris_Jones — Sun, 23 Oct 2022 22:03:07 GMT

I'd definitely recommend upgrading to StorageGRID 11.6.0.4 and then running the EC Rebalance. You get some nice graphs in the Support > Metric > EC Overview section that give you progress status and estimated completion time details.

Re: St.node util on some nodes

aronk — Mon, 24 Oct 2022 00:09:37 GMT

If you are using Erasure coding then you can make use of the EC rebalancing as mentioned above. If you are only storing replicas, then it is a bit more difficult to rebalance the stored objects. You can override the read-only watermarks under Configuration/Storage Options. you can also adjust the alert thresholds in Alert/Rules, but as long as there are sufficient nodes with free capacity to to meet the ILM rules, there is no need to do anything about the older full nodes.