Tech ONTAP Blogs

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Tech ONTAP Blogs

OpenSearch on NetApp ONTAP (FSxN): Technical Reference Architecture & Deployment Guide

Subbuj
NetApp
NetApp
yesterday
46 Views

Introduction

OpenSearch is a powerful, open-source search and analytics engine, forked from Elasticsearch and Kibana, and is widely used for e-commerce, observability, anomaly detection, and more. While OpenSearch is designed to run on local disks, many enterprises leverage NetApp ONTAP storage solutions—such as AWS FSxN, Azure NetApp Files, Google Cloud NetApp Volumes, and on-premises AFF arrays—for scalable, efficient, and resilient storage.

This guide explains how to architect, deploy, and benchmark OpenSearch clusters using NetApp ONTAP storage (NFS/iSCSI), with configuration details for both VM and Docker-based deployments.

Reference Architecture Overview

Subbuj_0-1761069937418.jpeg

Key Design Considerations

  • Cluster Sizing: Minimum 3 nodes for production, each with at least 8 vCPUs.
  • Shards: Keep shard sizes between 10–50 GB. Each index consists of primary and replica shards (replication factor of 3 is typical).
  • Storage: Use NetApp ONTAP volumes (NFS/iSCSI) for shared, scalable, and resilient storage.
  • Networking: Deploy nodes across availability zones for high availability. Configure VPC/VNet, subnets, and security groups as per cloud provider.
  • Dashboards: Deploy OpenSearch Dashboards behind a load balancer for HTTPS access.
  • Benchmarking: Use OpenSearch Benchmark for performance testing.

Step-by-Step Deployment Guide

  1. Networking Setup
    • AWS: Create a VPC, subnets (one per AZ), and security groups for OpenSearch nodes and ONTAP volumes.
    • Azure: Create a Resource Group, VNet, and delegated subnet for Azure NetApp Files.
    • Google Cloud: Create a VPC and subnets; configure firewall rules.
  2. Provision NetApp ONTAP Storage
    • AWS FSxN: Create FSx for NetApp ONTAP volumes (NFS/iSCSI).
    • Azure NetApp Files: Create volumes in the delegated subnet.
    • Google NetApp Volumes: Provision volumes as per requirements.
    • On-Prem AFF: Create NFS/iSCSI volumes.
    • Best Practice: Provision one volume per OpenSearch node for optimal performance. Size volumes based on expected data growth and required IOPS.
  1. VM-Based Deployment
    1. Provision VMs
      • Deploy Linux VMs (Ubuntu recommended) in each AZ/subnet.
      • Assign sufficient CPU (≥8 vCPUs) and memory (≥32 GB RAM).
    2. Install NFS Utilities & Mount Volumes 
      sudo apt update && sudo apt upgrade -y
      sudo apt install nfs-common -y
      sudo mkdir -p /mnt/nfs/osdata
      echo "<nfs_server>:/vol1 /mnt/nfs/osdata nfs defaults 0 0" | sudo tee -a /etc/fstab
      sudo mount -a
    3. Install OpenSearch
    4. Configure OpenSearch Data Path
      • Configure OpenSearch Data Path  
        Edit opensearch.yml:
        path.data: /mnt/nfs/osdata
        cluster.name: OpenSearch-cluster
        node.name: <node_name>
        discovery.seed_hosts: ["<node-1-IP>", "<node-2-IP>", "<node-3-IP>"]
        cluster.initial_master_nodes: ["<node-1-IP>", "<node-2-IP>", "<node-3-IP>"]
    5. Deploy OpenSearch Dashboards 
      • Provision a VM, install Dashboards, and configure to connect to the cluster.
  1. Docker-Based Deployment 
      1. Mount NFS Volume 
        sudo mkdir -p /data/data1
        sudo mount -t nfs <nfs_server>:/vol1 /data/data1
      2. Run OpenSearch Container 
        docker run -d --network host \
          -v /data/data1:/usr/share/opensearch/data \
          -e "cluster.name=OpenSearch-cluster" \
          -e "node.name=docker1" \
          -e "discovery.seed_hosts=<node-1_IP>,<node-2_IP>,<node-3_IP>" \
          -e "cluster.initial_master_nodes=<node-1_IP>" \
          -e "path.data=/usr/share/opensearch/data" \
          opensearchproject/opensearch:latest
      3. Run Dashboards Container 
        docker run -d --network host \
          -e "OPENSEARCH_HOSTS=https://<node-1_IP>:9200,https://<node-2_IP>:9200,https://<node-3_IP>:9200" \
          -e "OPENSEARCH_SSL_VERIFICATION=none" \
          -e "OPENSEARCH_USERNAME=admin" \
          -e "OPENSEARCH_PASSWORD=admin" \
          opensearchproject/opensearch-dashboards:latest

        ##**Note: Replace default credentials in production and configure security modules
  1. Benchmarking OpenSearch Performance 
    1. Install OpenSearch Benchmark: 
       git clonehttps://github.com/opensearch-project/opensearch-benchmark
       cd opensearch-benchmark
       python3 -m venv .venv
       source .venv/bin/activate
       pip install -r requirements.txt
    2. Run Benchmark Tests:
      List workloads:       opensearch-benchmark list workloads
      Execute test (sample mode): opensearch-benchmark execute_test --distribution-version=2.2.1 --workload=geonames --test-mode
      Full dataset: opensearch-benchmark execute_test --distribution-version=2.5.0 --workload=nyc_taxis
      Custom workload:
      opensearch-benchmark create-workload --workload=sjtestlogs --indices=dashboards_sample_data_logs --target-hosts="http://admin:admin@<load-balancer>:9200 
      opensearch-benchmark execute_test --distribution-version=2.5.0 --workload-path=./workloads/sjtestlogs

Performance Observations

  • NetApp ONTAP LUNs (iSCSI) and NFS volumes outperform local SSDs for large-scale, distributed OpenSearch clusters.
  • Shared storage simplifies node recovery, scaling, and operational overhead.
  • ONTAP features (deduplication, compression, tiering) reduce storage costs and improve efficiency.
  • High availability and encryption at rest are built-in with ONTAP.
  • Performance Insights: Our benchmarks show NetApp ONTAP volumes (NFS/iSCSI) deliver better performance and operational simplicity compared to local SSDs, especially for large, distributed clusters.

Pricing References

Conclusion

Deploying OpenSearch on NetApp ONTAP (FSxN) provides scalable, resilient, and cost-effective storage for search and analytics workloads—whether in the cloud or on-premises. By leveraging shared storage, you gain operational simplicity, high availability, and flexibility to scale as your data grows.

Next Steps: For help with architecture or troubleshooting, reach out to NetApp or us at “ng-netappdataplatform-team@netapp.com” and find more details at our DPT confluence page

Appendix: Quick Reference Commands

Mount NFS Volume (Linux)

sudo apt install nfs-common -y
sudo mkdir -p /mnt/nfs/osdata
sudo mount -t nfs <nfs_server>:/vol1 /mnt/nfs/osdata

OpenSearch Docker Run

docker run -d --network host \
  -v /mnt/nfs/osdata:/usr/share/opensearch/data \
  -e "cluster.name=OpenSearch-cluster" \
  -e "node.name=node1" \
  opensearchproject/opensearch:latest

OpenSearch Benchmark Example

opensearch-benchmark execute_test --distribution-version=2.5.0 --workload=nyc_taxis

OpenSearch on NetApp ONTAP (FSxN): Final Thoughts

OpenSearch is a distributed, open-source search and analytics engine, forked from Elasticsearch and Kibana. While designed for local disks, many organizations prefer the scalability and efficiency of NetApp ONTAP storage—available in AWS, Azure, Google Cloud, and on-premises.

OpenSearch on NetApp ONTAP is a robust solution for organizations needing scalable, resilient, and efficient search and analytics infrastructure—whether in the cloud or on-premises.

Why NetApp ONTAP for OpenSearch?

  • Scalability: Easily grow storage as data increases.
  • Resilience: High availability and built-in encryption.
  • Efficiency: Deduplication, compression, and tiering reduce costs.
  • Flexibility: Deploy anywhere—cloud or on-premises.
  • A unified central data repository eliminates the need to move data in the future, even when transitioning between technology stacks. It helps avoid ETL processes, refactoring, and redesigning your ecosystem—allowing you to focus on business needs and innovation improvements.
0 Kudos
All Community Forums
Public