NetApp + Onehouse: Unifying Data From BI to AI, Across Datacenters, Hyperscalers and Neoclouds
At NetApp INSIGHT 2026, NetApp previewed AI Data Engine integrations with Onehouse. But what does that actually mean?
Can you take all the data already sitting on NetApp, in object stores and file systems, and run fast analytics directly on it without moving it? Is it possible to get a state-of-the-art lakehouse running inside your datacenter? How can you feed exabytes of unstructured data accumulated over decades to AI?
Over the past few months, the NetApp and Onehouse engineering teams have been answering these questions as we integrated the Onehouse stack directly against NetApp StorageGRID and ONTAP. We tested analytical performance, benchmarked StorageGRID behavior using open Apache Iceberg™ and other open table formats, ran ETL and table management, synchronized catalogs and served interactive queries. The results are extremely encouraging.
More importantly, the partnership points at a much broader architectural shift that will unfold over the next few years. The last decade of data infrastructure was largely built around the hyperscalers. BigQuery, Snowflake and Databricks pushed distributed data processing forward tremendously, but the dominant consumption model has been simple: move your data into a major cloud, then consume the data services available there.
AI is starting to break that assumption. A 2026 survey of 1,800 IT leaders found 56% run or plan to run production AI inference on private cloud, and an IDC whitepaper found AI is the top reason enterprises are moving workloads off public cloud.
https://cloudian.com/press/enterprise-repatriating-ai-workloads/
At the same time, a large amount of valuable enterprise data never left the datacenter to begin with. Decades of documents, images, engineering data and research archives remain on enterprise file systems, while GPU infrastructure is spreading into neoclouds and private AI factories and generating entirely new datasets there.
Instead of pulling all of this data back into yet another centralized platform, we think the data platform should be able to go where the data already lives. The NetApp + Onehouse integration deeply integrates run-anywhere storage with a run-anywhere open data lakehouse.
Two platforms built to run where your data lives
Onehouse brings a fast, full-featured, state-of-the-art lakehouse into your own infrastructure. We created the first data lakehouse at Uber to support analytics, data science and machine learning on one data foundation instead of constantly moving data between separate systems. That architecture has since become an industry standard: object storage provides the durable data layer, while the lakehouse adds database capabilities such as transactions, indexing, updates and schema evolution on top. The workloads keep expanding too—from BI to data science and machine learning, and now AI, probably the most exciting workload yet.
Today, the Onehouse platform spans ingestion with OneFlow, Spark and SQL processing with Quanton, table optimization, multi-catalog support with OneSync, and low-latency queries with Lakegres. We are extending the same foundation to unstructured data, vector search, inference routing, agent access and AI training pipelines. Unlike a monolithic cloud platform, these services are modular, run on Kubernetes and keep data in open table and file formats. The same Onehouse platform can run in a datacenter, hyperscaler or next to GPUs in a neocloud.
NetApp has spent decades solving the same portability problem one layer below. ONTAP spans file, block and object workloads across datacenters and major clouds, while StorageGRID provides large-scale S3-compatible object storage. NetApp is now extending that foundation into AI with the AI Data Engine for discovering and activating data across its storage estate. Looking ahead, NeoClouds and AI factories represent an exciting area for deeper collaboration. As Novus evolves, we see significant potential for Onehouse data services and NetApp AI infrastructure to help customers manage, prepare, and govern AI data closer to large-scale GPU environments. Together, NetApp can provide the storage plane across these environments while Onehouse brings the open database and data-platform layer above it.
One data platform for human and AI Agents
The reason this combination matters now is that the consumers of enterprise data are changing. Traditional lakehouse workloads were primarily driven by humans: applications generated data, engineers built pipelines, analysts queried the results with SQL and data scientists used the same foundation for machine learning. AI adds a second class of consumer that operates at a 100x more speed and scale. An agent might issue dozens of SQL or search requests while answering one question, a retrieval system may need vector and text search across millions of documents, and a training pipeline may continuously turn production traces and evaluation outcomes back into the next training dataset.
The underlying platform therefore has to serve both human and machine workloads without splitting enterprise data into separate systems. Structured tables still need ETL, CDC, optimization, BI and interactive SQL, while unstructured data increasingly needs extraction, embeddings, vector and text indexes, and low-latency serving to agents. Training systems need to consume large curated datasets next to GPUs, and the traces produced by those systems need to flow back into the data platform. The lakehouse is a natural place for these workloads to meet because it already provides the database layer over durable, inexpensive, exascale storage.
The important part is not the individual boxes. It is that the same database and data-management layer can stretch across datacenters, hyperscalers and neoclouds without centralizing data inside one vendor’s storage system or control plane.
StorageGRID + Onehouse: a state-of-the-art lakehouse on enterprise object storage
StorageGRID was the obvious place to start because modern lakehouses are built around object storage, and StorageGRID exposes the same S3-compatible interface the engines expect from cloud object stores. StorageGRID also brings more than S3 compatibility to a lakehouse architecture. The platform provides a single global namespace across sites, policy-based ILM for data placement, protection, retention, and compliance, and an active-active architecture designed for large-scale production deployments. Table commits need read-after-write behavior, and StorageGRID offers per-bucket consistency controls. With support for massive object counts, multi-petabyte scale, object versioning, and immutable protection, StorageGRID helps customers operate lakehouse data as a governed system of record rather than simply a collection of files stored in object storage. Spark ETL performs large scans and shuffles, MERGEs combine reads and writes, table formats continuously rewrite data and metadata, and interactive queries can generate huge numbers of small ranged reads against storage. We wanted to know whether these core lakehouse workloads could run on StorageGRID with the performance we expect from the cloud.
The tests ran on a 4-node Kubernetes (RKE2) cluster in the NetApp lab, using typical datacenter server-class machines with roughly 80 Intel CPU cores and ~700 GB of RAM per node—about 320 cores and 3 TB of memory in total. Please check the infrastructure used in the test
We exercised the main parts of the lakehouse stack:
- Large-scale Spark ETL and transformations. Quanton ran 1 TB TPC-DS workloads directly against StorageGRID, exercising the scans, joins, aggregations and shuffles that dominate real Spark pipelines.
- Open table writes and updates. Quanton wrote and updated Iceberg tables using MERGE-style pipelines common in production ETL.
- Continuous table management. Onehouse Table Optimizer handled compaction and file optimization while tables received writes every few minutes.
- Catalog and engine interoperability. Tables were registered through an Iceberg REST Catalog-compatible endpoint and accessed directly from Onehouse, Presto and open-source Spark.
- Interactive SQL. Lakegres queried the same lakehouse data with a five-worker configuration, adding a latency-sensitive workload with very different storage access patterns from batch Spark.
Benchmark results
The results were consistent across the stack: Spark ETL, table maintenance, interactive queries and the storage layer itself all performed well with StorageGRID underneath them.
1. Interactive SQL: Lakegres vs. Presto
Lakegres completed the 1 TB TPC-DS workload in 865 seconds versus 1,378 seconds for Presto on the same five-worker configuration—roughly 37% lower aggregate query runtime, or about 1.6x faster. Based on our routine cloud benchmarks, Lakegres tends to pull further away as data scales toward 10 TB, as joins start shuffling data instead of remaining small broadcast joins. But we could not run that larger test here because the four-node k8s cluster did not have enough local storage capacity to handle the shuffle volume.
2. Spark ETL: Quanton vs. OSS Spark
At Onehouse, we have published our ETL benchmarking methodology along with open-source tools to reproduce the benchmarks yourself. To measure the “ET” part of ETL performance, we ran the same 1 TB TPC-DS workload directly in the NetApp environment using Apache Spark, comparing our Quanton engine with stock open-source Spark. Quanton completed the suite in 795 seconds versus 3,489 seconds for OSS Spark—4.4x faster on the same StorageGRID-backed Kubernetes environment. The performance gains we see from Quanton in the cloud carried over directly when the engine ran on-prem against StorageGRID.
We also validated the write-heavy “L” side of lakehouse ETL using synthetic Iceberg MERGE pipelines and continuous table maintenance on StorageGRID. The behavior was consistent with the workloads we have already documented in our S3 Tables testing: Quanton accelerated the MERGE path, while Onehouse Table Optimizer continuously compacted and optimized the resulting Iceberg tables, without any surprises. Quanton can typically run these I/O-intensive workloads 4x faster than leading cloud-native solutions like AWS S3 Tables.
3. Storage throughput: StorageGRID vs. Amazon S3
We wanted to dig deeper into the underlying storage layer to understand the differences between cloud object storage like Amazon S3 and StorageGRID. For this, we built a stress-test harness focused specifically on ranged reads, the staple storage operation issued by engines like Quanton and Lakegres when reading columnar file formats like Apache Parquet. The stress test harness was implemented as a Spark application that drives random range reads against pre-generated object in parallel from each executor, run with six executors. The harness client was run from the on-prem cluster against StorageGRID and in an AWS account against S3, to ensure the network setup is fair. StorageGRID delivered 8.56 GB/s versus 5.15 GB/s from Amazon S3 at lower concurrency, while median request latency was 203 ms versus 358 ms.
At very high concurrency, the StorageGRID environment eventually saturated the lab’s shared network at ~9 GB/s; adding storage nodes or network bandwidth scales throughput further. That gives us a clear area for further tuning, but the important result is that StorageGRID was already capable of feeding a high-performance analytical engine with substantial throughput benefits.
4. Cloud vs. on-prem: equivalent Intel compute
Next, we wanted to answer how the entire stack compared on-prem versus in the cloud. On-prem stack ran StorageGrid + Onehouse on the on-prem k8s cluster and the cloud stack ran S3 + Onehouse on Amazon EKS. As the astute reader may have noticed, our 10 TB Quanton results from the cloud use AWS Graviton processors. But our StorageGRID test environment runs on Intel processors. We performed another head-to-head TPC-DS run using comparable Intel instances for both Quanton on StorageGRID and Quanton on AWS S3. Both configurations used five workers at 16 cores and 64 GB each, for 80 worker cores and 320 GB of memory.
Across the 103-query 1 TB TPC-DS suite, including all a/b query variants, the NetApp + StorageGRID setup completed in 836.8 seconds versus 1,068.9 seconds on AWS Intel + S3—about 22% lower aggregate runtim e. A newer AWS Graviton configuration completed the same workload in 677 seconds, so matching the CPU architecture was important to compare the StorageGRID/on-prem and S3/AWS setups more meaningfully. In the validation, CPU architecture caused the gap, not the storage.
The result we care about is that bringing the lakehouse on-prem did not require giving up cloud-class performance. StorageGRID sustained the throughput and access patterns required across batch ETL, table maintenance and interactive querying. Taken together, these tests cover the core pieces of a production lakehouse: fast ETL, incremental writes, table maintenance, open catalogs and interactive SQL. For enterprises that already have large StorageGRID estates, Onehouse can bring those capabilities directly to the existing storage rather than requiring the data to be copied into another cloud data platform or HDFS. The data stays in open formats and under the customer's control; the compute and data-management layer moves to where the data already lives.
ONTAP + Onehouse: Unlocking unstructured data to AI Models
One of the most profound shifts from the rise of AI models has been their ability to analyze data. Companies have been collecting data in unstructured data formats as pdfs, documents, files, images, videos for decades and a good amount of that is already stored in NetApp storage products like ONTAP. Historically, analyzing this data has been bottlenecked by both human data-entry operators and data analysts. Today, embedding models can extract content from unstructured data and LLMs can analyze them, all at GPU speeds. However, just pointing an AI agent to a dump of files on storage will not yield accurate results. If the lakehouse is fundamentally a database over object storage, there is no reason that the database should stop at rows and columns. The same transactional, indexed and queryable foundation should be able to manage the files, embeddings, metadata and derived representations AI needs as well.
That is the direction we have been pushing with open table formats and Quanton. Hudi 1.2 moves the lakehouse itself closer to this model: it adds native VECTOR, BLOB and VARIANT types, while its pluggable table-format layer can natively write Iceberg, including semi-structured VARIANT data. File-format support extends beyond Parquet and ORC to newer formats like Lance and Vortex. Put together, this provides a unified analytical and multimodal lakehouse storage foundation for Quanton to sit on top of, reading unstructured files, chunking and embedding them, and joining them with structured tables.
Below is a full example of how Quanton can turn a dump of files/objects in storage to tables with metadata, chunks and vectors that a model can readily reason with.
import quanton_unstructured
from pyspark.sql import functions as F
quanton_unstructured.register(spark)
# Read PDFs, DOCX, HTML, MD, TXT directly from ONTAP / object storage.
# content is the original file: inline bytes for small objects,
# or a path/offset/length pointer for larger ones.
# Quanton also parses and chunks each document on read; `chunks` is
# a pre-split array<string> ready for embedding (default ~2K chars).
docs = (
spark.read
.format("quanton_unstructured")
.load("s3://enterprise-files/")
.where("parse_status = 'SUCCESS'")
)
# Explode the reader-generated chunks into one row per chunk,
# while carrying the source BLOB and chunker metadata with each row.
chunks = (
docs.select(
"uri", "content", "content_type", "modification_time", "chunker_used",
F.posexplode("chunks").alias("pos", "chunk"))
.withColumn("chunk_id", F.concat_ws("#", "uri", "pos"))
)
# Generate a vector for every chunk.
# embed_partition uses BAAI/bge-base-en-v1.5 via sentence-transformers.
ready = (
chunks.repartition(8)
.mapInPandas(embed_partition, schema=EMBED_SCHEMA)
)
# ready now contains the source BLOB, chunks and their AI representation:
# uri | content (BLOB) | chunk | embedding (VECTOR) | parser/chunker metadata
# Hudi: VECTOR/BLOB/VARIANT + Lance/Vortex base files.
# Iceberg/Delta metadata can be exposed through Hudi's pluggable table-format layer via XTable.
# The same Hudi table-services layer can sit over Lance or Vortex base files.
(ready.write
.format("hudi")
.option("hoodie.table.base.file.format", "LANCE")
.option("hoodie.datasource.write.recordkey.field", "chunk_id")
.mode("overwrite")
.save("s3://lakehouse/enterprise_corpus"))
ONTAP makes that architecture especially compelling because an enormous amount of enterprise unstructured data already lives there. The source objects can remain on NetApp while Onehouse builds the queryable and searchable data layer around them. Lakegres can serve structured lookups and low-latency SQL against the resulting lakehouse data, while theOnehouse AI Gateway governs how agents access models and tools. An agent can search years of documents, join the matching content against structured business tables and use the resulting context during inference without first requiring the enterprise to move its entire corpus into yet another proprietary AI data store. Ultimately, we want to extend OneFlow, our ingestion service to also keep vectors and content in sync as data changes on NetApp storage, similar to how it works for cloud storage like Amazon S3 or Google Cloud Storage.
That is what “AI-ready data” should mean. AI does not require throwing away the architecture we built for analytics and machine learning. It extends the lakehouse into new data types, indexes and access patterns while preserving the same fundamental idea: keep one durable data foundation and bring intelligence to where data is.
Novus + Onehouse: extending the same data platform into AI factories
The same principle applies at the other end of the spectrum, where new data is being created next to very large GPU clusters. Neoclouds and private AI factories are becoming new centers of data gravity because training and inference themselves produce enormous amounts of data. Training generates checkpoints, intermediate datasets and evaluation results, while production inference produces prompts, responses, agent traces, tool calls and reward signals that increasingly feed directly into post-training. These look like AI workloads on the surface, but underneath they are also large-scale data-management workloads.
NetApp Novus is aimed at exactly this environment, with a disaggregated architecture designed to feed very large AI clusters with enormous throughput. The database and processing layer running against StorageGRID in a datacenter could therefore run next to GPU infrastructure in a neocloud without forcing data back through a hyperscaler data platform first.
The resulting loop is straightforward. Raw training data sits on NetApp, Quanton performs filtering, joins, deduplication and curation, and open table formats version the resulting training datasets. GPU training systems consume those datasets, while production traces and evaluation outcomes flow back into the lakehouse and become inputs to the next round of curation. Our AI training data pipelines already follow this model: Onehouse manages and versions the data while specialized systems such as Fireworks, Together AI and other training platforms handle GPU-heavy model training. Storage, data processing and training remain independently replaceable instead of collapsing into one giant vertically integrated service.
A data platform that follows the data
We started with StorageGRID because object storage is the lakehouse’s native layer, and those results gave us the confidence to extend to ONTAP and Novus. We now have real engineering results behind that integration, and there is a clear path to giving StorageGRID customers a modern open lakehouse without forcing them to migrate their data into another proprietary platform. But the more interesting thing we learned through the work is that the same architecture extends naturally beyond StorageGRID into ONTAP, unstructured enterprise data and eventually the new storage systems being built underneath AI factories. Building on standardized open lakehouse formats makes it easier to exchange data across different environments and data estates: existing structured data and unstructured data can leverage StorageGRID and ONTAP and new AI training (inputs and outputs) can leverage Novus.
The common thread is portability. NetApp has spent decades building infrastructure that allows enterprise data to live across datacenters and clouds, and is now extending that footprint into AI infrastructure with Novus and the AI Data Engine. Onehouse has been taking the capabilities users expect from modern cloud data platforms — ingestion, fast compute, table management, catalogs, interactive serving and now AI data pipelines — and making them available as modular services that run on Kubernetes and operate on open data. One provides the storage plane, the other provides the database and data-platform layer, and neither requires the customer to centralize everything inside one vendor’s proprietary cloud.
That is why we are excited about where this partnership can go next. We plan to push the StorageGRID integration further, expand the ONTAP path for unstructured and AI data, integrate more deeply with NetApp’s AI Data Engine and explore how the same architecture fits as Novus evolves. The end state is simple to describe: whether the data lives in an enterprise datacenter, a hyperscaler or a neocloud next to thousands of GPUs, the same open data platform should be able to run where the data lives.
Enterprise data should not have to move to where the data platform is. The data platform should be able to move to the data.
That is what we are building together. For any future questions or support, please feel free to contact info@onehouse.ai or nkarthik@netapp.com
