How do you achieve consistent high performance for Oracle workloads in the cloud—without overprovisioning storage or tuning blindly?
This document demonstrates how Google Cloud NetApp Volumes (GCNV) over iSCSI delivers predictable, enterprise-grade performance for Oracle Database workloads. Through configuration best practices, it shows how Oracle, Linux multipath, and iSCSI path parallelism can be combined to achieve high random I/O throughput, low latency, and scalable performance for OLTP environments.
Testing focused on random 8 KB I/O—the pattern most representative of OLTP database access—using a working set much larger than the Oracle buffer cache so results reflect real storage capability rather than memory-resident reads.
Results achieved:
|
Workload |
Performance |
|
100% Random Read |
~157K read IOPS |
|
90% Read / 10% Update |
~146K total IOPS |
|
Best Operating Latency |
Sub-millisecond |
The read-heavy workload shows how GCNV behaves under maximum random-read demand. The 90/10 workload adds a modest write component typical of update-heavy OLTP, which is useful when sizing storage for applications that are not read-only.
|
The key takeaway is simple: Make Oracle issue physical I/O, expose sufficient storage parallelism, and measure performance at the Linux multipath device layer. Latency values in this document are host-observed device latency from iostat on the Linux multipath device. |
Solution Architecture
Figure 1 shows the Oracle over GCNV iSCSI I/O path.
End-to-end performance depends on every layer in this stack. Oracle issues physical I/O to datafiles on XFS; the filesystem passes I/O to a single multipath device that aggregates parallel iSCSI sessions across two portals; GCNV serves the LUN from an ONTAP-mode pool. Bottlenecks in any layer—cache, path count, queue depth, or ALUA policy—will show up in the results if not addressed.
Figure 1 — Oracle over GCNV iSCSI I/O Path
All storage performance metrics in this article were collected at the Linux multipath device layer: dm-0. (dm-0 is the Linux multipath disk device that Oracle uses for datafile I/O. It is used to measure and to report the IOPS and latency GCNV actually delivered to the host)
Reporting at dm-0 ensures the numbers represent what the array actually delivered to the host, independent of Oracle buffer cache or Oracle Automatic Workload repository (AWR) block accounting.
Performance Environment
The test ran on a single Google Cloud VM attached to a 5 TiB GCNV volume over iSCSI. The configuration below reflects the environment used for all benchmark runs in this study.
|
Component |
Configuration |
|
Cloud Platform |
Google Cloud |
|
Storage |
Google Cloud NetApp Volumes |
|
Protocol |
iSCSI |
|
Database |
Oracle Database 23c |
|
VM Type |
*any GCE SKU |
|
Filesystem |
XFS |
|
SLOB Working Set |
2 TB |
|
Benchmark Tool |
SLOB |
|
Measurement Source |
Linux iostat |
* Use any SKU, with compute and Storage decoupled with GCNV enabling you to use smaller SKUs instead of bigger VM to drive high IO. We support all shapes, test with your VM instance SKU for production workload as required. The 2 TB SLOB working set is intentionally much larger than the benchmark buffer cache so that I/O is driven to the LUN.
Benchmark Methodology
The benchmark used steady-state Oracle SLOB runs to ensure results reflected storage performance rather than Oracle cache effects.
SLOB (Silly Little Oracle Benchmark) generates configurable random read and update traffic against Oracle datafiles. Each run lasted 30 minutes at a fixed concurrency level so that iostat and AWR snapshots capture steady-state behavior rather than ramp-up.
Methodology highlights:
- 2 TB SLOB working set
- Random 8 KB I/O profile
- 30-minute steady-state runs
- Storage-bound workload design
- Oracle AWR used
- Linux iostat used for storage metrics
- Storage metrics captured
Two workload profiles were tested:
- 100% Random Read
- 90% Read / 10% Update
The 100% read workload characterizes maximum random-read behavior. The 90% read / 10% update workload characterizes mixed OLTP-style random I/O.
Oracle AWR is essential to prove the database was doing physical I/O and waiting on storage, but AWR block counts and wait averages are not a substitute for array-level IOPS. Large buffer caches, read-ahead, and multi-block reads can make AWR numbers diverge sharply from what the disk subsystem actually served.
AWR remains valuable for validating Oracle behavior, but storage performance should be reported from the Linux storage device layer.
For this study, storage metrics were collected from:
iostat -xmt 1
AWR reports were used to confirm that Oracle was I/O wait-bound and not CPU-limited. Published IOPS and latency figures in the results section come from iostat, which is the authoritative source for storage performance. Use iostat on the multipath device for customer-facing storage claims; use AWR to document that Oracle was genuinely storage-bound during the test window.
How to tune performance?
Performance was not achieved by a single parameter change. The recipe stacks Oracle, filesystem, multipath, and iSCSI settings so the database can drive parallel I/O all the way to GCNV. Skipping a layer—for example, measuring an individual path—typically understates achievable throughput.
Oracle Configuration
Key Oracle settings:
- filesystemio_options=SETALL
- Small benchmark buffer cache: 192 MB
The objective was to keep the workload storage-bound rather than cache-bound.
The 192 MB buffer cache was used only for benchmark validation to reduce cache effects. It is not a production sizing recommendation.
Oracle cache and prefetch behavior were controlled for this benchmark using DBA-reviewed test-only settings. These settings are not general production recommendations and should not be applied outside a lab or proof-of-concept environment without Oracle DBA review.
filesystemio_options=SETALL enables direct and asynchronous I/O through the filesystem so Oracle does not add unnecessary buffering on top of the storage path.
Linux Configuration
The Oracle datafiles were placed on an XFS filesystem hosted on the iSCSI-backed multipath device.
XFS was chosen for mature Oracle-on-Linux support. Mount and log options were set to reduce metadata overhead during sustained random I/O.
Filesystem settings included:
- XFS
- noatime
- logbufs=8
- logbsize=32k
Multipath settings included:
- Device mapper multipath
- ALUA
- service-time path policy
The filesystem was mounted from:
dm-0
not from an individual iSCSI path.
iSCSI Configuration
iSCSI settings included:
|
Parameter |
Value |
|
node.session.nr_sessions |
4 |
|
node.session.queue_depth |
256 |
|
node.session.cmds_max |
1024 |
The host used multiple iSCSI sessions and parallel paths to expose sufficient queue depth to the Oracle workload.
Multiple sessions per portal increase the number of outstanding I/Os the initiator can send to the array. Queue depth and cmds_max were raised from defaults to avoid artificial throttling on the host side during peak load.
Benchmark Results
The following tables summarize peak steady-state results at the best operating point for each workload. IOPS and latency were measured during the 30-minute run window.
100% Random Read
This profile represents read-intensive OLTP, reporting, or scan-heavy workloads where writes are minimal. It establishes the upper bound for random read IOPS on this stack.
|
Metric |
Result |
|
Read IOPS |
~156,540 |
|
Latency |
0.401 ms |
Key Observation
The environment sustained approximately:
|
157K random read IOPS while maintaining sub-millisecond latency at the best operating point. |
At this operating point, read throughput approached roughly 1.1–1.4 GB/s depending on block size accounting, with latency still below one millisecond—suitable for latency-sensitive OLTP.
90% Read / 10% Update
This profile adds a realistic write fraction. Total IOPS includes both read and write operations at the multipath device, which is how mixed OLTP should be sized on the array.
|
Metric |
Result |
|
Read/Write IOPS |
~145,784 |
|
Latency |
0.377 ms |
Key Observation
The workload reached approximately:
|
146K total IOPS while maintaining sub-millisecond latency. |
As concurrency increased beyond the optimal operating point, latency continued to rise while throughput remained largely flat, indicating normal saturation behavior.
The saturation pattern is expected: once all paths and queues are fully utilized, adding sessions increases wait time more than IOPS. Capacity planning should target the knee of the curve—here, near 146K total IOPS at sub-ms latency—not the point where latency degrades.
Deployment Checklist
Use this checklist when reproducing the setup for Oracle workload for all your environment needs (dev/test to Production). Each item maps to a layer in Figure 1 or the tuning flow in Figure 2.
- Provision storage for the desired performance target.
- Present storage using iSCSI.
- Configure multiple iSCSI sessions.
- Tune queue depth appropriately.
- Enable Linux multipath and ALUA.
- Place Oracle datafiles on the multipath device.
- Use XFS with recommended mount options.
- Enable Oracle direct and asynchronous I/O.
- Keep the benchmark storage-bound.
- Capture both AWR and iostat.
- Report storage metrics
Conclusion
Oracle on Google Cloud NetApp Volumes over iSCSI delivered consistent high random I/O performance to lift & shift critical workloads.
The results show that a properly tuned Oracle-on-Linux stack can consume the parallelism GCNV exposes over iSCSI, with predictable latency at peak throughput for both pure-read and mixed OLTP-style workloads.
The tested configuration achieved:
- ~157K random read IOPS for 100% read workload
- ~146K total IOPS for a 90% read / 10% update workload
- Sub-millisecond latency at peak throughput
The most important contributors to performance were:
- proper iSCSI session tuning,
- Linux multipath optimization,
- Oracle direct I/O configuration,
- storage-bound workload design,
- device-layer performance measurement.
These results demonstrate that Google Cloud NetApp Volumes can support demanding Oracle database workloads when the Oracle, Linux, and storage stack are tuned.
When implementing Oracle on GCNV, enterprises should conduct performance validation specific to their application needs: ensuring the database issues physical I/O by over-sizing the data set relative to the cache and optimizing the end-to-end I/O stack.