When companies need financial management and planning, they can turn to AdvisorEngine. This US-based wealth management service provider offers tools and services that help financial advisors assist clients manage portfolios and stay ahead of the curve.
But AdvisorEngine was having issues around the performance of the massive Microsoft SQL Server database estate supporting a line-of-business software as a service (SaaS) application. After a deep analysis of the system’s architecture, the company realized that modernizing the storage layer would make a big difference.
AdvisorEngine turned to Amazon FSx for NetApp ONTAP (FSx for ONTAP) to do that. In this post I’ll explain how FSx for ONTAP addressed AdvisorEngine’s challenges and the added benefits it provided in the process, including a sixfold performance improvement, a 50% cost reduction, and much more.
Here’s what we’ll cover:
- AdvisorEngine: A major fintech company with a massive SQL Server deployment
- This architecture presented some challenges
- How FSx for ONTAP solved AdvisorEngine’s database challenges
- What is FSx for ONTAP?
- AdvisorEngine saw a vast improvement using FSx for ONTAP
- The next step? Automating FSx for ONTAP using BlueXP workload factory
- Conclusion
AdvisorEngine: A major fintech company with a massive SQL Server deployment
AdvisorEngine provides a range of wealth management services and technologies—including portfolio management, digital onboarding, customer relationship management (CRM) systems, financial planning tools, and more. Together, the company oversees client assets totaling over half a trillion dollars.
One of these products is a customer-facing SaaS application that runs on Amazon Web Services (AWS). The application’s architecture included around 1,000 self-managed SQL Server databases running between two Always-On Failover Cluster Instances (FCIs) on AWS.
Here’s what the architecture AdvisorEngine had been using looked like:
- Two self-managed SQL Server Always-On FCIs, each with three nodes: one primary node, one node for high availability (HA) in a different Availability Zone (AZ), and one node for disaster recovery (DR) located in a separate AWS Region.
- Around 1,000 SQL Server databases split between the two FCIs.
- Backup via SQL Server’s internal backup process to Windows File Server.
- A third-party replication engine responsible for synchronous replication between the Elastic Block Store (EBS) storage volumes for the FCI nodes, and asynchronous data replication to the DR nodes.
This architecture presented some challenges
The main challenge was that the application was experiencing performance issues. There were drop-offs and spikes throughout the day. Performance was simply not consistent, and for a line-of-business application such as this, that was simply not acceptable.
AdvisorEngine was also relying on a third-party vendor to replicate data for disaster recovery (DR) purposes. That tool was not only affecting the performance of the production environments, it was also not meeting AdvisorEngine’s recovery time objective and recovery point objective targets, and data was frequently out of sync.
With the business growing and the application adding new users, a new solution had to be found. The first option—to expand the existing resources to meet the growing demand—would have been prohibitively expensive. And it wouldn’t solve the difficulty with the DR system.
The team at AdvisorEngine did a deep dive into the database and analyzed the entire architecture. They discovered that the file system performance was lagging because of the replication service’s processes. As such, the file system couldn’t handle the scale of the SQL Server workload.
The following images show the kind of performance drop-offs that AdvisorEngine was experiencing. The total transactions per minute peaked at 10,194 transactions per minute (TPM):
How FSx for ONTAP solved AdvisorEngine’s database challenges
What is FSx for ONTAP?
FSx for ONTAP is an AWS storage service that delivers NetApp ONTAP functionality natively on AWS. It provides a highly available file system and block storage that can scale to meet even the most demanding workloads, from vast GenAI knowledge bases to massive SQL Server deployments such as AdvisorEngine’s. Learn more about FSx for ONTAP here.
AdvisorEngine looked at several different storage services on AWS to solve the problem. Compared with the other services, FSx for ONTAP was less expensive and delivered a more performant experience. Even though AdvisorEngine and its IT teams had no previous hands-on knowledge with NetApp technology, FSx for ONTAP was the obvious choice as it provided a like-for-like experience in the cloud as on premises. FSx for ONTAP was also the only service on AWS that was able to provide block storage that could be shared across Availability Zones (AZs).
That’s an important point for this story: Even without previous experience with NetApp or ONTAP technology, the IT team was able to quickly understand its workflows and benefit from its value and efficiency.
Here’s what the new architecture with FSx for ONTAP looks like:
The architecture includes:
- An FSx for ONTAP multi-AZ high-availability (HA) pair stored in a primary AWS Region
- Another FSx for ONTAP multi-AZ HA pair which is stored in a different AWS Region and used for DR
- Asynchronous replication via SnapMirror® between the primary and DR FSx for ONTAP instances
- Backup via a third-party backup process to Amazon Simple Storage Service (Amazon S3) and later, Amazon S3 Glacier
- Virtual private cloud (VPC) peering between the primary and the DR FSx for ONTAP environments to a Windows Server FCI Quorum file share
AdvisorEngine saw a vast improvement using FSx for ONTAP
AdvisorEngine was able to accomplish major improvements with its application and its underlying SQL Server deployment by switching to FSx for ONTAP:
Performance improved sixfold
AdvisorEngine saw the performance of the application’s database increase sixfold, all while using a similar amount of Amazon Elastic Compute Cloud (Amazon EC2) instances. A major contributing factor to the increase in performance was that NetApp SnapMirror was now being used for replication as opposed to the inefficient third-party tool.
The diagram below shows how FSx for ONTAP has performed versus the peaks and valleys of the old architecture. Where the old environment was delivering sporadically, FSx for ONTAP is delivering consistently:
Costs came down by 50%
AdvisorEngine was able to significantly reduce the costs of running the SQL Server deployment: AdvisorEngine’s costs are down 50% using FSx for ONTAP.
There was also a 75% cost avoidance by switching to FSx for ONTAP. That’s because scaling up the previous storage solution’s bandwidth would have required the use of larger and more expensive Amazon EC2 instance types. Because FSx for ONTAP uses network performance, there was no need to scale up compute that way.
Another part of the savings comes from the new DR setup (see more on that below). That replication takes place between FSX for ONTAP instances via SnapMirror. This is a highly efficient technology that updates the data incrementally. That reduces the time and costs of replicating data to the DR copy, all without the added expense of using a third-party technology.
Learn more about how to lower your AWS database costs with FSx for ONTAP.
A high level of resilience for business continuity
FSx for ONTAP is highly available by default, with each instance consisting of two FSx for ONTAP nodes that are kept in constant sync. AdvisorEngine is using the deployment option that places the two FSx for ONTAP nodes in separate AZs, providing the highest possible availability. Even if a whole AZ goes down, the system will seamlessly transfer all operations to the healthy node, preserving business continuity.
Note that FSx for ONTAP is the only AWS storage service that can share block storage across AZs out of the box, with no need for additional instances or costs.
Better data protection, without a third-party tool for DR
FSx for ONTAP provides AdvisorEngine with a way to protect its data across AWS Regions with its built-in cross-Region replication capabilities.
The main technology that makes this possible is NetApp SnapMirror. Because SnapMirror uses highly efficient incremental replication, it works very quickly and prevents any of the performance issues the previous replication technology caused.
This setup also saves on compute spending because DR is part of the storage layer and not related to compute, meaning AdvisorEngine doesn’t have to constantly write from live resources.
For additional protection, AdvisorEngine is backing up data on an Amazon S3 bucket. A policy later tiers old backups to Amazon S3 Glacier. These backups are created from the DR copy which is a mirror of the production data, so there is absolutely no impact on the performance of the live environment.
Here’s what the DR and backup architecture with FSx for ONTAP looks like:
Easy-to-create, cost-efficient database clone copies
By leveraging NetApp FlexClone® technology, FSx for ONTAP enables AdvisorEngine to create instant and lightweight clone copies of database environments. These lightweight environments take up a bare minimum of storage space, which means they don’t incur the additional storage charges that other technologies face for creating full copies. Plus, creating clones is non-disruptive for the production environment.
AdvisorEngine uses these copies for various development needs, including validation, QA, and testing. Because they take only minutes to create, the entire development cycle is now much faster. Before using FSx for ONTAP, AdvisorEngine couldn’t regularly run staging environments. FSx for ONTAP and its cloning capabilities changed all that, which improved their continuous integration and delivery (CI/CD) processes.
Here you can see how the clone-based staging environments fit into the overall architecture:
The next step? Automating FSx for ONTAP using BlueXP workload factory
AdvisorEngine has a unique opportunity to make their FSx for ONTAP deployment faster and safer using BlueXP™ workload factory for AWS.
Workload factory is a free service within the BlueXP GUI that makes it possible to build workloads—including database environments—right the first time thanks to:
- Automated deployment: Workload factory makes it possible to automatically deploy the infrastructure needed for new and existing databases. The FSx for ONTAP storage layer and EC2 instances are optimized for database workload requirements.
- Continuous optimization: Keeping tabs on the database is an ongoing and automatic process, and workload factory monitors and reports to make sure the architecture is optimized and aligned with best practices. Read more about how continuous optimization works.
- Automating IaC: Workload factory provides pre-generated infrastructure-as-code (IaC) snippets for APIs, Terraform, and CloudFormation. Codebox, where workload factory delivers the snippets, provides a resource to support ongoing day-1 and day-2 operations.
- Faster onboarding: For users who are just getting started with FSx for ONTAP, workload factory helps onboard your database to the new storage environment. It starts by assessing your current storage system to provide total cost of ownership (TCO) calculation and deployment recommendations, to help you optimize your environment’s configuration.
- Best-practice guardrails: Workload factory provides best-practice recommendations that are automatically applied to the environments it provisions. These keep your database running optimally from the very start.
Learn more about enhancing database operations with BlueXP workload factory for AWS or try it now.
Conclusion
FSx for ONTAP block storage gave AdvisorEngine’s SQL Server deployment six times more performance, 50% lower costs, enhanced CI/CD pipelines, and simplified and enhanced resiliency.
Want to learn more about FSx for ONTAP and this architecture? Visit these resources:
