NetApp Cloud Series
The latest buzz is all about the cloud. NetApp has been working hard to bring you storage solutions to meet your private, public, and hybrid cloud needs. Check out the first two articles in our NetApp Cloud series to learn more and keep an eye on the Tech OnTap® newsletter for future cloud articles.
- Storage Services for the Private Cloud (Features WFA)
- OpenStack
Software-Defined Storage Must Address Enterprise and Cloud
The term software-defined storage is often restricted to storage software running on commodity hardware. However, this vision isn't broad enough for a world in which the majority of enterprise application data is stored on traditional external storage systems. Additionally, in the not-too-distant future, a significant amount of enterprise data will move out of on-site data centers and into off-premises clouds. This means that your software-defined storage strategy must be broad enough to encompass both traditional enterprise storage systems and a rapidly evolving ecosystem of cloud service providers.
At NetApp, we view software-defined storage (SDS) as a key enabler of the software-defined data center (SDDC) and the cloud. You have to be able to automate storage provisioning and storage services so that users and applications can get immediate access without having to know details about where the storage is or what it looks like. Also, the same SDS capabilities must be available across a variety of platforms, including both traditional storage devices and commodity hardware.
NetApp SDS capabilities are delivered through a combination of the NetApp® Data ONTAP® operating environment and OnCommand® management tools. Data ONTAP runs on NetApp FAS hardware; is part of the FlexPod® converged infrastructure; virtualizes EMC, HDS, and NetApp E-Series arrays through NetApp FlexArray; and runs as a virtual storage appliance on commodity server hardware. There are OnCommand management tools that operate in all of these environments as well.
In a software-defined environment, management software becomes critical to storage success, and it must deliver high levels of automation and integration as well as greater intelligence and flexibility. We continue to evolve our OnCommand tools to do just that.
This article examines the latest OnCommand developments including how OnCommand tools contribute to SDS and cloud success.
Figure 1) The OnCommand management portfolio.
OnCommand Overview
The OnCommand management portfolio provides advanced analytics and fully integrated monitoring of capacity, availability, performance, and protection plus automation to facilitate software-based service delivery.
The portfolio includes five tools specifically for the management of NetApp storage:
- OnCommand System Manager. Device-level management for individual storage systems or clusters. Ideal for one-off and nonrepeatable management and configuration tasks.
- OnCommand Unified Manager. Monitor and alert on NetApp storage availability, capacity, performance, and data protection relationships. Ideal for managing NetApp storage at scale (that is, when you need to manage more than a small number of NetApp systems or clusters).
- OnCommand Performance Manager. Brand-new tool for performance monitoring and troubleshooting for clustered Data ONTAP version 8.2 and later. Integrates with Unified Manager to provide a single point of control over NetApp storage.
- OnCommand Workflow Automation. Automation and delegation of all repeatable storage management and storage service tasks. WFA also integrates with orchestration systems to enable automation across the data center for IT self-service and the cloud.
- OnCommand Balance. Ideal for virtualized environments, Balance provides performance analytics across storage, networks, and virtual machines; predicts performance issues; and facilitates troubleshooting.
In addition, a sixth product is specifically tailored for multivendor storage resource management (SRM):
- OnCommand Insight. Designed to meet the needs of the C-level executive in addition to storage directors, architects, and administrators, Insight helps you manage complex enterprises that need visibility beyond storage to optimize capacity and performance. It also addresses needs for advanced reporting on cost accountability (including chargeback and showback).
At NetApp, we've been moving fast to enhance these tools to further address the requirements of software-defined IT and the cloud. OnCommand System Manager was recently updated; the new 3.1 version reached general availability in April. As noted above, Performance Manager is brand new—it was just released at the end of April—and we also released updates to Unified Manager, Workflow Automation, and Insight.
- Unified Manager 6.1 includes integration with Performance Manager, monitoring enhancements, and the ability to manage by events.
- Workflow Automation 2.2 includes new data protection workflows and enhanced integration with Unified Manager to automate data protection.
- OnCommand Insight 7.0 includes a variety of scalability (300% performance increase) and usability improvements as well as the ability to draw from additional data sources.
In the remainder of this article we introduce Performance Manager and describe how Workflow Automation can be used to enable software-defined storage services.
Introducing OnCommand Performance Manager
In order to operate storage effectively in software-defined storage and cloud environments, you need to be able to identify, track, and act on storage performance issues as they arise. OnCommand Performance Manager is built from the ground up to help you achieve this goal. It supports storage systems running NetApp clustered Data ONTAP 8.2 and later and provides comprehensive performance monitoring and troubleshooting using new automated analytics. Real-time analysis with recommended solutions dramatically cuts the time to diagnose and fix problems in scale-out storage. Our previous-generation product, Performance Advisor, remains available for customers running 7-Mode and using Unified Manager version 5.x.
OnCommand Performance Manager tracks metrics for the following components within a scale-out FAS cluster, all of which can affect cluster performance:
- Network
- Network processing
- Policy group limits
- Cluster interconnect
- Data processing
- Aggregate
Where Performance Manager Fits
Given that both OnCommand Balance and OnCommand Insight offer performance-monitoring capabilities, a common question is where Performance Manager fits in the OnCommand portfolio.
Simply put, Performance Manager is the minimum baseline for performance monitoring. It monitors and troubleshoots deep into clustered Data ONTAP and it's available as a free download as part of your NetApp FAS purchase.
Balance provides troubleshooting and optimization at the next level in a server virtualization environment. It monitors clustered Data ONTAP, virtual machines, and hosts. It correlates data across all three and provides guidance.
Insight is a full storage resource management solution for large, complex multivendor storage in virtual and physical environments. Consider adding Insight if you have storage from such vendors as EMC, HP, IBM, and Dell.
Figure 2) Where OnCommand Performance Manager fits in the NetApp performance monitoring portfolio.
Advanced Analytics
Performance Manager is designed to detect performance issues automatically, determine the root cause, generate alerts, and provide advice for resolution. This includes identifying best practices and showing you steps to take to fix a problem. This approach is much different than that used by tools that just provide a bunch of metrics in spreadsheet format and leave it to you to figure out the root of the problem.
Performance Manager was built by the same engineering team that created OnCommand Balance. They used the analytics expertise developed for Balance and proven intellectual property (IP) to cut out the noise and provide only the information you need to know, showing you the root cause of each problem with simple color coding. (See Figure 3.)
Figure 3) Performance Manager uses advanced analytics and color coding to let you see the source of the problem.
The great thing about this is that you save a ton of time by not having to manage user-defined thresholds on each object. With dynamic thresholds, you get a continuous performance baseline that adjusts to workload changes and correlates affected volumes to resource contention.
Our previous-generation product, Performance Advisor, remains available for customers running 7-Mode and using Unified Manager version 5.x.
Depending on the data it is tracking, Performance Manager can keep data monitoring sets for up to 90 days, providing a comprehensive view of performance over time. This historic record can help you spot trends and head off impending performance issues before they affect storage performance.
Getting Started with Performance Manager
Performance Manager is available for download (no license required) from support.netapp.com. (Login required.) It runs either as a standalone product or integrates with Unified Manager version 6.1, which is also a free download available without a license. The Unified Manager dashboard has been revamped to include a new performance pillar. This pillar presents consolidated performance activity on managed clustered Data ONTAP storage systems.
The integration provides loose coupling for the initial release. This means that, although you can see summarized snapshot information on performance activity via Unified Manager, you need to cross-launch the Performance Manager web interface to view details. Performance Manager will be integrated into Unified Manager in the Unified Manager release.
The Performance Manager product is installed as a virtual appliance in a VMware® environment and is easily configured using common web browsers. It works with clustered Data ONTAP systems running version 8.2 and later by collecting and analyzing relevant metrics from each cluster under management.
To learn more about Performance Manager, check out this getting started video from NetApp University.
Workflow Automation
OnCommand Workflow Automation (WFA) is a key enabler of software-defined storage and makes it possible to integrate NetApp storage into almost any software-defined or cloud environment.
WFA is a highly flexible automation framework that allows you to automate almost any storage process—from the simplest workflow to the most complex. The WFA toolset provides a consistent way to define and execute common and/or repeatable processes that you might otherwise perform from the command line, from a script, or using System Manager.
With WFA you can define workflows to create volumes, LUNs, qtrees, shares, and so on, and give each the proper level of data protection, QoS, and storage efficiency automatically. Storage resources are dynamically allocated according to your plan, and you can make calls to other services such as those from VMware.
You can think of WFA as an orchestration tool concerned chiefly with NetApp storage and integrating closely with other orchestration tools through REST APIs. A previous Tech OnTap article on the private cloud covered the capabilities of Workflow Automation in more detail.
Figure 4) Workflow Automation enables NetApp storage services to be integrated with a wide variety of data center orchestration frameworks.
What's New in Workflow Automation 2.2?
Version 2.2 of Workflow Automation includes a variety of new and enhanced features, including:
- New sample workflows for SnapMirror® and SnapVault® technologies (certified)
- Content versioning so that WFA entities such as commands and workflows can be versioned for change management
- Support for new data sources: Unified Manager 6.1 and VMware vCenter™ Server 5.5
- Integration with Unified Manager 6.1 to provide automated data protection with SnapVault
- Passing parameter between commands—PowerShell™ cmdlets and Perl functions are included in WFA and you can use them to exchange information that is obtained during workflow execution between commands
- New platform support: IBM
Enabling SDS with NetApp and VMware
Over the past several months, one of us (Jack) has been busy designing a complete SDS solution using VMware vCloud® Automation Center (vCAC) 6.0 and OnCommand Workflow Automation.
vCenter Orchestrator (vCO) makes calls through REST to WFA to activate storage creation. In vCAC 6 we created a storage service catalog; advanced services call vCO workflows and grant the appropriate permissions to the storage service catalog. When a user clicks on "Request" for the storage service he or she wants and inputs a few parameters for the service (vCenter IP address, storage type, storage size, and so on), NetApp storage is automatically provisioned.
Details of the whole process are being documented on Jack's blog. Check out the overview video and find links to all the related posts here. This is still a work in progress, so you may have to check back to read to the end of the series. Also note that the blog entries up to this point reference WFA 2.1 and Unified Manager 6.0. The procedures will work with the updated software versions described in this article.
You can read more about the tools and processes involved in this type of orchestration in TR-4217: Automating and Orchestrating the Software-Defined Data Center with VMware and OnCommand Workflow Automation. (Once again this TR references older versions of software but everything should apply to the later versions as well.) Two additional technical reports are currently under development and should be available soon. Keep an eye on the WFA community for information on those.
Getting Started with Workflow Automation
Workflow Automation is a value-added product (no license required) that is part of the OnCommand management software that you are entitled to with each NetApp FAS purchase; it's available for download from support.netapp.com. (Login required.) NetApp recommends installing WFA on a dedicated server (or VM) running Windows® along with the NetApp Data ONTAP Windows PowerShell toolkit module. In order to fully utilize the capabilities of WFA you need to have installed OnCommand Unified Manager in your environment (it serves as a data source).
In addition to the WFA software, a variety of "Pirate Packs" are available that package together NetApp developed WFA workflows for various use cases. In particular, you may want to check out the Pirate Pack OnCommand Workflow Automation package for VMware vCenter Orchestrator.
In addition, a number of Pirate Packs can be found in the "Featured Content" section of the WFA community. You can also browse templates, examples, and videos; search content for more Pirate Packs; and browse discussions.
Conclusion
NetApp OnCommand management tools give you the capabilities you need to run your storage environment more effectively. Software-defined storage capabilities smooth operations and form a foundation for advanced automation and the cloud. The new Performance Manager gives you the tools to identify and eliminate performance bottlenecks. This becomes critical as your storage continues to scale and you begin to operate your data center to deliver defined services and guaranteed service levels. WFA allows you to automate the processes you use every day and integrate those processes with third-party solutions and the cloud in a uniform way using open REST interfaces.
