NetApp Cloud Insights simplifies observation of on-prem and cloud infrastructure and provides analytics and troubleshooting capabilities to help solve the most complex problems. It’s pretty mind-boggling the amount of data Cloud Insights collects and serves up in a variety of dashboards, widgets, and metric queries. Fortunately, this is all simplified significantly with a large number of ready-made dashboards that help to zero in on specific types of problem areas and categories of data. While Cloud Insights is a heterogeneous tool designed to collect data from a wide range of devices, there is a library of templates, called ONTAP Essentials, that makes it easy for NetApp customers to get started quickly.
In this article we will provide an overview of how to get started with NetApp Cloud Insights and provide a handful of examples that provide useful solutions to real-world problems.
Cloud Insights is available in both a Basic free edition (for NetApp customers) and Premium edition (30-day free trial available). To get started, you will need to first sign up at the NetApp Cloud Insights website.
Overview and Architecture
It starts with collecting data from your data center environment, on-premises or in the cloud, and sending that data to Cloud Insights. This is done by installing an Acqusition Unit on a Windows or Linux system and configuring data collectors to pull data from the devices in your environment. Cloud Insights also supports open-source Telegraf agents that are installed on servers (or VMs) for collecting and reporting metrics.
The assets in Cloud Insights can be tagged with annotations that provide a method of organizing and classifying data. Dashboard can be created using a wide variety of widgets for displaying the data and Metric Queries can be created for detailed tabular views of data.
With all of this in mind let’s dive in and go over the configuration steps needed to get started.
Install Acquisition Unit and configure data collectors
The first step in configuring Cloud Insights is to install an Acquisition Unit (AU) on a Windows or Linux VM. The AU sits behind your firewall, VPC, or VNet and provides data collection capabilities for assets such as ONTAP storage systems and VMware vSphere clusters. For our use case, we used a Windows virtual machine for the installation.
1. Log into Cloud Insights from the system you will be installing the AU and navigate to Observability > Collectors > Acquisition Units. Click on the button to install a new Acquisition Unit. Choose your operating system and download the installer.
2. Click on the Copy Access Key button and paste the key into the installer when prompted.
3. Once the install is complete, navigate to Observability > Collectors > Data Collectors and press the button to install a new Data Collector. From here you can search for hosts, services, or infrastructure devices to add. First, we’ll add our ONTAP storage systems by searching for ONTAP, clicking on ONTAP Data
Management Software, and providing our AU and storage system credentials.
4. Next, add a vSphere cluster to monitor. Search for vSphere and click on VMware vSphere. Select the correct AU and add the vCenter credentials. Optionally, expand Advanced Configuration for more options.
View topology details with Cloud Insights
Now that we have our assets discovered, there are multiple ways to organize and decipher data related to our environment. In this example we’ll take a look at how to navigate relationships between assets and view topology information. There are quite a few ways to browse information in Cloud Insights, so this is just one possible method.
1. Start by navigating to Observability > Collectors > Acquisition Units and click on the AU for your site or data center. Then click on a vSphere cluster.
2. From here click on one of the ESXi hosts and then View Topology to see a pictorial view of the host’s connectivity to ONTAP storage systems.
3. From the home page of the ESXi host you can view details about the hosts including a host summary, performance metrics, correlated assets, connectivity information and virtual machines.
View information on Dashboards
Dashboards provide a highly customizable view of infrastructure data using a variety of widgets. A blend of widgets can provide a flexible way to visualize and interpret infrastructure data to aid in efficient troubleshooting and optimization.
There is a lot of useful information available on creating custom dashboards. However, there are also many ready-made dashboards in the dashboard gallery that can be imported and customized to suit your needs. In this example we’ll show how to import a dashboard from the gallery.
1. To add a dashboard from the gallery, navigate to Observability > Explore > All Dashboards and click on the + From Gallery button to view the list of available dashboards.
2. Select a dashboard from the list and click on Add Dashboards.
3. From the Dashboard Groups list click on My Dashboards and then select a dashboard to view it.
4. A dashboard is made up of multiple widgets that can be edited, resized, or moved within the page. Note the Edit button in the upper right corner of the page. Click this and access the menu for each widget to edit its attributes. You can also drag widgets to move them and grab the lower right corner to resize them.
Use correlation to locate noisy neighbors
Now that we have the basics out of the way we can begin to use Cloud Insights to make some important discoveries about the workloads running on our vSphere cluster. Cloud Insights uses information from multiple data collectors to understand end-to-end topologies and correlate assets.
In this next example we’ll start with a dashboard view and drill down to a VM running a high utilization workload. From there we’ll look for other VM’s that might be contending for resources. Noisy neighbors.
Note: We will be filtering the assets in our dashboard using annotations that have been previously added to our assets. This will be covered in a separate section at the end of this article.
1. We’ll start with a dashboard from the gallery called VMware Admin – Where do I have VM Latency?. Use a filter at the top by using an annotation for the Data Center field.
2. Select the VM that you want to investigate, to view its details.
3. Here we can see peer VMs that are displaying high utilization and might warrant further investigation as candidates for remediation.
4. Finally, we can view the details of the neighbor and look for information that might point to a problem or indicate that the offender should be relocated.
Identify opportunities to right size VMs
Identifying over and under-utilized resources is often a challenge as it’s not easy to capture and analyze the data required to make informed decisions. This is another area where Cloud Insights excels by allowing customization of the data in ways that highlight the information critical to making impactful business decisions.
In these examples we'll look at two dashboards, available from the dashboard gallery, that help to identify VMs that are candidates for right sizing.
The first dashboard is called VMware Admin – Where are opportunities to right size?. This dashboard shows multiple graphs that highlight top and bottom utilization.
In this case we used a wildcard to filter all of our ESXi hosts.
Another useful dashboard for this purpose is called VMware Admin – Where can I potentially reclaim waste? In this case we can view the capacity utilization of VM’s that are currently powered off.
Using Annotation Rules to tag assets
One thing we can do, to make it easier to filter and group assets, is to add annotations. We can do this individually or we can create an Annotation Rule to tag multiple assets simultaneously. This feature was used, to filter the assets seen on dashboards, previously in this article.
In this example we will add annotations to all our virtual machine assets so we can filter them by “Data Center”. To do this we will first create a Query to obtain a list of all the application VM’s residing within specific datastores.
1.Navigate to Observability > Explore > + New Metric Query, add a name for the query at the top, and select VirtualMachine from the first drop down menu.
2. For Filter by Attribute select datastore.name and add the names of the datastores containing the virtual machines.
3. Finally, we will select Group By Virtual Machine.
Next, we’ll create an Annotation Rule to add the annotation to the assets specified by the query.
1. Navigate to Observability > Enrich > Annotation Rules and click the button to add a new rule.
2. Add a name for the rule. Select the Query to which the rule will be applied and select the Annotation field that you want to edit. Add the value for the annotation to be applied and click on Save.
3. Finally, in the upper right-hand corner of the Annotation Rules page, press the button to Run all Rules.
Conclusion
There’s almost no limit to the creative ways you can slice and dice data in Cloud Insights. In this article we only scratched the surface of what is possible. There is an entire category in Cloud Insights devoted to ONTAP Essentials that features a large selection of pre-built dashboards and metric queries for ONTAP storage systems. There are sections for Kubernetes and for Workload Security that allow you to perform forensic analysis of data usage. While this article provides a good primer, be sure to check out the following resources for more information.
Feature tutorials (with demo videos)
NetApp Cloud Insights: A New Way to Monitor – How To Guide
How to Build Custom Dashboards to Monitor Specific VMs in Cloud Insights - Blog
