article VSC: The Essential Tool for VMware on NetApp in Tech ONTAP Articles

VSC: The Essential Tool for VMware on NetApp

Tech_OnTap — Thu, 14 Mar 2013 13:44:46 GMT

More from Nick

Like what you learned in this article? You can find a lot more from Nick at datacenterdude.com. He blogs regularly on VMware and related topics, posts detailed how-to videos, and hosts a regular podcast that you can listen to on the site or subscribe to on iTunes.

You can also subscribe to Nick's YouTube channel: NickONTAP.

Nick Howell
Virtualization Solutions Architect

If you are using VMware® with NetApp® storage, there's one tool you absolutely should have: the NetApp Virtual Storage Console (VSC) for VMware vSphere®. VSC is a vCenter™ plug-in that simplifies storage management, improves efficiencies, enhances availability, and reduces storage costs whether you use SAN or NAS. VSC gives administrators working in vCenter a window into the storage domain—and the tools to manage virtual server and desktop environments running on NetApp storage without leaving the vSphere client.

With the new features we've added—and because VSC is free from the NetApp Support site—there's really no reason not to try it out.

The VSC plug-in is organized into four functional areas:

Monitoring & Host Configuration
Provisioning & Cloning
Optimization & Migration
Backup & Recovery

The currently released version of VSC is 4.1, and version 4.2 is now in beta. A tremendous amount of work went into VSC to deliver the payload of capabilities I describe in this article. The enhancements in the soon to be released 4.2 version really tie a nice bow on the entire 4.x development, and we're all very happy with what we'll deliver.

What's New in VSC 4.2?

First, I describe what's coming in VSC 4.2. If you are a current VSC user, this may be all the information you need. If you are new to VSC, you'll still find this section informative, since it applies to everything I discuss later.

VSC 4.2 includes:

Support for the clustered Data ONTAP® architecture
vCenter role-based access control (RBAC)
Data ONTAP shared credentials

Clustered Data ONTAP

For clustered Data ONTAP, our team put a lot of effort into enabling the user experience to change as little as possible for those who are already comfortable using VSC to manage NetApp storage. We've got full support for clustered Data ONTAP in all areas, and all wizards and workflows essentially look and feel the same for both 7-Mode and clustered Data ONTAP. In most cases, there are just a few additional items added for clustered systems. We want your experience when using VSC to remain familiar so that the transition to clustered Data ONTAP—when you take that step—is as painless as possible.

vCenter Role-Based Access Control

A lot of VSC users have been asking for enhanced RBAC for some time. NetApp spent the last year talking to users and working out exactly what was needed. We've added privilege controls for almost every task in every functional area, so that large enterprises and service providers can create administrative accounts that are strictly limited to particular functions.

Figure 1) VSC provides fine-grained role-based access control that makes it easy to define the specific tasks accessible to each user account.

We've also created a number of "canned roles" that you can use as a starting point for creating custom VSC accounts:

VSC Administrator
VSC Read-Only
VSC Provision
VSC Clone
VSC Scan/Migrate
VSC Backup
VSC Restore

Be sure and clone these sample templates before you use them. If you use and modify them without cloning, you can run into permissions problems if NetApp needs to modify these canned roles in the future.

We've also made it possible to hide the VSC entirely from accounts that should not have access to it. In addition, we provide a Read-Only mode that allows users to see what's happening without being able to change anything.

The vCenter-based RBAC features we included in 4.2 are intended for data centers that want to get more and more granular with their "who can do what to which objects" strategy. For the IT generalists out there who are responsible for managing both storage AND VMware, you can continue to use VSC to access and control everything you could before with one small change (adding the View privilege).

Data ONTAP Shared Credentials

VSC was created in part by pulling together functionality from a number of different tools. As a result, some parts of the tool maintained separate lists of hosts and storage controllers. With version 4.2 the entire tool—including Backup and Recovery, which was the last holdout—shares a single, centrally managed list. For those upgrading from an earlier version, please be aware that there are some limited circumstances in which permissions may need to be adjusted for backup jobs to run normally if they were running previously under a special user account.

I describe this point in more detail and also provide more information on RBAC in a recent blog post.

Understanding the Virtual Storage Console

Here I describe the key features of each of the VSC functional areas. If you're interested in learning more about a particular area, I made some pretty detailed walk-through videos for each area. These apply to VSC 4.1, but almost everything you will see is still applicable, and they show VSC in action as I explain many of the key features in each section.

Monitoring and Host Configuration

Monitoring and Host Configuration includes functionality that goes all the way back to VSC 1.0. It performs automatic discovery and shows you a complete list of all your NetApp storage controllers and associated ESXi™ hosts. (See Figure 1.) When initially setting up an environment, you'll spend a lot of time in this module getting your hosts and storage configured.

Figure 2) The main screen for Monitoring and Host Configuration lets you optimize host configuration and drill down to get details on NetApp storage.

Tuning for best practices. To achieve the best results with NetApp storage, a number of parameters on each ESXi host have to be tuned. Almost 70% of the calls that come into NetApp Support related to VMware are solved by applying these settings.

Without VSC, setting these parameters is a manual process that has to be done on each host. This is a hassle if you have a number of hosts to set up, and of course there is always the risk of making mistakes.

With VSC, you can apply these best practices to an ESXi host in a few seconds with a couple of mouse clicks. There's no heavy lifting and you can enable your ESXi host to be optimally configured per NetApp best practices, and verify that all hosts in your vSphere environment are standardized.

Storage management options. On the storage side, you can perform a variety of tasks including adding, removing, and configuring controllers and viewing the hosts connected to NetApp storage. You can also drill down into storage details. This can help VMware admins and storage admins better coordinate their efforts, breaking down the barriers between the two and allowing them to have more meaningful conversations with one another when issues arise.

Provisioning and Cloning

Provisioning and Cloning includes a variety of capabilities for managing datastores and virtual machines:

Create, resize, and delete datastores
Deduplicate datastores
Clone virtual machines
Apply guest customization specifications to virtual machines
Monitor storage savings
Redeploy virtual machines from a baseline image
Import virtual machines into Virtual Desktop Infrastructure connection brokers and management tools

The functions in this area let you take full advantage of many of the special features of NetApp storage, including FlexClone® volumes, deduplication, and thin provisioning.

Rapid datastore provisioning. One of the biggest—and possibly the most overlooked—value-add feature of VSC is its capability to rapidly create a datastore. For example, suppose you have a cluster that contains 20 ESXi hosts and you need a new NFS datastore. You simply click Create datastore and enter the size, the name, and the disk pool to use, and you are done. VSC creates the volume on the appropriate storage, exports it, and mounts the datastore on all hosts in the cluster with everything configured according to NetApp and VMware best practices. The whole process takes a matter of minutes. Performing the same thing as a manual process within vCenter, by comparison, can take several hours, with lots of opportunities for error. Although the details are slightly different, the same process works for creating a SAN datastore as well.

Figure 3) Provisioning a new datastore with VSC requires only that you enter the datastore size, the name, the NetApp aggregate to be used, plus thin-provisioning parameters if thin provisioning is desired.

Space-efficient cloning. VSC cloning capabilities use NetApp FlexClone technology to efficiently clone existing datastores and virtual machines while using a fraction of the storage that would normally be required. This can be done en masse and it's fully integrated with the latest versions of both VMware View™ and Citrix XenDesktop, so you can clone and import thousands of new virtual machines directly into connection brokers.

You can easily create a standard desktop and then use rapid cloning to clone that desktop thousands of times, register the resulting virtual machines with vSphere and your connection broker, and boot them all in a fraction of the time it takes using traditional methods.

Deduplication. Most virtual environments have lots of duplication. For example, a given datastore may have many copies of the same operating systems and application files. Another great VSC feature is the ability to manage NetApp deduplication and recover a lot of that storage capacity. With a few mouse clicks, you can enable deduplication on a datastore and then monitor how much storage space you're saving, all from within the vSphere client.

Optimization and Migration

Correcting misaligned VMs. Misalignment is quite likely the biggest and most misunderstood problem in virtual environments, affecting all virtualization platforms and all storage platforms. This problem occurs because, by default, many older guest operating systems, including Windows® 2000 and 2003 and various Linux® distributions, start the first primary partition at sector (logical block) 63. Unless you take the right steps to counteract it, this behavior leads to misalignment because the partition does not begin at a block boundary. As a result, every time the virtual machine wants to read a block, two blocks have to be read from the underlying LUN. (Note that this problem is fixed in newer guest operating systems and hypervisors such as Windows 7, Windows 2008, Red Hat 6, and vSphere 5, all of which have standardized the offset at 1024, or "1MB.")

Fixing a misaligned VM requires downtime, but the reality these days is that downtime is almost impossible to schedule. As a result, there are many virtual machines out there running in a misaligned state.

At NetApp, we've developed a way to alleviate alignment problems without having to take virtual machines offline. Optimization and Migration lets you quickly identify misaligned VMs and mitigate the effects. Essentially, what VSC does is take the approach that "two wrongs make a right." We've found a way to "lie" to VMware, telling it that misaligned virtual machines are aligned. Then we address the misalignment on back-end storage, so there is no longer any impact on the network or the ESXi host.

To achieve this, VSC first does a scan operation to identify misaligned virtual machines. For each misaligned VM, all you have to do is migrate it to a "functionally aligned" datastore, which is a LUN with a specialized offset. If you've already performed the procedure in the past, a functionally aligned datastore will already exist. If not, the wizard lets you create one. When you complete the scan, VSC triggers a Storage vMotion® action into the functionally aligned datastore, which doesn't disrupt the running virtual machine.

If you've got many misaligned virtual machines, VSC lets you perform these processes in bulk so you don't have to move each virtual machine one at a time. Right now the process only works for VMFS datastores, but NFS support is coming. Note that you should never move an aligned VM into a functionally aligned datastore; if you do, it will become misaligned.

Mass migration. You should also know that this migration capability isn't only for misaligned VMs. One of the hidden gems of VSC is that you can perform mass migrations to quickly and easily move VMs to a different datastore, regardless of whether they are on NetApp storage or a completely different storage platform.

Backup and Recovery

In VSC 2.0, SnapManager® for Virtual Infrastructure (SMVI) was baked in to VSC. The standalone SMVI product is not being updated, is no longer downloadable, and is approaching end-of-support status. If you're still using it, you'll want to move to VSC, especially if you need clustered Data ONTAP support.

VSC gives you a lot of options for protecting datastores and individual virtual machines using a combination of native VMware snapshots and NetApp Snapshot™ and SnapMirror® technologies. NetApp Snapshot technology creates a point-in-time copy of a LUN or volume. These Snapshot copies only consume additional storage space as changes are made, so they are very efficient. SnapMirror replication starts with a Snapshot copy and replicates only changed blocks to your secondary storage so it's fast and uses network bandwidth very efficiently.

Consistent backups. When you do backups with VSC, you have the option of doing a VMware snapshot first so that VMs are backed up in a consistent state. You can perform ad hoc backups when you need them and you can create backup jobs with a schedule that backs up—and replicates, if desired—a set of datastores or VMs or an entire data center. You also have the option to run specific scripts before and after a job runs. These can be used to do specific application-related tasks, like putting a database in hot backup mode and so on.

Recovery options. When it comes to recovery, you have the option to recover an entire datastore at once, a single VM, or individual files from a particular VM. You can restore a single VM in a matter of minutes.

Conclusion

NetApp Virtual Storage Console delivers a lot of advanced capabilities that take the time and complexity out of executing many important tasks in VMware environments. With VSC you can:

Enable ESXi hosts to be configured according to NetApp best practices for optimal performance
Create, resize, and delete datastores
Clone virtual machines
Fix alignment problems without disrupting running VMs
Migrate VMs and entire datastores with a few clicks
Create a flexible backup and restore environment that simplifies your operations, lets you back up and restore faster, and saves storage space

If you want to find out more about these capabilities, check out my set of VSC videos:

Overview (includes where to find VSC and installation)
Monitoring & Host Configuration
Provisioning & Cloning
Optimization & Migration
Backup & Recovery
NFS VAAI Plug-in

When VSC 4.2 becomes available, role-based access control will further simplify the lives of both VMware admins and storage admins in data centers of all sizes. If you're an IT generalist responsible for everything, VSC can really cut down the time you spend managing your virtual environment.

In addition, for larger environments, storage admins can configure VS to give VMware admins only the permissions they need to perform specific storage-related tasks, making life simpler for both.

By Nick Howell, Virtual Storage Architect

Nick started in IT around Y2K, eventually diving into server administration after receiving his CCNA and MCSE 2000. He discovered virtualization and shared storage in 2007, and immediately began implementing it where he worked. Nick is an outspoken advocate of "virtualizing everything." At NetApp, he helps customers design and deploy advanced virtualization solutions, with a particular focus on management technologies, backup/recovery, and other technologies that make life easier for the everyday administrator.

Re: VSC: The Essential Tool for VMware on NetApp

Thu, 28 Mar 2013 19:32:13 GMT

Is certificate management easier in 4.2? We tried to get certs working in previous versions but it was painful.

Re: VSC: The Essential Tool for VMware on NetApp

datacenterdude — Thu, 28 Mar 2013 20:44:39 GMT

cdm,

Thanks for the question! In VSC 4.1, we added some enhancements where a self-signed certificate is generated upon installation. You're also welcome to bring your own 3rd party certificate as well. Instructions for importing the cert's are in the Install & Admin Guide.

-Nick

Re: VSC: The Essential Tool for VMware on NetApp

dennisvisser — Sun, 31 Mar 2013 14:31:39 GMT

The documentation of flash accell describes that in VSC4.2 there will be an option to enable and configure it with Virtual Storage Console 4.2.

(So no need for the Flash Accell Management Console = FMC anymore) Is the support for Flash Accell be included in version 4.2 or will this be included in a later release?

Re: VSC: The Essential Tool for VMware on NetApp

datacenterdude — Mon, 01 Apr 2013 18:14:56 GMT

VSC 4.2 will include equivalent functionality to manage Flash Accel 1.1. Being that it is still very new, some of the "icing on the cake" stuff is still being baked in, and it will grow as Flash Accel itself matures. Think of VSC as a "delivery vessel" of sorts, similar to how we give you a deployment mechanism for the NFS VAAI plugin. We [VSC development] don't actually develop the VIB that makes up the NFS VAAI plugin, but we simply streamline the deployment and handle a lot of the backend configuration steps for you.

The same applies to Flash Accel. We don't develop the agents, we're just streamlining the deployment and mgmt of the host-side cache.

VSC, to an extent, can be considered a replacement for the FMC, but part of your test/eval process should include trying both and settling on one. Trying to use both at the same time (at the time of this writing) can prove difficult.

Re: VSC: The Essential Tool for VMware on NetApp

Thu, 04 Apr 2013 19:31:57 GMT

Have any changes been made to the scheduling options VSC Backup and Recovery? It would be nice to be able to block periods of time where backups should not occur.

Re: VSC: The Essential Tool for VMware on NetApp

Thu, 04 Apr 2013 20:18:48 GMT

Is VSC 4.2 going to address support for Protection Manager/Snapvault integration with VMDKs over NFS?

Re: VSC: The Essential Tool for VMware on NetApp

datacenterdude — Thu, 25 Apr 2013 17:25:21 GMT

As far as I know, no changes have been made to the way scheduling is handled. Good feedback, though, and I agree that would be a nice feature! I'll make sure it gets to Prod Mgmt.

Thanks!

Re: VSC: The Essential Tool for VMware on NetApp

datacenterdude — Thu, 25 Apr 2013 17:26:56 GMT

Since you asked about VSC 4.2 specifically, the answer is No.

However there is extensive work in progress for a future version of VSC that will incorporate the SnapCreator framework, enabling all of your backend SnapVault and PM integrations to become a reality.

Re: VSC: The Essential Tool for VMware on NetApp

BrettEdelen — Thu, 23 May 2013 17:53:48 GMT

what is the 'Other' category for in the Optimization and Migration Section? (i understand Aligned and Misaligned)

I have vm's in two datastores that show up under 'Other', not sure if that means they need migrating, or what?

thanks.

Re: VSC: The Essential Tool for VMware on NetApp

Tue, 04 Jun 2013 13:15:58 GMT

I've scoured the guide, the web and checked with our NetApp POC but there are no instructions that I can find that tell me how to apply my CA signed certificates.. The admin guide only tells us how to regenerate a self signed, nothing about assigning CA signed.. If instructions exist please point them out.

Re: VSC: The Essential Tool for VMware on NetApp

Tue, 04 Jun 2013 14:17:27 GMT

is it true, that with OnTap 8.1.x VSC 4.2 does not need FlexClone licensed to allow several cloning & restore operations with NFS datastores, because a new API introduced with 8.1.x? Which operations can be performed and which operations will still need FlexClone licensed?

thx

Re: VSC: The Essential Tool for VMware on NetApp

maness — Mon, 10 Jun 2013 18:22:23 GMT

Check out https://kb.netapp.com/support/index?page=content&id=1013807 for instructions on how to use your own CA signed certificate.

Re: VSC: The Essential Tool for VMware on NetApp

maness — Mon, 10 Jun 2013 18:29:13 GMT

On page 101 of the Installation and Administration Guide:

Optimization and Migration places the virtual machine in the Misaligned > Other folder if it cannot be aligned. This can happen if the virtual machine:

• Has disks that are inaccessible

• Has a disk size of 0

• Has more than one disk with different offsets

• Has multiple disks spanning multiple datastores

• Lacks partitions

• Is an independent disk

• Reports an error during read attempts

• Is a dynamic disk

Note: If you use the Optimization and Migration capability when you have a dynamic disk, the capability might give you a false indication of alignment.

Re: VSC: The Essential Tool for VMware on NetApp

SPAUL4321 — Fri, 28 Jun 2013 15:37:20 GMT

I use VSC 4.1 and I agree its a great tool. Among other things, it's a great example of how Netapp's snapshot and flexclone technologies work together (with VMware) to achieve a common objective in such a seamless fashion.

One of the things I'd like to see is the ability to manually select the igroup during provisioning of a datastore -- similar to the way Snapdrive allows it in the wizard for "connecting" or "creating" disks.

We have some ESXi hosts which are not in Vcenter but connected to the same igroup. VSC goes ahead and creates a new igroup during provisioning. Though this is helpful for new admins (as it simplifies the decision making process), experienced admins would like to have the option to control this aspect, if possible. Thanks.

-s

Re: VSC: The Essential Tool for VMware on NetApp

pmanuel — Tue, 16 Jul 2013 22:19:08 GMT

hello Nick

can a windows VM aligned ( windows 2003) get misaligned after some time (8 months to a year) ? I'm asking because there is a customer who apparently happens that and they follow the best practices for virtual environment; and happens only on a windows machines,
all the linux machines are aligned ( the customer have a vmware 5.0 )

thanks in advanced 🙂

Re: VSC: The Essential Tool for VMware on NetApp

maness — Wed, 17 Jul 2013 03:18:05 GMT

It's possible if new partitions are later created that are larger than the existing partitions and are misaligned.

Re: VSC: The Essential Tool for VMware on NetApp

Wed, 17 Jul 2013 18:05:39 GMT

VM guests with vmdks with multiple offsets are aligned to match it's largest partition's offset. So, if a previously smaller partition has grown larger than the largest partition over a period of time (your customer's 8-month period), the tool will most likely report that the VM guest is misaligned.

Re: VSC: The Essential Tool for VMware on NetApp

pmanuel — Thu, 18 Jul 2013 12:36:14 GMT

thanks you for your answer guys 🙂

Re: VSC: The Essential Tool for VMware on NetApp

Mon, 22 Jul 2013 13:10:13 GMT

Is there any way to space reclaim for Linux , Solaris VMs using VSC 4.2 with Vcenter server 5.0.0 . I am able to reclaim the space for Windows VMs .