ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Tech_OnTap — Wed, 11 May 2016 19:08:17 GMT

May 2016

Ezra Tingler
Senior Storage Engineer
NetApp

My name is Ezra Tingler, and I work in NetApp’s Customer-1 Storage Services organization. The Customer-1 Storage Services team is responsible for the architecture, procurement, deployment, and maintenance of the storage infrastructure that services all of our internal applications. The team is organized into service line owners, each of whom owns a particular aspect of our storage technologies. As expected, our storage infrastructure is built on the latest and greatest NetApp technologies. As a large consumer of NetApp technology, the Customer-1 Storage Services team also serves as a reference to external NetApp customers when it comes to showcasing the correct utilization of NetApp products.

On this team, I own the Storage Ecosystem Service line, which means I am responsible for storage hardware lifecycle management. My main goal is to ensure that our storage software and hardware creates a stable, functioning storage ecosystem from which other service lines and applications are provided data services. Currently, our team is migrating all data hosted on NetApp siloed storage (i.e. 7-mode) to clustered storage.

Challenge

As part of this migration project, I’ve been installing additional storage nodes and clusters. When I first started this project, the average configuration time for a high availability (HA) clustered controller pair was about four hours, spread out over two to three days. The four hours did not include the time needed to configure the cluster inter-connect switches or initialize disks; that took another two to twelve hours depending on disk type. Typical office interruptions added even more time as I had to figure out where I had left off and what I still needed to configure. This sporadic schedule resulted in missed deadlines and some configuration inconsistencies. I knew there had to be a better way to go about this project.

The Solution

I challenged myself to see if I could automate the storage configuration process to save time and reduce errors. Although I’m not a developer, I found it easy to write a configuration script using the NetApp Manageability Software Development Kit (NM SDK). I run my script once the disks are initialized, cluster setup is complete, and the cluster inter-connect switches are properly configured. All in all, the script configures 23 specific items:

Rename the cluster nodes
Rename the any existing interfaces to match the new node names
Cluster interfaces and node management interfaces
Rename the root aggregates to match the new node names
Install licenses
Configure the service processors
Set the flow control to none on all 10g ports
Create broadcast domains and assigns the proper ports
Create interface groups and add the correct ports
Create VLANs
Create failover groups and assign the correct ports
Create backup interfaces (intercluster)
Create user roles
Create users
Set the RAID scrub schedule
Create aggregates
Disable aggregate snapshots on all aggregates
Create cluster peers
Configure NTP
Configure SNMP
Configure CDP
Configure web services
Configure AutoSupport

As the script runs, it reads predefined configuration information from a file I create, and applies this configuration to the cluster nodes. The only thing I need to do before running the script is to edit the config file with unique information, such as node names, IP addresses, and password keys.

After using this script, I was amazed by the results. The four-hours it used to take has been reduced to about five minutes. Using the configuration script, I can now install 24 storage nodes in two hours rather than 96 hours, a time savings of 94 hours or 2 1/2 work weeks. Errors caused by interruptions have also been eliminated, and automating this process has freed up my time so that I can work on other projects.

If you are a storage admin, you can easily create your own configuration script using the SDK. I used a tool included in the SDK called Z-Explorer that contains a complete list of all ZAPI calls for the cluster. With Z-Explorer, most of the development work is done for you. It took me just a few weeks to fully automate my clustered storage builds. This KnowledgeBase article is a good place to start.

It was a fun project because I could write the script without feeling like I had to be a developer. I wrote the scripts in Perl, but the SDK works with almost any language you are familiar with. I also used the SDK online forum to get advice from others. People on this forum were always quick to answer my questions.

The Future

I’m now using the SDK to automate and streamline other storage tasks to save time and reduce errors. My next project is a quality assurance (QA) script that will login to a cluster and verify if nodes are properly configured per NetApp IT Standards and NetApp best practice guidelines. I plan to automate the cluster interconnect switch configuration in the same way as well as create an E-Series configuration script.

If you are interested in seeing the actual scripts, I am in the process of making them available on github.com. You can look for them here. In future TechOnTap newsletters, I’ll make sure to post the URL links once they are available. I’ll also be appearing on an upcoming Tech OnTap podcast to speak more about my automation process with other subject-matter experts from across the industry.

Ezra Tingler, Senior Storage Engineer at NetApp, has more than 20 years of experience in storage administration and UNIX systems.

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

HENRYPAN2 — Thu, 12 May 2016 17:33:25 GMT

Saluting Ezra!

Henry PAN

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

itdavisllp — Fri, 13 May 2016 22:59:04 GMT

"My next project is a quality assurance (QA) script that will login to a cluster and verify if nodes are properly configured per NetApp IT Standards and NetApp best practice guidelines"

How will this be different from Config Advisor which already does a very good job of this?

Thanks

Adrian

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

HENRYPAN2 — Fri, 13 May 2016 23:05:35 GMT

Cool Adrian,

Wish your script better than Config Advisor.

Be careful in this Friday the 13th.

Henry

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

EzraTingler — Mon, 16 May 2016 13:45:35 GMT

@itdavisllp, Config advisor does an excelent job for validating the majority of the configuration and proper cabliing. It is and will continue to be part of our standard QA process. The QA script and workflow will go a step further to ensure the configuration is completed to our internal IT standards. Its like an deeper verification of naming standards and settings which are not checked by Config Advisor.

Ezra Tingler

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

johnh703 — Wed, 01 Jun 2016 20:39:07 GMT

This seems like something that would be very useful, can you tell us exactly where it's located? I clicked on the link, I see lots of repositories in the link, I can't which one is yours. Thanks.

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

ADailey — Fri, 23 Dec 2016 22:24:51 GMT

Did anyone ever reply to johnh703's request on exactly where this script is in github? I too would like to review this script, but don't see it in any of the repos?

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

OZWALKERZ — Mon, 18 Jun 2018 04:02:59 GMT

Did this script become available?

We've got a new controller pair on the way, and this might be useful!

Cheers,

Stuart

article ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes in Tech ONTAP Articles

ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes

Re: ONTAP Automation: How to Do a 4-hour Configuration in 5 Minutes