Snap Creator Pack for OnCommand Workflow Automation

[rmatt]

Attached is the latest release of the Snap Creator data source and initial workflows for WFA 2.0.1 and Snap Creator 4.0 environments.  With the introduction of APIs with Snap Creator 4.0, and using the Snap Creator PowerShell DLL, data protection can now be integrated into provisioning services.  Example workflows include:

• Create a new Snap Creator Profile
• Create a new Snap Creator Configuration (Separate Commands)
• Create Volume with Snap Creator Backup
• Start a Snap Creator Workflow

Multiple commands are included to create Snap Creator configurations:

• Start a Snap Creator Workflow
• Create New Snap Creator Profile - Empty
• Create New Snap Creator Config - Empty
• Create Full Snap Creator Configuration
• Update Snap Creator Volume Configuration Parameters
• Update Snap Creator Policy Schedule Configuration Parameters
• Update Snap Creator Agent Configuration Parameters

There is also a Snap Creator Data Source for pulling information from a Snap Creator server.  With the PowerShell cmdlets, Snap Creator data can be pulled into WFA for new Filters and Finders.

In order to use this Pack, the following pre-requisites must be addressed:

• Snap Creator 4.0 or newer must be installed
• The Snap Creator PS Toolkit (available for download using a NetApp support account here) needs to be installed and accessible on the WFA server
• The Snap Creator server must be reachable from the WFA server

Version Differences:

1.0: Initial release.

1.1: Updated to change all references to Get-NaCredentials to Get-WfaCredentials.  This release has not been fully qualified yet based on the code change but I am making it available sooner rather than later based on the change to WFA 2.1RC1.

[sprakash]

Awesome stuff Matt!

[hill]

I second that... truly awesome stuff Matt!

[raovolvoadmin]

Thanks Matt.Really useful stuff.

[bachman]

Still the ONE, the ONLY, Matt Robinson! Great stuff, and thank you!

[cknight]

Great job, Matt.

This WFA pack requires the Snap Creator PowerShell Toolkit, available here.

[smithch]

This looks pretty awesome.  I'm trying to get it working with the instructions from SnapCreator/WFA and running into issues with an error about tables being missing when I try to set up the SnapCreator power shell data source:

Table 'snap_creator.snap_creator_scheduler_frequency' doesn't exist

My installation of all components seems to be fine and functioning.  What am I missing here?

Thanks very much!

Best Regards,
CMS

[goodrum]

Edit the datasource and reset the scheme.  This will recreate the tables and database in your local WFA environment.

Jeremy Goodrum, NetApp

The Pirate

Twitter: @virtpirate

Blog: www.virtpirate.com

[mshearer]

I imported the pack but it did not create the data source or schemas, what do I need to do to create these?  Thanks.

[goodrum]

The data source will need to be created manually.  Do you see it as an available Data Source Type (Designer > Data Source Types)?  If the type exists then, create a new Data Source (Execution > Data Sources) and point to your SnapCreator server.  Once the new Data Source has been created, you will need to Edit the datasource and reset the scheme (Right click the Data Source and Edit; the Reset button is at the bottom of the pop-up).  Run a new acquisition after that.

Let me know if you are still having issues

Jeremy Goodrum, NetApp

The Pirate

Twitter: @virtpirate

Blog: www.virtpirate.com

[mshearer]

Good deal, I was able to reset the schema, now in regards to collecting information, what settings do I need to use for the data source in terms of port, method and script or driver type?

[goodrum]

I am confused.  The only time that you should set that information is when you are setting up the Data Source Type.

Here is a screenshot of my SnapCreator Data Source

Jeremy Goodrum, NetApp

The Pirate

Twitter: @virtpirate

Blog: www.virtpirate.com

[francoisbnc]

Hi,

"Start a Snap Creator Workflow" workfine, however, datasource acquisition return an error :

Error getting data source credentials -- please see log file for more details: System.Management.Automation.CommandNotFoundException: The term 'Get-NaCredentials' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Where this 'Get-NaCredentials' is coming from?

Any clue?

[mshearer]

"Get-NaCredentials was changed to Get-WfaCredentials and is no longer exported expect to WFA.  This prevents accidental access to the cache" I had run into this as well and found that answer here -> https://communities.netapp.com/message/115193#115193

[francoisbnc]

Thx Matthew

[rmatt]

I updated this for 1.1 to change Get-NaCredentials to Get-WfaCredentials, but I haven't fully tested it yet with 2.1RC1 (as my setup is 2.0.1 right now).  If you are using 1.1 at this stage, recognize that it's only been edited to make that change across the Data Sources and Commands and I'll need to give it a full test with 2.1RC1 over the next week or so.

--Matt

[mshearer]

I was testing out adding a new volume to an existing snap creator configuration and keep getting the error "You cannot call a method on a null-valued expression.” After some tracking I figured out it occurs in this section of code, but I cannot seem to figure out the issue is or why it is an issue, the best I figure is that some null values are getting passed and causing a problem.  Anyone have any thoughts on this?  Thanks.

-----------------------------------------------------------

#  add in existing entries (minus the volumes for the new Vserver)

$j = 0

foreach ( $CVol in $Config.volumes ) {

if ( $CVol.filer.hostname.ToUpper() -ne $VserverName.ToUpper() ) {

$Config.volumes[$j].filer = $CVol.filer

$Config.volumes[$j].name = $CVol.name

$j++

}

}

-----------------------------------------------------------

[goodrum]

I ran into this issue today as well and had to dive in to see what what going on.  It looks like the command was written in a way that precluded adding new volumes in an already registered Vserver.  I modified the command and made the necessary changes to make this work.  I have uploaded the changed version

Jeremy Goodrum, NetApp

The Pirate

Twitter: @virtpirate

Blog: www.virtpirate.com

============================================

Param(

          [parameter(Mandatory=$true, HelpMessage="Server IP Address or Name")]

          [string]$Server,

          [parameter(Mandatory=$true, HelpMessage="Profile Name")]

          [string]$ProfileName,

          [parameter(Mandatory=$true, HelpMessage="Configuration File Name")]

          [string]$ConfigName,

          [parameter(Mandatory=$true, HelpMessage="Transport Protocol")]

          [ValidateSet( "HTTP", "HTTPS" )]

          [string]$Transport="HTTPS",

          [parameter(Mandatory=$true, HelpMessage="Cluster IP Addresses or Name")]

          [string]$ClusterName,

          [parameter(Mandatory=$true, HelpMessage="VServer Management IP Addresses")]

          [string]$VserverName,

          [parameter(Mandatory=$true, HelpMessage="VServer Management User Name")]

          [string]$VserverUserName,

          [parameter(Mandatory=$false, HelpMessage="VServer Management Password")]

          [string]$VserverPassword,

          [parameter(Mandatory=$true, HelpMessage="Volume List (separated by commas)")]ac

          [string]$Volumes

)

# try to import the Snap Create PowerShell module

try {

          Import-Module SnapCreator

} catch {

          $error = "ERROR: Failed to Import-Module SnapCreator!  Check installation path!"

          throw $error

}

# grab the Snap Creator credentials (WFA->Execution->Credentials)

$Credentials = Get-WfaCredentials -Host $Server

if ( !$Credentials )  {

          $error = $( "ERROR: Failed to get Snap Creator credentials for " +

                    $Server +

                    ".  Check Execution->WFA Credentials for valid IP address, " +

                    "username and password entries." )

          throw $error

}

# set the default port (if not defined)

if ( !$Credentials.Port ) {

          $Port = 8080

} else {

          $Port = $Credentials.Port

}

try {

          Get-WFALogger -Info -message $("Connecting to Snap Creator Server at " + $Server )

          Connect-ScServer -Name $Server -Credential $Credentials -Port $Port

} catch {

          $msg = $_.Exception.Message

          throw $msg

}

# get the SC profile specified

try {

          # get cluster by provided name

          Get-WFALogger -Info -message $( "Getting information for profile named " + $ProfileName + " ..." )

          $Profile = Get-ScProfile -ProfileName $ProfileName

} catch {

          $msg = $_.Exception.Message

          throw $msg

}

# get the SC configuration entry

try {

          # get cluster by provided name

          Get-WFALogger -Info -message $( "Getting new configuration file named " + $ConfigName + " ..." )

          ## Renamed the Config variable to be OrigConfig to prevent variable thrashing and to allow

          ## for an easier tracking of the changes.

          $OrigConfig = Get-ScConfig -ProfileName $ProfileName -ConfigName $ConfigName

          $Config = Get-ScConfig -ProfileName $ProfileName -ConfigName $ConfigName

} catch {

          $msg = $_.Exception.Message

          throw $msg

}

Get-WFALogger -Info -message $( "Setting general configuration parameters ..." )

$Config.connection.protocol = $Transport

if ( $Transport -eq "HTTP" ) {

          $Config.connection.port = 80

} else {

          $Config.connection.port = 443

}

# grab our vserver credentials and get ready for setting them when we process our volume list

Get-WFALogger -Info -message $( "Getting credentials for cluster name/IP address " + $ClusterName + " ..." )

$ClusterCredentials = Get-WfaCredentials -Host $ClusterName

if ( !$ClusterCredentials )  {

          $error = $( "ERROR: Failed to get cluster credentials for " +

                    $ClusterName + ".  Check Execution->WFA Credentials for valid IP address, " +

                    "username and password entries." )

          throw $error

}

# convert the cluster passwords

$CClearPassword = ConvertFromSecureToPlain -SecurePassword $ClusterCredentials.Password

$CSCPassword = ConvertTo-ScSecureString -Input $CClearPassword

# see if there are any existing controllers

if ( !$Config.connection.filers ) {

          try {

                    Initialize-ScObjectProperty $Config.connection filers 1

                    Initialize-ScObjectProperty $Config.connection.filers[0] filer 1

          } catch {

                    $msg = $_.Exception.Message

                    throw $msg

          }

          $i = 0

} else {

          # let's count what we have

          $i = 0

          ## Switched the variable to use the historical version

          $Filers = $OrigConfig.connection.filers

          $Found = $false

          foreach ( $F in $Filers ) {

                    # don't count the Vserver we are adding

                    if ( $F.filer.hostname.ToUpper() -eq $VserverName.ToUpper() ) {

                              $Found = $true

                    }

                    $i++

          }

          # we need a new entry created if we don't find it in our existing list

          if ( $Found -ne $true ) {

                    $i++

          }

          # now let's initialize the right number

          try {

                    Initialize-ScObjectProperty $Config.connection filers $i

          } catch {

                    $msg = $_.Exception.Message

                    throw $msg

          }

          $i = 0

          foreach ( $F in $Filers ) {

            # now set the credential information for everything but the new controller (if any)

            if ( $F.filer.hostname.ToUpper() -ne $VserverName.ToUpper() ) {

              $Config.connection.filers[$i].filer = $F.filer

              $i++

            }

          }

}

if(!$VserverPassword) {

  $VserverPassword = $CSCPassword

}

# now add this entry if it didn't exist or update the values based on a pre-defined vserver

Get-WFALogger -Info -message $( "Adding Vserver " + $VserverName + " to configuration ... with password " +  $VserverPassword)

Initialize-ScObjectProperty $Config.connection.filers[$i] filer 1

$Config.connection.filers[$i].filer.hostname = $VserverName

$Config.connection.filers[$i].filer.username = $VserverUserName

$Config.connection.filers[$i].filer.password = $VserverPassword

$Config.connection.filers[$i].filer.protocol = $Transport

# add in the clustered Data ONTAP values

Initialize-ScObjectProperty $Config.cMode credentials 1

Initialize-ScObjectProperty $Config.cMode.credentials[0] credentials 1

$Config.cMode.clusterName = $ClusterName

$Config.cMode.AddClusterUser( $ClusterName, $ClusterCredentials.Username, $CSCPassword )

$Config.cMode.cModeSet = $true

# now walk through the volume list, separated by commas and get a count

Get-WFALogger -Info -message $( "Splitting list of volumes ..." )

$VolumeList = $Volumes.Split(",")

$j = 0

## Added this variable to allow for easier tracking of any existing volume in the configuration

$ExistingConfig = ""

if ( !$OrigConfig.volumes ) {

          # now initialize and set all of the filers

          Get-WFALogger -Info -message $ "No volumes found.  Initializing the volume list "

          try {

                    Initialize-ScObjectProperty $Config volumes $VolumeList.count

          } catch {

                    $msg = $_.Exception.Message

                    throw $msg

          }

} else {

          ## see how many entries we have in place and add those in this VServer to the

          ## variable ExistingConfig

          foreach ( $CVol in $OrigConfig.volumes ) {

            if ( $CVol.filer.hostname.ToUpper() -eq $VserverName.ToUpper() ) {

              foreach ($NewVol in $VolumeList) {

                if($CVol.name -eq $NewVol) {

                   $VolumeCount++

                   $ExistingConfig += ($CVol.name + ",")

                }

              }

            }

            $j++

          }

          ## Determine the count of new Volumes plus the existing list of volumes

          $j += ($VolumeList.count - $VolumeCount)

          try {

                    Initialize-ScObjectProperty $Config volumes $j

          } catch {

                    $msg = $_.Exception.Message

                    throw $msg

          }

          #  add in existing entries to the new Config

          $j = 0

          foreach ( $CVol in $OrigConfig.volumes ) {

               $Config.volumes[$j].filer = $CVol.filer

               $Config.volumes[$j].name = $CVol.name

               $j++

          }

}

## Split the existing list of Volumes into an Array

$ExistingVolumes = $ExistingConfig.split(",")

## Add to the new config the list of new volumes

foreach ( $Volume in $VolumeList ) {

          ## Verify that the volume in the list doesn't exist in the current configuration

          if($ExistingVolumes -notcontains $Volume) {

            Get-WFALogger -Info -message $( "Adding volume " + $Volume + " to configuration ..." )

            $Config.volumes[$j].filer = $Config.connection.filers[$i].filer

            $Config.volumes[$j].name = $Volume

            $j++

          }

}

# now write out the new config

try {

          # get cluster by provided name

          Get-WFALogger -Info -message $( "Saving new Snap Creator configuration ..." )

          Update-ScConfig -ProfileName $ProfileName -Config $Config

} catch {

          $msg = $_.Exception.Message

          throw $msg

}

Get-WFALogger -Info -message $( "Configuration updated successfully." )

============================================

[mshearer]

That worked, thank you!