Data Infrastructure Management Software Discussions

Highlighted

Re: Return parameters for operation results

Hi,

 

Is this the best way to properly generate the output when listing snapshots for a specific volume?

 

Since OCUM6 and OCUM7 don't have volume snapshots anymore, I've resorted to querying volume snapshots via WFA4 workflows. We toyed with using a datasource job in WFA4 to do it, but it takes multiple hours for each job run, and puts a strain on our vservers.

 

My workflow command basically runs the following (I've purposefully omitted error-handling code, so keep this paste simple):

 

Connect-WfaCluster $Cluster

$vol = Get-NcVol -Name $VolumeName -Vserver $VserverName
$snaplist = $vol | Get-NcSnapshot

Get-WFALogger -Info -message $("Vserver: " + $VserverName + " Volume: " + $VolumeName + " Snapshots:" + $snaplist)

Add-WfaWorkflowParameter -Name "SnapList" -Value $snaplist -AddAsReturnParameter $true

 

 

The Get-WFALogger command only shows the snapshot name for some reason. I had assumed it would show all the default columns. Anyone know why this is?

 

The Add-WfaWorkflowParameter obviously fails, and gives me the following error:

ERROR  [get volume snapshot list] Cannot convert 'System.Object[]' to the type 'System.String' required by parameter 'Value'. Specified method is not supported.
ERROR  [get volume snapshot list] Failed executing command. Exception: Cannot convert 'System.Object[]' to the type 'System.String' required by parameter 'Value'. Specified method is not supported.

Highlighted

Re: Return parameters for operation results

 

Is this the best way to properly generate the output when listing snapshots for a specific volume?

 

-----

 

No.

 

 

But the error that is being thrown : Cannot convert 'System.Object[]' to the type 'System.String' required by parameter 'Value'. Specified method is not supported.

 

Is NOT generated at the Get-WfaLogger cmdlet. This error is being thrown by Add-WfaWorkflowParameter

 

 

Your code has problems at the 2 cmdlets. 

 

 

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Highlighted

Re: Return parameters for operation results


@sinhaa wrote:

 

Is this the best way to properly generate the output when listing snapshots for a specific volume?

 

-----

 

No.

 

 

But the error that is being thrown : Cannot convert 'System.Object[]' to the type 'System.String' required by parameter 'Value'. Specified method is not supported.

 

Is NOT generated at the Get-WfaLogger cmdlet. This error is being thrown by Add-WfaWorkflowParameter

 

 

Your code has problems at the 2 cmdlets. 

 

 


Hi @sinhaa

 

Apologies if I misstated myself. I absolutely understand that the two errors are regarding the workflow parameter code. It doesn't like the fact I'm passing an Object[] as a String. I need to somehow take the snapshot list output, and massage it into a String in order to successfully pass it as a WfaWorkflowParamter, correct?

 

A secondary issue is that the Get-WfaLogger, when I test the WFA cmd, only shows me the snapshot name, which to me seems a bit odd, since Get-NcSnapshot -Template shows 3 columns of data. I'm thinking that might just be how its outputted when it tries to print a table in WfaLogger, not exactly sure.

 

 

Highlighted

Re: Return parameters for operation results

@jauling_chou

 

Use the below code. Powershell is a great language, it can do virtually anything.

 

 

This will work.

 

 

Connect-WfaCluster $Cluster

$vol = Get-NcVol -Name $VolumeName -Vserver $VserverName
$snaplist = $vol | Get-NcSnapshot | Out-String

Get-WFALogger -Info -message $("Vserver: " + $VserverName + " Volume: " + $VolumeName + " Snapshots:" + $snaplist)

Add-WfaWorkflowParameter -Name "SnapList" -Value $snaplist -AddAsReturnParameter $true

sinhaa

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Highlighted

Re: Return parameters for operation results

Hi,

 

There are a few approaches, you can return a comma delimted string however this can be limiting if you want additional snapshot properties or require a third party orchestrator to interperet the output as an object with multiple properties.

 

If the volume contains a high number of snapshots (255 maximum) it's possible that you might also run into issues with a maximum string length of the WFA return paramater. I can't find any documentation that defines the maximum string length of a WFA return paramater but i would assume there is a limit (perhaps 1024 characters???). It's possible you might reach that limit if returning comma delimited strings in a single WFA return paramater for 255 snapshot names

 

Also if you are selecting a snapshot name (for the basis of creating a clone) then you might want to return additional properties of each snapshot such as it's creation date (rather than relying on the snapshot name to be an accurate definition of the snapshots actual creation date). WFA limits the return paramater data types to strings, integers or booleans. There is a workaround to this limitiation and that is to return a URL (which links to an XML or JSON file that your WFA command code will dynamically create and save to the the 'log-download" directory in .XML file format). EG

 

C:\Program Files\NetApp\WFA\jboss\standalone\deployments\log-download.war\$WfaJobID.xml

 

Using this method WFA can set a single return paramater URL than can be interpreted by other applications or ochrestrators as an object (not a string).

You can then have any other process read the content of the URL return parameter via https from the WFA server.

 

This can be useful if you wanted a list of volume snapshots on an ad-hoc basis and don't requrie a schedule datasource that enumerates snapshots for all clusters and all volumes. If you only want to list a single volumes snapshots on an occassional basis then it's rather inefficent to configure a WFA datasource that is constantly requesting all volume snapshots unnecessarily from all clusters. Why not just request a list of the volume snapshots when and if you actually need them?

 

Here is some example code for you. It will

 

  • enumerate snapshots (matching a snapshot naming pattern) for a specified cluster, vserver and volume
  • create the results as an XML file
  • return the URL to the XML file to be read or downloaded.

 

#'------------------------------------------------------------------------------
Param(
   [Parameter(Mandatory=$True, HelpMessage="The cluster name or IP Address")]
   [String]$ClusterName,
   [Parameter(Mandatory=$True, HelpMessage="The Vserver name")]
   [String]$VserverName,
   [Parameter(Mandatory=$True, HelpMessage="The Volume name")]
   [String]$VolumeName,
   [Parameter(Mandatory=$False, HelpMessage="The snapshot pattern to match")]
   [String]$SnapShotPatternMatch,
   [Parameter(Mandatory=$True, HelpMessage="The hostname or IP Address of the WFA server")]
   [String]$HostName,
   [Parameter(Mandatory=$False, HelpMessage="The port number of the WFA server")]
   [String]$PortNumber,
   [Parameter(Mandatory=$True, HelpMessage="The site name hosted on the WFA server")]
   [String]$SiteName,
   [Parameter(Mandatory=$False, HelpMessage="The Protocol used by the WFA server")]
   [ValidateSet("http","https")]
   [String]$Protocol
)
#'------------------------------------------------------------------------------
#'Enumerate the WFA install path from the registry
#'------------------------------------------------------------------------------
[String]$registryPath = "hklm:\system\currentcontrolset\services\na_wfa_srv";
Try{
   [System.Object]$result  = Get-ItemProperty -Path $registryPath -ErrorAction Stop
   [String]$wfaExeSpec     = $result.ImagePath.Split("/")[0].Trim() -Replace("""", "")
   [String]$wfaServicePath = $wfaExeSpec.SubString(0, $wfaExeSpec.LastIndexOf("\"))
   [String]$installPath    = $wfaServicePath.SubString(0, $wfaServicePath.LastIndexOf("\"))
   Get-WFALogger -Info -Message "Enumerated WFA installation Path from Registry key ""$registryPath"" as ""$installPath"""
}Catch{
   Get-WFALogger -Error -Message $("Failed Reading Registry Path ""$registryPath"". Error " + $_.Exception.Message)
   Throw "Failed Reading Registry Path ""$registryPath"""
}
#'------------------------------------------------------------------------------
#'Connect to the cluster.
#'------------------------------------------------------------------------------
Connect-WFACluster $ClusterName
#'------------------------------------------------------------------------------
#'Enumerate WFA envrionment variables.
#'------------------------------------------------------------------------------
[Int]$jobId           = $(Get-WfaRestParameter "jobId")
[String]$userName     = $(Get-WfaRestParameter "userName")
[String]$workflowName = $(Get-WfaRestParameter "workflowName")
[Int]$workflowId      = $(Get-WfaRestParameter "workflowId")
#'------------------------------------------------------------------------------
#'Create the XML content in an array.
#'------------------------------------------------------------------------------
[Array]$lines  = @();
[Array]$lines += "<?xml version=""1.0"" encoding=""UTF-8"" standalone=""yes""?>"
[Array]$lines += "<root revision=""0"" schemaVersion=""0"">"
[Array]$lines += "   <cluster>"
[Array]$lines += "      <cluster_name>$ClusterName</cluster_name>"
[Array]$lines += "      <vserver>"
[Array]$lines += "         <vserver_name>$VserverName</vserver_name>"
#'------------------------------------------------------------------------------
#'Enumerate the volume snapshots.
#'------------------------------------------------------------------------------
[String]$command = "Get-NcSnapshot -Volume $VolumeName -Vserver $vserverName -ErrorAction Stop"
Try{
   $snapShots = Invoke-Expression -Command $command -ErrorAction Stop
   Get-WFALogger -Info -Message "Executed Command`: $command"
}Catch{
   Get-WFALogger -Error -Message $("Failed Executing Command`: $command. Error " + $_.Exception.Message)
   %hrow "Failed enumerating snapshots for volume ""$VolumeName"" on vserver ""$VserverName"""
}
#'------------------------------------------------------------------------------
#'Add the snapshot property values to the XML content.
#'------------------------------------------------------------------------------
[Array]$lines += "         <volume>"
[Array]$lines += "            <volume_name>$VolumeName</volume_name>"
[Array]$lines += "            <snapshots>"
ForEach($snapShot In $snapShots){
   If($SnapshotPatternMatch){
      If($snapShot.Name -Match $SnapShotPatternMatch){
         [Array]$lines += "               <snapshot>"
         [Array]$lines += "                  <snapshot_name>" + $($snapShot.Name)    + "</snapshot_name>"
         [Array]$lines += "                  <created>"       + $($snapShot.Created) + "</created>"
         [Array]$lines += "               </snapshot>"
      }
   }Else{
      [Array]$lines += "               <snapshot>"
      [Array]$lines += "                  <snapshot_name>" + $($snapShot.Name)    + "</snapshot_name>"
      [Array]$lines += "                  <created>"       + $($snapShot.Created) + "</created>"
      [Array]$lines += "               </snapshot>"
   }
}
[Array]$lines += "            </snapshots>"
[Array]$lines += "         </volume>"
[Array]$lines += "      </vserver>"
[Array]$lines += "   </cluster>"
#'------------------------------------------------------------------------------
#'Add the WFA job variables to the XML.
#'------------------------------------------------------------------------------
[Array]$lines += "   <wfa>"
[Array]$lines += "      <workflow_name>" + $workflowName + "</workflow_name>"
[Array]$lines += "      <workflow_id>"   + $workflowId   + "</workflow_id>"
[Array]$lines += "      <job_id>"        + $jobId        + "</job_id>"
[Array]$lines += "      <username>"      + $username     + "</username>"
[Array]$lines += "   </wfa>"
[Array]$lines += "</root>"
#'------------------------------------------------------------------------------
#'Create the XML file.
#'------------------------------------------------------------------------------
[String]$fileSpec = "$installPath\jboss\standalone\deployments\$SiteName.war\$jobId.xml"
Try{
   Out-File -FilePath $FileSpec -Encoding ASCII -InputObject $lines -ErrorAction Stop
   Get-WFALogger -Info -Message "Created file ""$fileSpec"""
}Catch{
   Get-WFALogger -Error -Message $("Failed Creating file ""$fileSpec"". Error " + $_.Exception.Message)
   Throw "Failed Creating XML file ""$fileSpec"""
}
#'------------------------------------------------------------------------------
#'Add the volume snapshot names as a comma delimited string.
#'------------------------------------------------------------------------------
If($PortNumber){
   [String]$url = "$Protocol`://$HostName`:$PortNumber/$SiteName/$jobId.xml"
}Else{
   [String]$url = "$Protocol`://$HostName/$SiteName/$jobId.xml"
}
#'------------------------------------------------------------------------------
#'Add WFA return parameters
#'------------------------------------------------------------------------------
Get-WFALogger -Info -Message "Adding WFA retun paramater URL`: $url"
Add-WfaWorkflowParameter -Name "URL" -Value $url -AddAsReturnParameter $True
#'------------------------------------------------------------------------------

 

Note: the above code reads the WFA installation path from the registry so it's not hard coded to the default "C:\Program Files\NetApp\WFA" so it will work if a custom install location is used.

 

Example output (filtered to return all snapshots matching the pattern "daily")

 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<root revision="0" schemaVersion="0">
   <cluster>
      <cluster_name>cluster1</cluster_name>
      <vserver>
         <vserver_name>vserver1</vserver_name>
         <volume>
            <volume_name>volume_001</volume_name>
            <snapshots>
               <snapshot>
                  <snapshot_name>daily.2017-03-31_0010</snapshot_name>
                  <created>03/31/2017 11:10:00</created>
               </snapshot>
               <snapshot>
                  <snapshot_name>daily.2017-04-01_0010</snapshot_name>
                  <created>04/01/2017 11:10:00</created>
               </snapshot>
            </snapshots>
         </volume>
      </vserver>
   </cluster>
   <wfa>
      <workflow_name>get_volume_snapshots</workflow_name>
      <workflow_id>133</workflow_id>
      <job_id>44116</job_id>
      <username>admin</username>
   </wfa>
</root>

Return parameter:

 

https://XX.XX.XX.XX/log-download/$WfaJobID.xml

 

I can send you an example workflow if you like. Hope that gives you some ideas. Let me know if you have any questions.

 

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Highlighted

Re: Return parameters for operation results

@mbeattie @jauling_chou

 

The best approach in my opinion would be to use Export-Clixml and save the entire object into a temp timpe-stamped file. And pass this file-path Parameter to the next command.

 

On source command:

 

 

Connect-WfaCluster $Cluster

$vol = Get-NcVol -Name $VolumeName -Vserver $VserverName
$snaplist = $vol | Get-NcSnapshot 

Get-WFALogger -Info -message $("Vserver: " + $VserverName + " Volume: " + $VolumeName + " Snapshots:" + ( $snaplist | Out-String ))

$fileName = ".\myfile_" + (Get-Date -UFormat %s) + '_.xml'

Export-Clixml -Path $fileName -InputObject $snaplist

Add-WfaWorkflowParameter -Name ‘snapList’ -Value $fileName -AddAsReturnParameter $true

 

 

 

Now get the object in the destination command

 

 

 

$fileName = Get-WfaWorkflowParameter -Name ‘snapList’

$snaplist = Import-Clixml -Path $fileName

#Now do something with the object
foreach ($snap in $snaplist ) { $snap.Name , $snap.Created }


# Optionally delete this temporary file. WFA will anyway clear it off after some-time

Remove-Item $fileName

 

 

sinhaa

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Highlighted

Re: Return parameters for operation results

Thank you @mbeattie and @sinhaa for your responses. Looks like this is putting me on the right track.

 

In response to @mbeattie:

So I took the PowerShell, then created a WFA command from it:

$snaplist = $vol | Get-NcSnapshot | select Name,Created,Dependency
$snaplist_txt = $snaplist | Out-String
Get-WFALogger -Info -message $("Vserver: " + $VserverName + " Volume: " + $VolumeName + " Snapshots:" + $snaplist_txt)

 

The output when I test the WFA command truncates the snapshot name, which is the same behavior if I run it from CLI:

01:39:02.036 INFO  [get volume snapshot list] ### Command 'get volume snapshot list' in 'POWER_SHELL' ###
01:39:03.177 INFO  [get volume snapshot list] Using cached cluster connection
01:39:04.115 INFO  [get volume snapshot list] Vserver: VSERVER_HOSTNAME Volume: VOL1 Snapshots:
Name                       Created                    Dependency               
----                       -------                    ----------               
snapmirror.8e0fd5c2-e74... 4/4/2017 1:05:20 AM                                 
snapmirror.37f0555f-cb2... 4/4/2017 1:05:22 AM                                 
snapmirror.62cc231c-1c0... 4/4/2017 1:05:24 AM                                 
snapmirror.03af9c87-bde... 4/4/2017 1:06:06 AM        busy                     
01:39:04.161 INFO  [get volume snapshot list] Command completed, took 2125 milliseconds

So I piped the first line of code to Format-Table -AutoSize, but any table formating results in an empty list to WFA for some reason. Also, not really sure how to process a output in table format if I call the WFA workflow via REST (which is my primary use case)

 

 

With regards to @sinhaa:

 

Is there a way to pass XML as the output parameter without exporting/importing via a temporary file? We've only used WFA via REST, so I'm not sure what you mean in the second code snippet where you wrote "Now get the object in the destination command"

 

EDIT:

What I'm ultimately trying to do is to create a workflow that returns a list of snapshots for a given volume in XML form (much like a filter would). Not sure if that's really possible though.

 

Much appreciated guys, thanks!

 

 

Jau

 

Highlighted

Re: Return parameters for operation results

@jauling_chou

 

 

 

We've only used WFA via REST, so I'm not sure what you mean in the second code snippet where you wrote "Now get the object in the destination command"

 

-----

 

Jau,

 

The main purpose of cmdlet Add-WfaWorkflowParameter is to save a dynamically obtained data into a variable and then pass this value to another command to process further. Example: cmd-1 (source) creates a job on ontap and a job-id is returend, now I use Add-WfaWorkflowParameter to save this job-id into a variable 'jobId'. Now another cmd-2(destination) in the same workflow can get this job-ID using Get-WfaWorkflowParameter and then process further. 

 

Add-WfaWorkflowParameter also has parameter -AddAsReturnParameter which when set to true can provide the value of this dynamic data as 'Return Parameter' of the workflow excution.

 

Now this cmdlet has a major limitation that it can only save simple text values. If I have obtained a powershell object, I can't transfer it to the cmd-2. In your example, the snaplist is an object and hence Add-WfaWorkflowParameter can't transfer it to next cmd.

 

 

What I'm ultimately trying to do is to create a workflow that returns a list of snapshots for a given volume in XML form (much like a filter would). Not sure if that's really possible though.

 

----

 

If its powershell  + WFA then everything is possible ( well almost).

 

Once you get this list of snaplist, create your XML like @mbeattie gave in your example. You need to decide which attributes of a given snapshot you want .

 

 

Then do this.

 

 

$xmlDoc = [xml] $lines
$xmlValue = $xmlDoc.OuterXml

Add-WfaWorkflowParameter -Name "Snaplist" -Value $xmlValue -AddAsReturnParameter $True

 

Now when this workflow executes ( via REST as you want) , the workflow execution job will return the XML formatted list of snapshots as Return Parameters.

 

sinhaa

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Highlighted

Re: Return parameters for operation results

Thanks so much @sinhaa!

 

I struggled a bit, since the XML that I built with @mbeattie's instructions were imbedded inside the value field in the return parameters when I process the REST output, but once I grabbed that field and then processed it as its own XML doc, then I was able to extract the data I needed. Great stuff guys, thanks so much!

 

Cloud Volumes ONTAP
Review Banner
All Community Forums
Public