Here's some more information. When I posted above, I was trying to use API direct to the controller using New-WFAZapiServer. I've switch to using the ONTAP toolkit to see if I can get further. Unfortunately, running Connect-WFAController in a loop only works for the first iteration, all others fail with invalid credentials. I have confirmed both by running workflows and by using the Test Connection feature of Credentials that the credentials are correct (in fact, they are exactly the same for all 4 controllers in question). Here's some logging showing the behavior (10.61.169.30-33 are controllers that have valid credentials, but only the first works):
2013-04-03 17:23:24,067 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-14) Volume Root:Starting Volume Root Acquisition
2013-04-03 17:23:28,442 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-28) Volume Root:Scanning filer at 10.61.169.30
2013-04-03 17:23:28,442 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-38) Volume Root:Get-NaCredentials -Host 10.61.169.30
2013-04-03 17:23:28,473 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-4) Volume Root:Connect-NaController (with credentials) -Name 10.61.169.30
2013-04-03 17:23:29,192 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-22) Volume Root:Connected to controller
2013-04-03 17:23:31,035 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-24) Volume Root:Scanning filer at 10.61.169.31
2013-04-03 17:23:31,035 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-20) Volume Root:Get-NaCredentials -Host 10.61.169.31
2013-04-03 17:23:31,082 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-8) Volume Root:Connect-NaController (with credentials) -Name 10.61.169.31
2013-04-03 17:23:31,098 EDT WARN [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-52) Volume Root:Failed to connect to controller: 10.61.169.31, Incorrect credentials for 10.61.169.31.
2013-04-03 17:23:31,113 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-64) Volume Root:Scanning filer at 10.61.169.32
2013-04-03 17:23:31,113 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-63) Volume Root:Get-NaCredentials -Host 10.61.169.32
2013-04-03 17:23:31,145 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-23) Volume Root:Connect-NaController (with credentials) -Name 10.61.169.32
2013-04-03 17:23:31,192 EDT WARN [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-16) Volume Root:Failed to connect to controller: 10.61.169.32, Incorrect credentials for 10.61.169.32.
2013-04-03 17:23:31,192 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-6) Volume Root:Scanning filer at 10.61.169.33
2013-04-03 17:23:31,207 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-25) Volume Root:Get-NaCredentials -Host 10.61.169.33
2013-04-03 17:23:31,238 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-2) Volume Root:Connect-NaController (with credentials) -Name 10.61.169.33
2013-04-03 17:23:31,254 EDT WARN [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-17) Volume Root:Failed to connect to controller: 10.61.169.33, Incorrect credentials for 10.61.169.33.
2013-04-03 17:23:31,395 EDT INFO [com.netapp.wfa.command.execution.instance.impl.ExecutionInstanceDaoImpl] (http-0.0.0.0-80-9) Volume Root:Volume Root Acquisition Complete
Here's the relevant code:
Get-WFALogger -Info -message "Starting Volume Root Acquisition"
$connectionInfo = get-ConnectionInfo
$dfm = New-WFAZapiServer -Host $connectionInfo["host"] -Port $connectionInfo["port"] -Credentials $connectionInfo["credentials"] -Type "DFM"
$filerIPs = Get-DfmFilers($dfm)
foreach ($ip in $filerIPs) {
Get-WFALogger -message "Scanning filer at $ip" -Info
Try {
$filer = Connect-WFAController -Array $ip
}
Catch [system.exception] {
$err = $_.Exception.Message -replace ".`n", ", "
Get-WFALogger -Warn -message "$err"
continue
}
if (!$filer) {
Get-WFALogger -Warn -message "Cannot connect to filer at $ip, empty credentials"
continue
}
$rootVol = Get-NaVolRoot -controller $filer
$domain = (Get-NaOption -controller $filer dns.domainname).value
$name = Get-NaSystemInfo -controller $filer
Add-Content $volRootFile ([byte[]][char[]] "\N`t$rootVol`t$name`t$name.$domain`t$ip`n") -Encoding Byte
}
Get-WFALogger -Info -message "Volume Root Acquisition Complete"
I'm not sure what I could be doing wrong here. It sure looks like Connect-WFAController may be hanging on to some state. If I try to use Get-NaCredentials explicitly, I'm back to the problem I had with the New-WFAZapiServer approach (null creds from Get-NaCredentials).
Attached is the Data Source Type. Any ideas?
Thanks!
Brian