Powershell dies on connect-nacontroller command

[johneaton]

I'm a novice with Powershell, but have many years scripting experience, so hopefully you powershell gurus can help me out.

Not to get bogged down with the 'big picture', a quick synopsis - Running SMSAP backups, then using SnapMirror to destination filer, and SnapVault to a third.

I've got my scripts working fine, and have been for some time. We are now is getting them running with the scheduler - OPALIS. After a number of OPALIS vs Powershell issues were resolved, my SMSAP backup script is working. Now moving on to SnapMirror Update. As soon as the script hits the connect-nacontroller command, powershell silently dies, and a sucess return code is sent to the OPALIS agent.

Environment - Win 2008 R2 (64 bit), Powershell 2.0, Netapp toolkit 1.4, and OPALIS 1.55 (32 bit). Due to OPALIS being 32 bit, and to hide the lengthy command line from it, we call a batch script which calls Powershell fully pathed. By default this is invokes cmd.exe *32, hence the full path to Powershell *32. I don't believe this to be part of the issue, but trying to provide critical details. The Powershell scripts work from the command line (cmd & PS CLI) with both 32 and 64 bit PS.

bat file:

c:\windows\syswow64\WindowsPowerShell\v1.0\powershell.exe -Noninteractive "&'E:\SMSAPScripts\SmsapMirrorUpdate.ps1' %1; exit $LASTEXITCODE" <NUL

exit %ERRORLEVEL%

PS1 file (with extra comment output to find failure point, but functions, setup, arg validation, etc removed for clarity):

#Connect to filer

WriteLog "Connect to filer"

connect-nacontroller $Dst_controller

if ( ! $? )

{

    write "Unable to connect to Filer: $($Dst_controller)"

    exit 3

}

WriteLog "Connected to filer"

# Up the filer timeout to prevent errors on connections

# default fot toolkit v1.4 (current) is 10 seconds, 60 on v1.5

$global:CurrentNaController.TimeoutMsec = 60000

WriteLog "Starting update"

foreach ( $VolName in $mr_Volume_Names )

{

    Invoke-NaSnapmirrorUpdate -Destination $VolName > $null 2>>$LogFile

    if ( ! $? )

    {

        ErrorExit "Cannot start update of SnapVault source: $($VolName)" 20

    }

}

WriteLog "Script SnapMirrorUpdate.ps1 completed"

Resultant log:

Starting Script SnapVaultUpdate.ps1 for SAP Database SE1, Thu Aug 25 12:11:35 2011

reading config file, Thu Aug 25 12:11:35 2011

Connect to filer, Thu Aug 25 12:11:35 2011

Does anyone have a clue why Powershell would silently die on the connect-nacontroller command?

I'm pretty much at a dead stop, without any leads at this point.

Thanks for any help!

John

[ajeffrey]

I'm sure some of the Toolkit gurus will chime in here too but the first thing I would do is upgrade to the latest version of the toolkit.  There was a fix for timeout issues in there and it couldn't hurt. 

[johneaton]

Thanks. I am aware of this, but upgrading requires approvals and chg mgmt so not a quick process. Since it works fine from the command line (hundreds of times) I doubt timing is an issue.

I just tried another test, using the windows task manager, and it works with that. I'm gravely concerned this is yet another OPALIS bug/issue.

John

[timothyn]

There are a lot of variables to consider here with the batch file, 32 bit/64 bit execution, running non-interactive, etc...

I have a hunch that the credentials the script is running under are not valid for RPC auth against the controller.  If you connect through the commandline you can add credentials to the toolkit's credential store:

connect-nacontroller MyController -Credential (Get-Credential)
Add-NaCredential -SystemScope

After that your script should be able to use the credentials you entered, and will use HTTP/HTTPS to connect/authenticate instead of RPC/Windows Credentials.  That might clear you up.

Maybe someone with more Opalis experience can chime in?

[johneaton]

Credentials cached log ago. I did all the development on a test system, documented the entire procedure (so the customer can do it without any background) and then we rolled it out to the prod server to integrate with OPALIS. As noted, it works fine from the command line, and now verified with the windows task manager. Since powershell silently dies, it's probably not a syntax/functional issue.

The poweshell event logs don't help either. The event logs for the failed job look the same as good jobs.

John

[jausch]

Unfortunately, this is a known issue with Opalis.  This is better in 6.3 but not fully resolved.

Opalis does not support a full PowerShell environment so some things work differently than with a regular PowerShell window.

I'm afraid that from a supporability point of view that if it works correctly in PowerShell then we know that the PS toolkit is working.  If it doesn't work in Opalis, that is due to a lack of PS support within Opalis, not due to a PS toolkit failure.

One trick that we've used successfully here is to use the invoke-command cmdlet.  You can then have the PS script run in a "native" PowerShell context.

Something like this:

invoke-command -computerName "MyServer" {

Insert commands here....

}

[johneaton]

Alex,

I was afraid that this might be the case, given all the other issues I've seen and read about regarding OPALIS. I'll give the invoke-command process a try.

Thanks, John

[johneaton]

Alex,

I tried this method, but I don't see that a new PS process gets invoked. FYI, there is no remote computer involved.

This comment from get-help seems to backup this operation:

"You can also use Invoke-Command on a local computer to evaluate or run a string in a script block as a command. Windows PowerShell converts the script block to a command and runs the command immediately in the current scope, instead of just echoing the string at the command line."

Here is what I tried:

c:\windows\syswow64\WindowsPowerShell\v1.0\powershell.exe -Noninteractive "invoke-command -scriptblock { E:\SMSAPScripts\SmsapMirrorUpdate.ps1 %1 } " <NUL

John

[johneaton]

Tried a lookback to see if it creates a new PS process, it does, but now the script fails on the connect-nacontroller:

RPC Error - FormatMessage failed to create error message

Same exact command and script, but only added the -computer argument, and enabled remoting. I suspect the error has something to remote mode, but this is out of my league.

Begging the cutomer to put Opalis to rest....only other option is Redwood, but it's not in production yet. Any one have any Redwood & Powershell pointers/warnings for me? I've only worked with it on Linux in the past.

John