NetApp PSTK E-Series 'A non-protocol error occured'

[Dave_Korns]

First, of all, I'm not sure the best place to ask this, here or over in E-Series Plug-in section, but I'll start here. 

Second, I admit to having weak E-Series background but fairly strong on PowerShell and DataONTAP toolkit.

Environment:

 - purely lab to dev/test E-Series automation with PowerShell and WFA

 - Current running:

    - Win2012 system-A

       - SANtricity Sim v11.20

       - SANtricity WSP v1.3

    - Win2012 system-B

       - Netapp PowerShell Tool 4.1

       - WFA 3.1P2 (thou not involved in this problem/question)

I've included a hunk of test-code belwo demonstrate my problem. I'm trying to determine why my New-NeThinVolume cmdlet is going: 'A non-protocol error occured'

I probably don't fully understand SANtricty and WSP security but I'm stuck. I'm flip flopping between two user/pswds in user.properties file: eseries and admin. When I run this code with 'eseries' user the error is: 'The operation cannot complete because you did not provide a valid administrator password. '

---

#Tue May 10 17:05:21 PDT 2016

admin=MD5\:6c6c7c5d71ad44b9e806d5023aadef4b,admin

eseries=MD5\:6c6c7c5d71ad44b9e806d5023aadef4b,storage.rw

ro=MD5\:3605c251087b88216c9bca890e07ad9c,storage.ro

rw=MD5\:038c0dc8a958ffea17af047244fb6960,storage.rw

---

When I change to the 'admin' user, the error becomes 'A non-protocol error occured'. I'm sure there is something wrong with the cmdlet formation but I haven't a clue. PSTK documentation and expamples seem sparse. Most other cmdlets I try behave the same ... though Get-NeDisplaySystems seems to consistently work.

Is it how I'm forming the systemId and/or poolid correctly? 

---- the code ---

 $NPMPath = "C:\Program Files (x86)\NetApp\NetApp PowerShell Toolkit\Modules\NetApp.SANtricity.PowerShell\"

try {

$Hostname = "hq-stor-es"

$Port= 8080

    $Username = "admin"

    $Password = "netapp123"

    $secPassword = ConvertTo-SecureString $Password -AsPlainText -Force

$PSCred = New-Object system.management.automation.pscredential( $Username, $secPassword )

} catch {

throw "Error getting data source credentials. Please see the log file for more details"

}

$URI = $( "http://" + $Hostname + ":" + $Port )

if ( $Port -eq 443 -or $Port -eq 8443 ) {

$URI= $( "https://" + $Hostname + ":" + $Port )

}

try {

$Url = $( $URI + "/devmgr/" )

$NECred = New-NeCredential -ProxyUrl $Url -SystemCredential $PSCred

}  catch {

throw $( "Error running New-NeCredential.  Verify SANtricity PowerShell Module is installed on the WFA server." )

}

write-output ""

write-output "LISTING OUT ALL STORAGE SYSTEMS ..."

$StorageSystems = Get-NeStorageSystem -Credential $NECred

for ( $ssi = 0; $ssi -lt $StorageSystems.count; $ssi++ ) 

    {

        write-output " "

        write-output " "

write-output $("   ID: " + $StorageSystems[$ssi].id )

write-output $(" Name: " + $StorageSystems[$ssi].name )

write-output $("Model: " + $StorageSystems[$ssi].model )

write-output $("  WWN: " + $StorageSystems[$ssi].wwn )

  write-output $("IPadr: " + $StorageSystems[$ssi].ip1 )

  }

write-output " "

write-output "ATTEMPTING TO CREATE NEW THIN VOLUME ..."

$tvObj = New-NeThinVolume  -Credential $NECred -SystemId 1 -PoolId 3233343536373839303132333100000000000000 -Name myVolume -SizeUnit gb -VirtualSize 128 -RepositorySize 4 -MaximumRepositorySize 4

 --- console log of running above in ISE ---

 PS C:\Windows\system32>

   ID: 1

 Name: CUA34286ESE001

Model: 5504

  WWN: C0A87E78040404040404040404040404

IPadr: 127.0.0.1

   ID: 2

 Name: CUA34286ESE2901

Model: 5481

  WWN: C0A87E78070707070707070707070707

IPadr: 127.0.0.1

   ID: 3

 Name: MIA34286ESE001

Model: 5600

  WWN: C0A87E78090909090909090909090909

IPadr: 127.0.0.1

ATTEMPTING TO CREATE NEW THIN VOLUME ...

New-NeThinVolume : A non protocol error occurred.

At Z:\VMshare\My Accounts\~POSH-UTILS\e-series-test.ps1:44 char:10

+ $tvObj = New-NeThinVolume  -Credential $NECred -SystemId 1 -PoolId 3233343536373 ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: ({"poolId":"3233...ositorySize":4}:String) [New-NeThinVolume], NetAppPowerShellException

    + FullyQualifiedErrorId : parameterError,Netapp.Santricity.PowerShell.NewThinVolumeCmdlet

PS C:\Windows\system32>  

[Aparajita]

I would recommend asking in the ESeries PlugIn section. This section is majorly peopled by Data ONTAP experts ...

[Dave_Korns]

Thanks. I did that also.

[khulques]

I'll take a divide and conquor approach.  The first issue with the eseries user, I will speculate that the web services proxy needs to be restarted after securepasswords was run.  That will enable this user to work via PowerShell.

For the second issue with 'non-protocol error', there are a couple things to try.  There is a problem or issue with the parameter combination for the cmdlet call.  Start with the web services trace log file.  It will reveal which parameter is not correct.  Remember, thin volumes can only be created from Disk Pools, not Volume Groups.  You can also try out parameter combinations with the web services active documentation.  The following screen shot demonstrates how to use them.  

Open the web services documentation and navigate to the POST for Thin Volumes.  Using the Try It Out feature, copy and paste the parameter values you are supplying to the cmdlet into the payload for the POST.  The url is http[s]://host:port/devmgr/docs.  Thin volume endpoints reside in the Volumes grouping.

Future releases of the SANtricity PowerShell toolkit will improve help documentation and increase the level of debugging information so that the web services documentation is not required.

[Dave_Korns]

Thanks Kevin. That gave me enough clues to move forward.

I've got 3 simulated systems (SystemIDs 1,2  and 3) and I discovered that I had fiddled with sysID:1 and had enabled an admin password on it. I got it turned off, but on this user/pswd topic I have these questions:

• How do the WSP per-controller/system attributes adminPasswordSet and readOnlyPasswrdSet relate to the WSP accounts themselves? Totally separate? Or when WSP is attempting an operation against a system with "adminPasswordSet": true, does it pass the WSP user/pass that the WSP-caller logged into WSP with? 
• The example admin user in users.properties is 'admin' where-as SANtricty StorMgr suggests it is 'Administrator' ... but maybe there is no relationship? or do I need to make them the same?
• The impression I get now is I should be able to create the ThinVolume with just WSP rw role, correct? or is that an admin operation?

So now New-NeThinVolume is giving me:

New-NeThinVolume : The operation cannot complete because either (1) the current state of a component does not 

allow the operation to be completed, (2) the operation has been disabled in NVSRAM (example, you are modifying 

media scan parameters when that option (offset 0x31, bit 5) is disabled), or (3) there is a problem with the 

storage array. Please check your storage array and its various components for possible problems and then retry the 

operation.

At Z:\VMshare\My Accounts\~POSH-UTILS\e-series-test.ps1:58 char:10

+ $tvObj = New-NeThinVolume  -Credential $NECred `

+          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: ({"poolId":"3233...ositorySize":4}:String) [New-NeThinVolume], NetAppP 

   owerShellException

    + FullyQualifiedErrorId : parameterError,Netapp.Santricity.PowerShell.NewThinVolumeCmdlet

... and since I don't have I tend to think my problem is the 3rd case (mostly because I had no ideas about (1) or (2). Then in looking around at things I stumbled across the fact that I've been running and re-running my short script above and IT HAS BEEN creating volumes for me but giving the above error in addition:

So I presumed some secondary aspect of the command was causing the error. Looking at trace.log didn't have any clues:

--- end of trace.log ---

12/05/2016 10:32:40 com.ni.eseries.aa.restapi.LoggingFilter.doFilter INFO: requestInfo|version:01.53.9000.0006|user:null|client:192.168.126.4|method:POST|url:/devmgr/v2/storage-systems/1/thin-volumes|query:null|status:422|time:188
12/05/2016 10:32:40 com.ni.eseries.aa.restapi.LoggingFilter.doFilter INFO: requestInfo|version:01.53.9000.0006|user:null|client:192.168.126.4|method:POST|url:/devmgr/v2/storage-systems/1/thin-volumes|query:null|status:422|time:188
12/05/2016 10:32:40 com.ni.eseries.aa.restapi.LoggingFilter.info INFO: requestDetail|{"poolId":"3233343536373839303132333100000000000000","name":"myVolume","sizeUnit":"gb","virtualSize":128,"repositorySize":4,"maximumRepositorySize":4}
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.info INFO: requestInfo|version:01.53.9000.0006|user:eseries|client:192.168.126.4|method:GET|url:/devmgr/utils/login|query:uid=eseries&pwd=netapp123|status:200|time:3
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.info INFO: requestInfo|version:01.53.9000.0006|user:null|client:192.168.126.4|method:GET|url:/devmgr/v2/storage-systems/1|query:null|status:200|time:2
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.info INFO: requestInfo|version:01.53.9000.0006|user:eseries|client:192.168.126.4|method:GET|url:/devmgr/utils/login|query:uid=eseries&pwd=netapp123|status:200|time:3
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.info INFO: requestInfo|version:01.53.9000.0006|user:null|client:192.168.126.4|method:GET|url:/devmgr/v2/storage-systems/1/ssc/pools/|query:null|status:200|time:4
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.info INFO: requestInfo|version:01.53.9000.0006|user:eseries|client:192.168.126.4|method:GET|url:/devmgr/utils/login|query:uid=eseries&pwd=netapp123|status:200|time:3
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.doFilter INFO: requestInfo|version:01.53.9000.0006|user:null|client:192.168.126.4|method:POST|url:/devmgr/v2/storage-systems/1/thin-volumes|query:null|status:422|time:270
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.doFilter INFO: requestInfo|version:01.53.9000.0006|user:null|client:192.168.126.4|method:POST|url:/devmgr/v2/storage-systems/1/thin-volumes|query:null|status:422|time:270
12/05/2016 10:35:48 com.ni.eseries.aa.restapi.LoggingFilter.info INFO: requestDetail|{"poolId":"3233343536373839303132333100000000000000","name":"myVolume","sizeUnit":"gb","virtualSize":64,"repositorySize":4,"maximumRepositorySize":4}

--- end snippet ---

... just seems to show my series of requests preceeding the New-NeThinVolume followed by the New-NeThinVolume calls (1st 128gb, then 64gb).

As cleanup I went into SANtricity StorMgr and deleted the bunch of volumes I'd created by selecting all 8 of them. Got this sequence of errors but it seemed to delete them anyway as they're gone from StorMgr and WSP as well. But does this error provide any clue as to why their original creation also generated the above error? 

I'm going to experiemtnt with WSP to create thinVolumes until I clear this up.

[Dave_Korns]

I went back and tested similar thinVolume creates using WSP, PowerShell-New-NeThinVolume and even WFA making the same call and all WORK (in that they create the volume) but all issue the same error message as above. 

Leads me to believe something about my environment ... so I next attempted to use SANtricity StorMgr to create a thinVolume. AMW -> Storage -> Volume -> Create -> shows all options grayed out. Is this a clue?

[khulques]

Keep in mind, the WSP is a proxy and the credentials listed in users.properties are specific to the proxy.  Array passwords are separate.  To more clearly separate the two concepts we created Get-NeProxyCredential.  While it's really just a helper function, script developers gave feedback that it helped differentiate the two and made thier work easier to read.  

The api to the SANtricity OS is called SYMbol.  This API provides two types of commands: active and passive.  An active command is any command that changes the configuration of the system by adding, deleting or modifying an object.  Passive commands list objects and provide telemetry data.  In the WSP the storage.rw role allows active and passive commands while storage.ro is limited to passive commands.  For the context of your work, stick with users assigned to the storage.rw role.

There are changes coming to the SANtricity OS that necessitate an admin concept.  I'm not clear on how the admin user is different from a user assigned to the storage.rw role.  I can find out details if it is important but for know my advice is to stick to storage.rw.

[khulques]

Sometimes strange things can happen with the simulator.  If you can make a new instance of the simulator and it behaves the same way, then it will be best to screen-share to do some detailed analysis.