ONTAP Discussions

Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

netapple

Hi all,

 

we run a Powershell script to create snapshots from Windows 2012 R2 machine on a FAS2554. It ran flawlessly until last week when we upgraded the FAS Ontap Version. Upgrade path was as follows: 9.3P12 -> 9.5P18 -> 9.6P9.

Now the script fails and as I found it can't even connect to the FAS. A simple Connect-NcController fails with error "API invoke failed". Other facts are:

 

- Authentication works, because when I type wrong credentials, I get a corresponding error message

- I tried https and http and both are enable on the target FAS

- A "Test-Connection" either to port 80 or 443 is successful

- Certificates have been renewed and are not expired

 

After trying everything I can think of, I am completely at a loss. Any help would be greatly appreciated.

 

Thanks

Dan

 

12 REPLIES 12

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

Ontapforrum

Are there any ems logs (related to ssl) on the controller side giving any hints ?

 

Not sure if this is the cause, but no harm in checking this out.

 

BUG:
https://mysupport.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=1186901

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

netapple

Thanks for your response. I doubt that this is the problem because OCUM for example (which is now AIQUM) is not affected, all certs have already been recreated and connection over HTTP (which should ignore the certs) gives the same error.

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

netapple

What is ems log? I can do an event show but it does not show any messages regarding my connection attempts. I don't know of any other logs to look at.

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

Ontapforrum

By EMS, I simply meant 'event logs'.

 

It may be worth checking as well.
/mlog/audit.log (As it contains API related events)


I suggest: Please follow the steps to set the debug logs and re-produce that error, this will give us more information to find the root cause.
How to collect debug logs for the Data ONTAP PowerShell Toolkit:
https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/How_to_collect_debug_logs_for_the_Data_ONTAP_PowerShell_Toolkit

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

netapple

Thanks for your further suggestions. Unfortunately there is nothing in the audit.log.

 

We already tried the debug level of Powershell toolkit but that did not reveal any helpful information.

 

Meanwhile I heared of someone else trying to use the toolkit against an Ontap 9.6P9 using an ontap simulator and he experiences the exact same problem. Seems to me, they have changed something in that release, because her reported it works fine with 9.6.P6 or P4 for example.

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

netapple

Btw debug log gives the following output (just tried it with specifying SSLVersion but without the output is the same):

 

2020-08-07 11:51:04,731 INFO DataONTAP.C.PowerShell.SDK.ConnectNcController - Cmdlet invocation: Connect-NcController 1
0.99.16.51 -SSLVersion "TLSv1.2"
2020-08-07 11:51:04,751 DEBUG NetApp.Ontapi.NaServer - Application name set to PowerShell Toolkit
2020-08-07 11:51:04,751 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - No credentials provided, consulting cache

2020-08-07 11:51:13,769 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - Trying HTTP/HTTPS
2020-08-07 11:51:13,775 DEBUG NetApp.Ontapi.NaServer - NaServer port changed to 443
2020-08-07 11:51:13,776 DEBUG NetApp.Ontapi.NaServer - NaServer Protocol changed to HTTPS
2020-08-07 11:51:13,776 DEBUG NetApp.Ontapi.NaServer - Proxy changed to
2020-08-07 11:51:13,777 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - Connecting to <ipaddress> via HTTPS
2020-08-07 11:51:13,845 DEBUG NetApp.Ontapi.NaServer - <netapp version='1.0' xmlns='http://www.netapp.com/filer/admin'><
system-get-version /></netapp>

2020-08-07 11:51:44,051 DEBUG NetApp.Ontapi.NaServer - NaServer port changed to 80
2020-08-07 11:51:44,051 DEBUG NetApp.Ontapi.NaServer - NaServer Protocol changed to HTTP
2020-08-07 11:51:44,051 DEBUG NetApp.Ontapi.NaServer - Proxy changed to
2020-08-07 11:51:44,051 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - Connecting to <ipaddress> via HTTP
2020-08-07 11:51:44,052 DEBUG NetApp.Ontapi.NaServer - <netapp version='1.0' xmlns='http://www.netapp.com/filer/admin'><
system-get-version /></netapp>

 

Connect-NcController : API invoke failed.
At line:1 char:1
+ Connect-NcController <ipaddress> -SSLVersion "TLSv1.2"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidResult: (<ipaddress>:NcController) [Connect-NcController], NaException
+ FullyQualifiedErrorId : HttpConnectionFailed,DataONTAP.C.PowerShell.SDK.ConnectNcController

2020-08-07 11:52:14,266 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - Failed to connect to <ipaddress>
2020-08-07 11:52:14,268 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - Setting value of global:CurrentNcControll
er
2020-08-07 11:52:14,268 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - Starting EMS logging
2020-08-07 11:52:14,268 DEBUG DataONTAP.C.PowerShell.SDK.ConnectNcController - Failed to complete EMS logging
System.NullReferenceException: Object reference not set to an instance of an object.
at DataONTAP.C.PowerShell.SDK.ConnectNcController.ProcessRecord(String name)

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

Ontapforrum

Output of the following command ?

 

::> system services web show

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

netapple

Good point but already checked. Here's the output anyway:

 

External Web Services: true
Status: online
HTTP Protocol Port: 80
HTTPS Protocol Port: 443
HTTP Enabled: true

 

Meanwhile I heard from Netapp support that toolkit 9.7.1 is missing some DLLs that are planned to be implemented in 9.8. Quite unspecific and vague but at least an explanation. It also means there is no point in further investigation for me/us.

 

Thanks all for your hints and help.

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

Ontapforrum

Interesting info, indeed vague. How did it work earlier before the 9.6P9? That means, those missing DLLs didn't really matter. So, something have change on 9.6P9 rather than PS Tool kit version.

 

Anyway, usually, i just look for up bugs in the BUG TOOL for that particular ONTAP version and then I would raise a ticket with NetApp, which I guess you already did and you heard from them as well.

 

For 9.6P9 there is only one bug listed

https://mysupport.netapp.com/site/bugs-online/product/ONTAP/BURT/1342292

 

I don't know if this is related anyway but have a good weekend!

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

parisi

Someone else (actually 2 in the same thread) mentioned a similar issue with P9:

 

https://community.netapp.com/t5/ONTAP-Discussions/ontap-api-not-working-with-veeam-backup-and-PRTG-after-updating-to-ontap9-6P9/td-p/158095

 

So yes, it seems like perhaps something got broken in that patch. A case is the best route here.

 

In the meantime I'll spin up a VSIM of the release and see if I can figure it out.

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

parisi

Ok, this is a confirmed bug in P9.

 

When you open a case, have the TSE add it to bug 1342292. We'll have a public report soon.

 

There doesn't seem to be a workaround for this other than reverting to the previous P release or moving to the latest 9.7 release.

View solution in original post

Re: Powershell Toolkit "API invoke failed" after upgrade to ONTAP 9.6P9

netapple

Just wanted to let you know. We rolled back to 9.6P8 and now the toolkit works again as expected.

Earn Rewards for Your Review!
GPI Review Banner
All Community Forums
Public