Thanks for the response. I need to check on the possibility of sharing the code. Will get back on this by next week. Also, I have a question w.r.t verifying the execution of the API calls with the newer version that you pointed me to. I understand that the version is SDK 5.0. Can you please let me know if verifying the API call execution with apitest.exe of SDK 5.0 is equivalent to verifying the same with 5.0 version of manageontap.jar in the java code?
apitest is a command-line utility to test APIs. This utility is suitable for API users who are at the beginner's level.
Verifying with API test should be equivalent to verifying with the manageontap.jar.
Your error could be related to passing incorrect XML parameters, that's why I asked for the code. Also, you could test your APIs on ZEDI that comes as a part of the package you have. It generates complete code which should be correct. The code I gave in my first comment is a complete working one from ZEDI. So you could refer that or share your code with me so that we could solve this.
I am Prasanna's collegue and would be taking this up further. As Prasanna had mentioned, we might not be able to share the exact piece of code, but we do have the apitest and the Z-Explorer outputs. All the APIs were run using 4.0 and 5.0 SDK versions. Please find the same attached with this post.
apitest_50_Raw Outputs.zip - contains API outputs, run using SDK 5.0
apitest_Raw_Outputs.zip - contains API outputs, run using SDK 4.0
In the raw output (I opened aggr-space-list-info_raw_50.log) it stops at
A few suggestions...
Can you run a packet trace from data ONTAP (pktt start all -i X.X.X.X -d /vol/volume) where X.X.X.X is the IP address you're running apitest from and volume is replaced with an actual volume on the controller that has space to capture a packet trace.Issue the zapi with HTTP (not https as that complicates using a packet trace). Then stop the packet trace (pktt stop all). You can just use one command and one version that returns this error. Then attach here along with the output the packet trace from the controller. I'm interested to see if this is the same data that left the controller in to the network (by gathering a packet trace).
I'd also caution against programmatically using just -info API calls if there are -iter & -next APIs for the same information. As the number of all resource types (volumes, aggregates, shares/exports, LUNs, whatever) on a system grows, I've seen complications arise out of only grabbing a large bucket of output with just -info when -iter and -next would be better. For this type of call (using aggr-space-list-info as an example) you'd do better to use the aggr-list-info to build an array of the aggregates and then call aggr-space-list-info for each aggregate. Please let me know if you see individually called aggregates a way to resolve this error.
It looks like any replies that hit the MTU size (1514) gets truncated XML. I had a colleague point out that the PSH flag from TCP also tells the recipient to go ahead and process the data, don't wait for more.
I'll have to defer to some of the developers on how to address that or if it's already addressed somewhere.
I filed all the details under bug 728756. The quick solutions would be jumbo frames (9k MTU) or using segmentation like I mentioned before (meaning build an array of the aggregate and then just query each individual aggregate).
Note: I just put that out there for tracking purposes. One of the other developers might know if we've already addressed this somewhere. 7.3 code isn't exactly new. ONTAP 8.x has been around a bit now.
I cross-checked out code, and we already implement the segmentation approach, as suggested in your previous post. Here is the list of APIs for which we are using -iter and other APIs:
Regarding use of jumbo frames, could you please be little more descriptive. I am not aware of how to implement this in the code, in order to avoid the truncation of the output. Could you please provide some sample code which explains using jumbo frames?
Jumbo frames would be a network configuration on the NICs of both the storage, the network, and the source issuing the API calls. We're investigating this further, but it seems these problems only happen in the packet trace you sent me when you have a response that is larger than 1 MTU/MSS (network interface PDU size). In your packet trace, Data ONTAP has a 1500 MTU set. When the zapi response is bigger than 1 MTU/MSS, the output is truncated. Jumbo frames (9k MTU) are fairly well known now even if they are technically considered nonstandard MTU. This would increase the amount of data that could be sent back from within Data ONTAP to 9k.
Changing your network communications channel between wherever you are issuing API calls from and to Data ONTAP may not be easy.
The easier suggestion would be to write the code to ask for smaller chunks of data. Build the object list within your code and then just query individual object details instead of asking for all of them at once (like in the aggr-list-space-info query that you provided the output for).
You called aggr-list-space-info with no aggregate value (effectively requesting the entire list). If you issued multiple commands for aggr-list-space-info aggr1 aggr-list-space-info aggr2...etc then the responses I believe could fit within the single MTU and then the problem is worked around without infrastructure changes.