CloudInsights very occasionally returning Vmdks with IDs that don't map to anything

[Sam-Miller]

Hi all, I've written a .NET Console app that imports in data from CloudInsights into our SQL database. It's making about 80,000 HTTP requests to the API that are throttled to no more than 50 consecutive requests. Once in a blue moon on a Friday the thirteenth, I get the following error response to all of my requests related to Vmdks:

https://xxxxxx.xxx.cloudinsights.netapp.com/rest/v1/assets/vmdks/3686379087605277227/performance?fromTime=1627302704044&toTime=1627303064044
Method: GET, RequestUri: 'https://xxxxxx.xxx.cloudinsights.netapp.com/rest/v1/assets/vmdks/3686379087605277227/performance?fromTime=1627302704044&toTime=1627303064044', Version: 1.1, Content: <null>, Headers:
{
X-CloudInsights-ApiKey: XXXXX
}
{
"errorCode": "NOT_FOUND",
"errorMessage": "Could not find virtual machine disk for id: 3686379087605277227"
}

 When I test out this request in Postman, it would appear that no Vmdk with that ID even exists. Any idea why the API would occasionally return Vmdks with IDs that don't map to anything? It's a very infrequent issue and only occurred maybe 3 times in the 150+ times I've run this program. What's strange is if I immediately rerun the program, it will have no issues. It happens so infrequently that it's not a massive issue, but it still something I'd like to resolve. My program runs hourly and if an execution fails for an hour, that hour's data will be missing. Thanks for the help!

[Sidd · Accepted solution]

Hello Sam, I hope your week is going well.

As to why you are getting an invalid vmdk ID, my guess is that something happened to that VMDK - it may have been deleted. Are you querying for all VMDKs and then getting the performance of each one individually? It may make sense to record the name and then follow up with your VMware team to see what happened to that VMDK, or to see if its still in CI, but its ID has changed. Apologies for not having a clear answer for this.

Another note is that you can always go back and query the hour you missed, or skip the individual failures and log them instead of failing.

As an aside, I see that you mentioned you are running a lot of requests. Have you looked at other methods of retrieving the data you are looking for, using a query?  For example, if you want to get VMDK performance, instead of querying each one individually you can use a query and paginate the results. Here is an example API call to get 100 VMDKs with performance:

https://xxxxxx.xxx.cloudinsights.netapp.com/rest/v1/query?objectType=Vmdk&fields=performance&limit=100&offset=0

In the response you will have the number of results so you can loop through and update the offset until you have queried all of the VMDKs (i.e. next call would be limit=100&offset=100). You can also add a fromTime and toTime parameters to limit the performance data to your desired window.

Using a query should also remove the possibility of querying an invalid ID.

I hope this helps you, let me know if you have any other questions regarding this information.

-Sidd

[Sam-Miller]

Hi Sidd,

Thank for the response! Good idea using the query endpoint. I'd imagine it'd reduce the number of requests by about 20K or so. I'd still need to do a bunch of individual requests for some entities. For example, I'm pulling down what Vmdks are mapped to each DataStore. I'm doing this by using 

https://xxxxxx.xxx.cloudinsights.netapp.com/rest/v1/dataStores/{id}/vmdks

I'm still quite new to NetApp so there could be a better way to do this. What do you think?

(P.S. I'm throttling these API requests down to 50 consecutive requests. Even though there's 80K, I find it's relatively stable and only have the occasional retry)

[Sidd]

The concept in the Cloud Insights API that can really help you here is expanding related assets, you can find it in the docs here:

https://docs.netapp.com/us-en/cloudinsights/API_Overview.html#expands

However, what is not obvious is that you can do the same in a query using the "fields" parameter. So you can query DataStores, then expand VMDKs, and then for VMDKs expand performance. Just keep in mind that performance expands can be expensive and if you expand too much that will also slow down the call. However the tradeoffs are usually worth it (fewer "large" API calls vs. more "small" API calls). I also advise to expand down the tree where you can, from one to many; i.e Storage -> StoragePool, VM -> DataStore and not the other way around.

Here is a call using a query to get DataStores and expand VMDK performance for each one:

https://xxxxxx.xxx.cloudinsights.netapp.com/rest/v1/query?objectType=DataStore&fields=vmdks.performance&limit=10&offset=0

You can also do the same with a DataStore call for a given ID, using yours as an example:

https://xxxxxx.xxx.cloudinsights.netapp.com/rest/v1/dataStores/{id}/vmdks?expand=performance
-OR-
https://xxxxxx.xxx.cloudinsights.netapp.com/rest/v1/dataStores/{id}?expand=vmdks.performance

There will a difference in how the data is returned, the first will be a list of vmdks with performance, the second will be the DataStore information with an entry "vmdks" which will contain the vmdks with performance.

-Sidd