Intermittent System.IO.FileNotFoundException from Get-NaHostDisk and Get-NaHyperV

[hugh_kelley]

I have a script that runs hourly inside our monitoring platform and occasionally (script runs once an hour, condition occurs once every few days) I get this exception

System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)

from either of these iterations:

foreach($disk in Get-NaHostDisk)

{}

foreach($disk in (Get-NaHyperV $vm.Name).Storage)

{}

It seems to self-resolve the next time the script runs and it happens on multiple servers (all running SMHV).  Any ideas why?

Hugh

[timothyn]

Do you have disks adding/removing or moving between servers on a regular basis?

The next time you see that, please run this command and paste the output here:

     $Error[0] | fl -force

That'll give us a stack trace with more info to debug.  Thanks!

[hugh_kelley]

It took a few weeks for the condition to return, but as requested:

Exception             : System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
                           at DataONTAP.HostUtilities.Vds.IVdsDisk.GetIdentificationData(VdsLunInformation& lunInfo)
                           at DataONTAP.HostUtilities.DiskDiscovery.DiskService.GetDiskResource(VdsDisk vdsDisk)
                           at DataONTAP.HostUtilities.DiskDiscovery.DiskService.GetDiskResources(IEnumerable`1 vdsDisks)
                           at DataONTAP.HostUtilities.DiskDiscovery.DiskService.GetAllDiskInfo()
                           at DataONTAP.PowerShell.SDK.Cmdlets.Windows.GetHostDisk.ProcessRecord()
                           at System.Management.Automation.CommandProcessor.ProcessRecord()
TargetObject          : 
CategoryInfo          : NotSpecified: (:) [Get-NaHostDisk], FileNotFoundException
FullyQualifiedErrorId : System.IO.FileNotFoundException,DataONTAP.PowerShell.SDK.Cmdlets.Windows.GetHostDisk
ErrorDetails          : 
InvocationInfo        : System.Management.Automation.InvocationInfo
PipelineIterationInfo : {0, 1}
PSMessageDetails      : 

[quincybowers]

I frequently get the exact same error when running Get-NaHostDisk.  Running a second time generally works.  But this delays things significantly.

Here is the stack trace I get:

Exception             : System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRE
                        SULT: 0x80070002)
                           at DataONTAP.HostUtilities.Vds.IVdsDisk.GetIdentificationData(VdsLunInformation& lunInfo)
                           at DataONTAP.HostUtilities.Vds.VdsDisk.GetLunInformation()
                           at DataONTAP.HostUtilities.DiskDiscovery.DiskService.GetDiskResource(VdsDisk vdsDisk)
                           at DataONTAP.HostUtilities.DiskDiscovery.DiskService.GetDiskResources(IEnumerable`1 vdsDisks
                        )
                           at DataONTAP.HostUtilities.DiskDiscovery.DiskService.GetAllDiskInfo()
                           at DataONTAP.PowerShell.SDK.Cmdlets.Windows.GetHostDisk.ProcessRecord()
                           at System.Management.Automation.CommandProcessor.ProcessRecord()
TargetObject          :
CategoryInfo          : NotSpecified: (:) [Get-NaHostDisk], FileNotFoundException
FullyQualifiedErrorId : System.IO.FileNotFoundException,DataONTAP.PowerShell.SDK.Cmdlets.Windows.GetHostDisk
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
PipelineIterationInfo : {0, 1}
PSMessageDetails      :

And here is some Powershell I have written to get past the issue for the time being.

$retries = 3
$success = $false 
while ((-not $success) -and ($retries -gt 0))
{
    $retries -= 1
    "Attempting to run [Get-NaHostDisk -DataOntap].  $retries retries remaining."
    try
    {
        $NaHostDisks = Get-NaHostDisk -DataOntap
        $success = $true
    }
    catch
    {
        "ERROR::$($_.Exception.ToString())`nWhile executing Get-NaHostDisk."
    }
}
if (-not $success)
{
    throw "Unable to get the HostDisk mapping from the NetApp."
}

A better solution to this would be appreciated.