Subscribe
Accepted Solution

Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

Strange issue I am experiencing at one of our sites.  vCenter 5, VSC 4.0.  Windows 2008 R2 VM, with Snapdrive 6.4.

Snapdrive service is running with a domain service account that has vCenter admin rights, and local admin rights on the VM, vCenter server, and VSC server.  Everything in Snapdrive is configured properly (I believe).  VM is running on an NFS datastore.

The basic issue is the VMDK drives don't show up in Snapdrive.  The entries from the Snapdrive log are below that I believe indicate the issue:

04/12-12:37:55.339PID:572TID:4596clrHelpers.h@1059
EnumVdisk method exception:-Failed to retrieve disks. Error FLOW-11019: Failure in ResolveCloneVirtualDiskInfoTask: java.lang.NullPointerException.
04/12-12:37:55.339PID:572TID:4596localerror.cpp@82
Is_VDS_ITF_Error(): false.
04/12-12:37:55.339PID:572TID:4596localerror.cpp@94
Is_WMI_ITF_Error(): false.
04/12-12:37:55.339PID:572TID:4596clrHelpers.h@1060
Failed to enumerate VMDK disks. Reasons Failed to retrieve disks. Error FLOW-11019: Failure in ResolveCloneVirtualDiskInfoTask: java.lang.NullPointerException..

HRESULT 0xc0041072.

In another site I have an identical setup (different servers, but a vCenter Server, VSC, VM, set up in the same way) and it works properly.  I should note that the above issue is not limited to a single VM; it affects any VM in the site that attempts to connect to the VSC.

I have tried reinstalling the VSC multiple times (on multiple servers; including directly on the vCenter server).  I've upgraded Java across the board.  I've reinstalled Snapdrive, tried different VM's, rechecked permissions.  I can't seem to find a config difference between my site that works and the site that doesn't.

Has anyone seen anything like this before?

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

It might be this https://kb.netapp.com/support/index?page=content&id=2015177&actp=LIST_RECENT&viewlocale=en_US&searchid=1334322961861

but the last time this type of thing happended it was because a vCenter database upgrade (4 to 5 upgrade) went bad.  Here is a link to that so you can look through the SMVI (VSC Backup and Recovery) logs to see if some of these issues are showing.  Before you go down this road, try to use the VSC to backup and restore the VMs to make sure that everthing there is working, because the fix is painful.  https://kb.netapp.com/support/index?page=content&id=2014919&actp=LIST_RECENT&viewlocale=en_US&searchid=1334322961861

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

I'm using SDW 6.3.1 but assuming things are more or less the same with vs 6.4.

Things to check for starters:

1. Check you have vmware tools installed in the VM, then check in the SDW gui overview screen that the 'System Type' is showing as a virtual machine, not a physical machine.

2. check you have configured the smvi server details (typically the vcenter server) using sdcli smvi_config list.  If not, or if it's incorrect, correct this.

3.  right click the snapdrive host in the GUI and check your vcenter settings

3. Restart the snapdrive service and check again.

The issue is that SD is not checking with vcenter

I've seen a strange issue to do with vmware tools' registry entries, but would need to dig the details out.  If the machine shows as a virtual machine, then this is not the problem.

Hope that helps,

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

System is definitely showing as an "ESX Virtual Machine".  Latest VM tools installed, SMVI server is configured properly, vCenter server set correctly.  I even tried putting in a bogus password to see if it would store it, but I correctly get an error that the credentials are bad. 

Snapdrive service has been restarted several times, and this is affecting multiple VM's at the site.  I believe because of this the issue is with either the VSC or vCenter.

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

Doesn't appear to be this.  I have none of these symptoms.  Oddly enough, VSC itself seems to be operating properly.  Until I tried uninstalling and reinstalling it, it WAS performing backups of VM's.

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

I would suggest you turn up logging on the VSC and VMware to debug, attempt to run a few backups, then gather those logs for NetApp Support to help you out. 

https://kb.netapp.com/support/index?page=content&id=1012550

https://kb.netapp.com/support/index?page=content&id=1011923

https://kb.netapp.com/support/index?page=content&id=1011824

They might also have you download nsanity from the toolchest and run that as well.

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

Some more data....definitely seems to be more VSC / SMVI Related.  I cleared the SMVI Logs and restarted the service, then restarted the snapdrive service on my VM.  This appears in the SMVI Logs:

2012-04-13 10:43:54,113 [discoverVirtualDisks:b12f1f7e24b4a1feef3a2772f84a3ca9:] ERROR - FLOW-11019: Failure in ResolveCloneVirtualDiskInfoTask: java.lang.NullPointerException

java.lang.NullPointerException

          at com.netapp.smvi.zapi.LunMapper.getLunSet(LunMapper.java:174)

          at com.netapp.smvi.zapi.LunMapper.put(LunMapper.java:170)

          at com.netapp.smvi.zapi.LunMapper.get(LunMapper.java:60)

          at com.netapp.smvi.zapi.LunMapper.get(LunMapper.java:53)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.createVDiskBackingInfo(ResolveCloneVirtualDiskInfoTask.java:381)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.createVDiskInfo(ResolveCloneVirtualDiskInfoTask.java:269)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.createVDiskInfo(ResolveCloneVirtualDiskInfoTask.java:263)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.execute(ResolveCloneVirtualDiskInfoTask.java:247)

          at com.netapp.common.flow.TaskInstanceTemplate.execute(TaskInstanceTemplate.java:336)

          at com.netapp.common.flow.Operation.executeCurrentStack(Operation.java:146)

          at com.netapp.common.flow.Operation.execute(Operation.java:61)

          at com.netapp.common.flow.Threadpool$OperationThread.run(Threadpool.java:257)

2012-04-13 10:43:54,113 [discoverVirtualDisks:b12f1f7e24b4a1feef3a2772f84a3ca9:] ERROR - FLOW-11008: Operation failed: java.lang.NullPointerException

java.lang.NullPointerException

          at com.netapp.smvi.zapi.LunMapper.getLunSet(LunMapper.java:174)

          at com.netapp.smvi.zapi.LunMapper.put(LunMapper.java:170)

          at com.netapp.smvi.zapi.LunMapper.get(LunMapper.java:60)

          at com.netapp.smvi.zapi.LunMapper.get(LunMapper.java:53)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.createVDiskBackingInfo(ResolveCloneVirtualDiskInfoTask.java:381)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.createVDiskInfo(ResolveCloneVirtualDiskInfoTask.java:269)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.createVDiskInfo(ResolveCloneVirtualDiskInfoTask.java:263)

          at com.netapp.smvi.task.filerestore.ResolveCloneVirtualDiskInfoTask.execute(ResolveCloneVirtualDiskInfoTask.java:247)

          at com.netapp.common.flow.TaskInstanceTemplate.execute(TaskInstanceTemplate.java:336)

          at com.netapp.common.flow.Operation.executeCurrentStack(Operation.java:146)

          at com.netapp.common.flow.Operation.execute(Operation.java:61)

          at com.netapp.common.flow.Threadpool$OperationThread.run(Threadpool.java:257)

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

Hmmm, strange...

How about in VSC -> Backup and Recovery -> Setup.  Check your vcenter server, port and user.  check the user has correct rights in vcenter roles.  Also worth checking in vcenter for roles applied at different levels within the hierarchy (eg at DC level, cluster, folder, etc) as these can override inherited rights. 

Have you checked VSC plugin is correctly registered with vcenter?  eg http://<vcenter_server>:8143/Register.html ?

Sorry if you've checked this already or if I'm teaching you to suck eggs.

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

I appreciate the extra eyes in making sure I haven't missed something.  Don't worry about that

vCenter server in VSC -> Backup and Recovery -> Setup is specified by IP address.  The account has admin privs at the vCenter level.  It was using a different account than my "working" setup, so I duplicated the exact settings here and they were accepted.

Registration page shows correct registration.  Decided to change it to register by hostname instead of IP just to see what would happen.  So far same results.

Re: Snapdrive unable to communicate with VSC: Java.lang.nullpointerexception

I think I have this solved.  The VM's had Raw LUN Mappings to a non-netapp array (EqualLogic); once these were removed everything worked properly.  Not sure why this would be a problem but it's not going to be a roadblock to my project (we're in the process of migrating off the EqualLogic's anyway).