Subscribe

E-Series SRA plug-in fails to enumerate paired devices

[ Edited ]

E5500 iSCSI arrays. Firmware up to date, every software is in matrix and so on.

 

After successfully installed and fine configured the array manager pairs (see the issue about ports to solve before here) there's no change to configure the protection group because none of the two array manager can see and enumerate the paired devices (I do not report here screenshot of SANTricity-trust on me, everything is good and working and AMG work fine).

 

All the VMware kb about this invite to check connectivity issues and running perfstat on both SRM servers reports that each needed port and comm is ok.

As you can see the pair of arrays and their manager is ok and enabled but there are no parired devices listed below and also rescanning give no results and the same error.

Here is an abstract from log. I read that the process starts, enumerates some LUN and can declare them not associated to AMG (right), discover some AMG group (the name is correct) but then it hangs and stops wiht that error "null pointer exception".

 

This happens on both site!



I've a case opened...investigating. But I suspect is a SRA sw issue to fix.

 

23-02-2015 17-16-36.jpg

 

23-02-2015 17-17-24.jpg

 

 

2015-02-23T13:48:42.272+0100 SRA Cmd000031  verbose retreived DiscoverDevicesParameters  :: RequestHandler.getParams 424, RequestHandler.getParams 415

2015-02-23T13:48:42.270+0100 SRA Cmd000031  info    NetApp Storage Replication Adapter for E-Series version 05.50.3000.0005 built 2014.05.14.08.49.51  :: CommandProcessorImpl.init 102, CommandProcessorImpl.go 75

2015-02-23T13:48:42.270+0100 SRA Cmd000031  verbose  command :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<Command xmlns="http://www.vmware.com/srm/sra/v2">

    <Name>discoverDevices</Name>

    <OutputFile>C:\Windows\TEMP\vmware-SYSTEM-3181398299\sra-output-106-95</OutputFile>

    <StatusFile>C:\Windows\TEMP\vmware-SYSTEM-3181398299\sra-status-107-57</StatusFile>

    <LogLevel>verbose</LogLevel>

    <LogDirectory>C:\ProgramData\VMware\VMware vCenter Site Recovery Manager\Logs\SRAs\NetApp E-Series</LogDirectory>

    <Connections>

        <Connection id="Array">

            <Addresses> this is ok!

                <Address id="IP1">10.254.149.31</Address>

                <Address id="IP2">10.254.149.32</Address>

            </Addresses>

        </Connection>

        <Connection id="PeerArray">

            <Addresses> this is ok!

                <Address id="IP1">10.254.249.31</Address>

                <Address id="IP2">10.254.249.32</Address>

            </Addresses>

        </Connection>

    </Connections>

    <DiscoverDevicesParameters>

        <ArrayId>60:08:0E:50:00:29:AE:EC:00:00:00:00:54:40:20:52</ArrayId>

        <PeerArrayId>60:08:0E:50:00:29:9B:40:00:00:00:00:54:41:95:76</PeerArrayId>

    </DiscoverDevicesParameters>

</Command>  :: CommandProcessorImpl.dispatchCommand 133, CommandProcessorImpl.go 77

2015-02-23T13:48:42.272+0100 SRA Cmd000031  info    Wrote C:\SRM\storage\sra\NetApp E-Series\.\tracks\2015-02-23_13-48-42-270_discoverDevices_c.xml  :: CommandProcessorImpl.trackRequest 310, CommandProcessorImpl.dispatchCommand 143

2015-02-23T13:48:42.272+0100 SRA Cmd000031  info    Processing discoverDevices request.

   :: CommandProcessorImpl.dispatchCommand 151, CommandProcessorImpl.go 77

2015-02-23T13:48:44.134+0100 SRA Cmd000031  info    StorageDeviceAccessor formed successfully for CnxParams[Array|10.254.149.31|...32]  :: RequestHandler.createAndInitializeSda 181, RequestHandler.setupArrayAccess 292

2015-02-23T13:48:44.134+0100 SRA Cmd000031  info    established access for array with id 60:08:0E:50:00:29:AE:EC:00:00:00:00:54:40:20:52  :: RequestHandler.setupArrayAccess 320, RequestHandler.setupArrayAccess 268

2015-02-23T13:48:45.255+0100 SRA Cmd000031  info    is ARVM supported ==>true  :: StorageDeviceAccessorImpl.getRvmInfoMap 881, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:45.253+0100 SRA Cmd000031  info    StorageDeviceAccessor formed successfully for CnxParams[PeerArray|10.254.249.31|...32]  :: RequestHandler.createAndInitializeSda 181, RequestHandler.setupArrayAccess 292

2015-02-23T13:48:45.253+0100 SRA Cmd000031  info    established access for array with id 60:08:0E:50:00:29:9B:40:00:00:00:00:54:41:95:76  :: RequestHandler.setupArrayAccess 320, RequestHandler.setupArrayAccess 268

2015-02-23T13:48:45.253+0100 SRA Cmd000031  trivia  array name is ITROMFS31  :: RequestHandler.setupArrayAccess 358, RequestHandler.setupArrayAccess 268

2015-02-23T13:48:45.253+0100 SRA Cmd000031  info    access setup for array ITROMFS31  :: RequestHandler.setupArrayAccess 360, RequestHandler.setupArrayAccess 268

2015-02-23T13:48:45.253+0100 SRA Cmd000031  trivia  peer array name is ITNAPFS31  :: RequestHandler.setupArrayAccess 366, RequestHandler.setupArrayAccess 268

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  concatVolume repos_0009 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX01 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX02 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  concatVolume repos_0001 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  concatVolume repos_0002 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  concatVolume repos_0003 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  concatVolume repos_0005 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  concatVolume repos_0006 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  concatVolume repos_0007 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.081+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX03 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX04 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.085+0100 SRA Cmd000031  trivia  discovering AsyncMirrorGroupMembers of AsyncMirrorGroup :  AMG-NA_DR  :: DiscoverDevicesHandler.go 129, CommandProcessorImpl.dispatchCommand 158 found something? Name is ok

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX05 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX06 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX07 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX08 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX09 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX10 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0000 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0004 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0008 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0022 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0010 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0011 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0012 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0013 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0014 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0015 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0016 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0017 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0020 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0019 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0021 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0023 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0024 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0025 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0026 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.082+0100 SRA Cmd000031  trivia  concatVolume repos_0027 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0028 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0018 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0029 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX11 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  standardVolume SAS_ESX12 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0049 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0058 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0059 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0066 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0047 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0048 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0068 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0070 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0071 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0072 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0073 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0074 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0075 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0085 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0086 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0083 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0084 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.083+0100 SRA Cmd000031  trivia  concatVolume repos_0088 is not participating in a mirror pair  :: StorageDeviceAccessorImpl.getRvmInfoMap 980, RequestHandler.setupArrayAccess 372

2015-02-23T13:48:51.086+0100 SRA Cmd000031  error   NullPointerException while Exception while handling command discoverDevices. Exception Message NullPointerException while .  :: CommandProcessorImpl.dispatchCommand 218, CommandProcessorImpl.go 77

java.lang.NullPointerException here stops and hangs

      at com.loem.devicemgt.ByteUtils.getWwnString(ByteUtils.java:178)

      at com.netapp.apg.sra.svc.handlers.RequestHandler.createArvmReplicationProps(RequestHandler.java:491)

      at com.netapp.apg.sra.svc.handlers.DiscoverDevicesHandler.go(DiscoverDevicesHandler.java:155)

      at com.netapp.apg.sra.svc.CommandProcessorImpl.dispatchCommand(CommandProcessorImpl.java:158)

      at com.netapp.apg.sra.svc.CommandProcessorImpl.go(CommandProcessorImpl.java:77)

      at com.netapp.apg.sra.svc.SraServiceImpl.processCommand(SraServiceImpl.java:233)

      at com.netapp.apg.sra.svc.SraServiceImpl.handleClientRequest(SraServiceImpl.java:186)

      at com.netapp.apg.sra.svc.SraServiceImpl$2.run(SraServiceImpl.java:149)

      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

      at java.lang.Thread.run(Unknown Source)

2015-02-23T13:48:51.086+0100 SRA Cmd000031  verbose  response :

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<Response xmlns="http://www.vmware.com/srm/sra/v2">

    <Error code="1002">

        <ntap:ExceptionMsg xmlns:ntap="http://www.netapp.com/apg/sra/v2">NullPointerException while </ntap:ExceptionMsg>

        <ntap:Command xmlns:ntap="http://www.netapp.com/apg/sra/v2">discoverDevices</ntap:Command>

    </Error>

</Response>  :: CommandProcessorImpl.dispatchCommand 235, CommandProcessorImpl.go 77

2015-02-23T13:48:51.088+0100 SRA Cmd000031  info    Wrote C:\SRM\storage\sra\NetApp E-Series\.\tracks\2015-02-23_13-48-42-270_discoverDevices_r.xml  :: CommandProcessorImpl.trackResponse 358, CommandProcessorImpl.dispatchCommand 241

Re: E-Series SRA plug-in fails to enumerate paired devices

Solved? I don't thinkl so, this is just a workaround, a bad one...

See here.