FAS3020 Repourpose, adding shelf from DECOMM FAS940

[bheusmann]

I have a FAS3020 filer with 2 DS14MK2 shelfs 2 x ESH2 modules in each. There is currently no data on the disks in this system. I want to add a shelf from my Decomm'd FAS940. It's a DS14MK2 with 2 x ESH modules. Is this compatible? The shelf’s are double connected disk loops for redundancy & load balancing i.e.0a is connected via Fibre to Shelf 1 ESH2 Module A IN with shelf 2 & 3connected accordingly to module A. 0c is connected via Fibre to Shelf 1 ESH2 Module B IN with shelf 2 & 3 connected accordingly to B modules. Shelf 3 has the ESH modules withtermination switch set to on.

Currently Shelf 1 is showing both Loop A and B, shelf 2 and 3 are not. When in maintenance mode, it appears only Shelf 1 is identified by the filer head. Any help on this situation? Thanks.

-B

[rwelshman]

Can you check the firmware?

"To mix ESH and ESH2 in the same loop, the ESH modules must be running FW94 or later, and the ESH2 modules must be running FW13 or later."

[bheusmann]

Here is the output of the sysconfig which shows the ESH2 module. I'm not exactly sure what FW revision the ESH modules are running since it is not accessible at this point by the FAS3020. What I find odd is Shelf 2 which was part of the original FAS3020 system before being torn down is not even shown and it has ESH2 modules. I am going to rebuild the FAS940 with 3 shelf's and see if I can find out what FW version the ESH modules are, because I just can't remember. System name is removed:

NetApp Release 7.3.2P3: Fri Dec 11 18:10:24 PST 2009

        System ID: 0101178534 ()

        System Serial Number:  ()

        System Rev: B0

        System Storage Configuration: Multi-Path

        System ACP Connectivity: No Connectivity

        slot 0: System Board 2797 MHz (System Board X B1)

                Model Name:         FAS3020

                Part Number:        110-00035

                Revision:           B1

                Serial Number:     

                Firmware release:   CFE 3.0.0

                Agent FW version    15

                LCD FW version      1.7

                Processors:         2

                Processor ID:       0xf29

                Microcode Version:  0x2d

                Processor type:     Intel Xeon

                Memory Size:        2048 MB

        slot 0: Dual 10/100/1000 Ethernet Controller VI

                e0a MAC Address:    00:a0:98:03:22:e6 (auto-1000t-fd-up)

                e0b MAC Address:    00:a0:98:03:22:e7 (auto-unknown-cfg_down)

                e0c MAC Address:    00:a0:98:03:22:e4 (auto-unknown-cfg_down)

                e0d MAC Address:    00:a0:98:03:22:e5 (auto-unknown-cfg_down)

                Device Type:        Rev 3

                        memory mapped I/O base 0xa1a00000, size 0x20000

                        I/O base 0x0000bdc0, size 0x40

                        memory mapped I/O base 0xa1a20000, size 0x20000

                        I/O base 0x0000bd80, size 0x40

                        memory mapped I/O base 0xa0800000, size 0x20000

                        I/O base 0x0000cdc0, size 0x40

                        memory mapped I/O base 0xa0820000, size 0x20000

                        I/O base 0x0000cd80, size 0x40

                        memory mapped I/O base 0xa0200000, size 0x1000

                        memory mapped I/O base 0xa0201000, size 0x1000

                        memory mapped I/O base 0xa0a00000, size 0x1000

                        memory mapped I/O base 0xa0a01000, size 0x1000

        slot 0: FC Host Adapter 0a (Dual-channel, QLogic 2322 rev. 3, 64-bit, L-port, <UP>)

                Firmware rev:   3.3.27

                Host Loop Id:   7       FC Node Name:   5:00a:098200:00a123

                Cacheline size: 16      FC Packet size: 2048

                SRAM parity:    Yes     External GBIC:  No

                Link Data Rate: 2 Gbit

                 16  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ7915S000074410L1Y)

                 17  : NETAPP   X273_S15K4073F15 NA02  68.0GB 520B/sect (3KP331PR00007648FYH3)

                 18  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ792BE000074410M4F)

                 19  : NETAPP   X273_S15K4073F15 NA02  68.0GB 520B/sect (3KP3BR2P00007701F8M1)

                 20  : NETAPP   X272_HPYTA073F10 NA03  68.0GB 520B/sect (V3WS669A)

                 21  : NETAPP   X272_HJURE073F10 NA14  68.0GB 520B/sect (41457202)

                 22  : NETAPP   X272_HPYTA073F10 NA03  68.0GB 520B/sect (V3VR0EWA)

                 23  : NETAPP   X272_S10K7073F10 NA07  68.0GB 520B/sect (3KT4TK1400007617VNBT)

                 24  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ78ZGD000074410KF3)

                 25  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ78YV300007441173H)

                 26  : NETAPP   X272_HJURE073F10 NA14  68.0GB 520B/sect (41547231)

                 27  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ9JWVH00007518FA69)

                 28  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ79086000074410NBL)

                 29  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ792E1000074411WC9)

                Shelf 1: ESH2  Firmware rev. ESH A: 20  ESH B: 20

                        I/O base 0x0000be00, size 0x100

                        memory mapped I/O base 0xa1a40000, size 0x1000

        slot 0: FC Host Adapter 0b (Dual-channel, QLogic 2322 rev. 3, 64-bit, L-port, <OFFLINE (hard)>)

                Firmware rev:   3.3.27

                Host Loop Id:   0       FC Node Name:   5:00a:098300:00a123

                Cacheline size: 16      FC Packet size: 2048

                SRAM parity:    Yes     External GBIC:  No

                Link Data Rate: 1 Gbit

                        I/O base 0x0000bf00, size 0x100

                        memory mapped I/O base 0xa1a41000, size 0x1000

        slot 0: FC Host Adapter 0c (Dual-channel, QLogic 2322 rev. 3, 64-bit, L-port, <UP>)

                Firmware rev:   3.3.27

                Host Loop Id:   7       FC Node Name:   5:00a:098000:00a123

                Cacheline size: 16      FC Packet size: 2048

                SRAM parity:    Yes     External GBIC:  No

                Link Data Rate: 2 Gbit

                 16  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ7915S000074410L1Y)

                 17  : NETAPP   X273_S15K4073F15 NA02  68.0GB 520B/sect (3KP331PR00007648FYH3)

                 18  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ792BE000074410M4F)

                 19  : NETAPP   X273_S15K4073F15 NA02  68.0GB 520B/sect (3KP3BR2P00007701F8M1)

                 20  : NETAPP   X272_HPYTA073F10 NA03  68.0GB 520B/sect (V3WS669A)

                 21  : NETAPP   X272_HJURE073F10 NA14  68.0GB 520B/sect (41457202)

                 22  : NETAPP   X272_HPYTA073F10 NA03  68.0GB 520B/sect (V3VR0EWA)

                 23  : NETAPP   X272_S10K7073F10 NA07  68.0GB 520B/sect (3KT4TK1400007617VNBT)

                 24  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ78ZGD000074410KF3)

                 25  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ78YV300007441173H)

                 26  : NETAPP   X272_HJURE073F10 NA14  68.0GB 520B/sect (41547231)

                 27  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ9JWVH00007518FA69)

                 28  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ79086000074410NBL)

                 29  : NETAPP   X272_SCHT6073F10 NA16  68.0GB 520B/sect (3HZ792E1000074411WC9)

                Shelf 1: ESH2  Firmware rev. ESH A: 20  ESH B: 20

                        I/O base 0x0000ce00, size 0x100

                        memory mapped I/O base 0xa0840000, size 0x1000

        slot 0: FC Host Adapter 0d (Dual-channel, QLogic 2322 rev. 3, 64-bit, L-port, <OFFLINE (hard)>)

                Firmware rev:   3.3.27

                Host Loop Id:   0       FC Node Name:   5:00a:098100:00a123

                Cacheline size: 16      FC Packet size: 2048

                SRAM parity:    Yes     External GBIC:  No

                Link Data Rate: 1 Gbit

                        I/O base 0x0000cf00, size 0x100

                        memory mapped I/O base 0xa0841000, size 0x1000

        slot 0: SCSI Host Adapter 0e (Adaptec AIC7892, <ONLINE>)

                HW Version 0.0

                Ultra160 SCSI, Low Voltage Differential

                        I/O base 0x0000af00, size 0x100

                        memory mapped I/O base 0xa1c00000, size 0x1000

        slot 0: ATA/IDE Adapter 0f (0x000001f0)

                0f.0                 SGCF256S MA1NDW           SMICF+AB 249MB 512B/sect (2006122101A647E1)

                        I/O base 0x0000ffe8, size 0x8

                        I/O base 0x0000ffdc, size 0x4

                        I/O base 0x0000ffe0, size 0x8

                        I/O base 0x0000ffd8, size 0x4

                        I/O base 0x0000fff0, size 0x10

                        memory mapped I/O base 0xa0000000, size 0x400

        slot 1: NVRAM (NetApp NVRAM V)

                Revision:           H0

                Serial Number:     

                Memory Size:        512 MB

                Battery Status:     OK (3636 mV)

                Charger Status:     OFF

                Running Firmware:   7 (3.5.0)

                Cluster Interconnect Port 1:  disconnected

                Cluster Interconnect Port 2:  disconnected

                        memory mapped I/O base 0xa1800000, size 0x100000

                        prefetchable memory base 0xa1000000, size 0x800000

                        prefetchable memory base 0x80000000, size 0x20000000

        slot 3: Quad 10/100/1000 Ethernet Controller V

                e3a MAC Address:    00:04:23:c2:a3:40 (auto-unknown-cfg_down)

                e3b MAC Address:    00:04:23:c2:a3:41 (auto-unknown-cfg_down)

                e3c MAC Address:    00:04:23:c2:a3:42 (auto-unknown-cfg_down)

                e3d MAC Address:    00:04:23:c2:a3:43 (auto-unknown-cfg_down)

                Device Type:        Rev 1, PBA C84206-001

                        memory mapped I/O base 0xa0600000, size 0x20000

                        I/O base 0x0000dfc0, size 0x40

                        memory mapped I/O base 0xa0620000, size 0x20000

                        I/O base 0x0000df40, size 0x40

                        memory mapped I/O base 0xa0640000, size 0x20000

                        I/O base 0x0000df80, size 0x40

                        memory mapped I/O base 0xa0660000, size 0x20000

                        I/O base 0x0000df00, size 0x40

        slot 4: SCSI Host Adapter 4a (Adaptec AIC7902, <ONLINE>)

                HW Version 0.0

                Ultra320 SCSI, Low Voltage Differential

                        I/O base 0x0000ed00, size 0x100

                        memory mapped I/O base 0xa0400000, size 0x2000

                        I/O base 0x0000ef00, size 0x100

        slot 4: SCSI Host Adapter 4b (Adaptec AIC7902, <ONLINE>)

                HW Version 0.0

                Ultra320 SCSI, Low Voltage Differential

                        I/O base 0x0000ee00, size 0x100

                        memory mapped I/O base 0xa0402000, size 0x2000

                        I/O base 0x0000ec00, size 0x100

Thanks.

-B

[jb2cool01]

What version of ONTAP are you running ESH is no longer supported on 7.3

[bheusmann]

The FAS940 NetApp filer is running version 7.2.6.1 of ONTAP. Can the FAS3020 be downgraded from v.7.3 in order to utilize the additional shelf? Would this impact performace or even be worth it in the long run? I might be able to procure 2 ESH2 modules for the additional shelf, but I am not going to hold my breath. Thanks.

-B

[jb2cool01 · Accepted solution]

Check cabling if your other shelf is not showing up. Not sure what else it could be.

I believe that the FAS3020 can have any OS upwards of 7.0.1 so you should be able to have whatever version you want. The upgrade advisor seems to say you can just 'upgrade' to an older version but you could use the netboot method if this didn't work.

[jb2cool01]

Also i'm not 100% sure about this but when i had a mixed loop i had the ESH modules before the ESH2 modules in the loop.

[bheusmann]

After banging my head consistently on the desk, I rechecked cables and loop connections.....and by nothing other than a dumb mistake and grabbing the wrong Molex connector for the ESH cards.....it turns out I mistakenly didn't notice I was using the Molex connector for the ESH module in the ESH2 modules. Obviously, this was what was causing the shelf not to be recognized in the loop. Using the correct Molex connectors for the ESH2 modules alleviated the problem. Thanks, and glad it was just a simple cable mismatch.

-B

[jb2cool01]

Glad you got sorted. I was a bit worried about why it wasn't working for you.

[bheusmann]

Just an update, I downgraded ONTAP to v.7.2.7 I also moved the ESH module to slelf 1 and actually removed the ESH2 modules from the shelf's. All three shelfs are identified now.

One problem, I want to re-use the FAS940 (I have to beg/borrow/steal as it is now, so any waste of hardware even if it is EOL is bad). The problem I'm running into is I only have 1 set of ESH modules with FC connectors to connect to the head. Since the ESH2 modules at the top of the loop didn't seem to work, I'm not sure how to proceed or if I can? I have FC's for the ESH2 modules, but I'm not sure if the 940 will again, see the ESH2 modules at the beginning of the loop followed by ESH modules?

Still having some weird issues. When I had only the ESH2 modules in both slelf's, only 1 was available. It appeared that for some reason termination was set to "on" (I thought they were auto) and that made any other shelf's unavailable.

-Bryan

[jb2cool01]

Sorry, i'm getting mixed up as to what kit is where. Are all shelves now on the 3020? an you knock up a simple list view of what makes up this system and what you have left over.

[bheusmann]

Sorry, I know it's a lil confusing, I've been scatterd all over the place. Here's a list view of each system.

(1) FAS3020

(3) DS14MK2 Shelf's

(4) ESH2

(2) ESH

Left over Parts:

(1) FAS940

(6) ESH

(2) ESH w/LC

(4) DS14MK2 Shelf (2 are spare shelf units with power supply's only)

I believe the best end goal to utilize both systems is to have the FAS3020 with the 2 ESH2 modules and 1 ESH moduled terminating the the loop. This would allow me to use the remaining ESH module w/LC in the FAS940 terminating the loop with the last ESH module.

-Bryan

[aborzenkov]

I have FC's for the ESH2 modules, but I'm not sure if the 940 will again, see the ESH2 modules at the beginning of the loop followed by ESH modules?

It will. You may need to check loop speed settings in case you have 1Gb/s controllers.

for some reason termination was set to "on"

Sure it was termination and not shelf speed? Speed mismatch will cause shelf to drop off loop.

[bheusmann]

I have verified all loop speed settings are set to 2Gb via the switch on the shelf's and all modules are 2Gb capable.

As far as I know it is termination. The ESH2 modules are suposed to be auto terminating, so there's no switch on the back like the ESH module. Here's the output of the 'storage show hub' from maintenance mode:

*> storage show hub

Shelf name:    0a.shelf1         

Channel:       0a         

Module:        A

Shelf id:      1

Shelf UID:     50:05:0c:c0:02:10:5f:59

Term switch:   N/A

Shelf state:   ONLINE

Module state:  OK

                               Loop  Invalid  Invalid  Clock  Insert  Stall  Util    LIP

Disk    Disk     Port            up      CRC     Word  Delta   Count  Count  Percent Count

Id     Bay    State         Count    Count    Count                                  

---------------------------------------------------------------------------------------

[IN  ]          OK                0        0        0     16       0      0     0     0

[OUT ]      TERM              0        0        0      0       0      0     0     0

[  16]     0    OK                0        0        0     24       0      0     0     0

[  17]     1    OK                0        0        0     24       0      0     0     0

[  18]     2    OK                0        0        0     24       0      0     0     0

[  19]     3    OK                0        0        0      8       0      0     0     0

[  20]     4    OK                0        0        0      0       0      0     0     0

[  21]     5    OK                0        0        0     -8       0      0     0     0

[  22]     6    OK                0        0        0      0       0      0     0     0

[  23]     7    OK                0        0        0     32       0      0     0     0

[  24]     8    OK                0        0        0     16       0      0     0     0

[  25]     9    OK                0        0        0     16       0      0     0     0

[  26]    10    OK                0        0        0      0       0      0     0     0

[  27]    11    OK                0        0        0     40       0      0     0     0

[  28]    12    OK                0        0        0     32       0      0     0     0

[  29]    13    OK                0        0        0     24       0      0     0     0

Shelf name:    0c.shelf1         

Channel:       0c         

Module:        B

Shelf id:      1

Shelf UID:     50:05:0c:c0:02:10:5f:59

Term switch:   N/A

Shelf state:   ONLINE

Module state:  OK

                               Loop  Invalid  Invalid  Clock  Insert  Stall  Util    LIP

Disk    Disk     Port            up      CRC     Word  Delta   Count  Count  Percent Count

Id     Bay    State         Count    Count    Count                                  

----------------------------------------------------------------------------------------

[IN  ]          OK                0        0        0     16       0      0     0     0

[OUT ]      TERM              0        0        0      0       0      0     0     0

[  16]     0    OK                0        0        0     16       0      0     0     0

[  17]     1    OK                0        0        0     16       0      0     0     0

[  18]     2    OK                0        0        0     24       0      0     0     0

[  19]     3    OK                0        0        0      8       0      0     0     0

[  20]     4    OK                0        0        0      0       0      0     0     0

[  21]     5    OK                0        0        0     -8       0      0     0     0

[  22]     6    OK                0        0        0      0       0      0     0     0

[  23]     7    OK                0        0        0     24       0      0     0     0

[  24]     8    OK                0        0        0      8       0      0     0     0

[  25]     9    OK                0        0        0     16       0      0     0     0

[  26]    10    OK                0        0        0     -8       0      0     0     0

[  27]    11    OK                0        0        0     32       0      0     0     0

[  28]    12    OK                0        0        0     32       0      0     0     0

[  29]    13    OK                0        0        0     24       0      0     0     0

*>