On Board Fibre change

stephenaponte · ‎2010-03-26

Good afternoon, hope all is well. I just wanted to get some thoughts on a project I am currently working on.

I am adding Disk Sheleves on a Filer and implementing Fibre SAN on the Filer for the first time. It currently has no Fibre connections available as the Onbaord is being used by the shelves and all I have is a HBA 4 Port Shelve card.

This paticular Filer has four LUNS. I need to:

a. Add a 4 Port Fibre card to a slot.

b. Move the Shelve Fibre cable from the onboard Fibre Ports and redirect them to the newly added Fibre card.

c. Use the on Board Fibre cards for SAN switching.

What I am having trouble planning is how do I redirect the shelves to work with the Fibre card I will add? I know its not plug and play" and I dont have much down time.

Any input would be greatly appreciated.

belislesm · ‎2010-03-26

stephenaponte wrote:

Good afternoon, hope all is well. I just wanted to get some thoughts on a project I am currently working on.

I am adding Disk Sheleves on a Filer and implementing Fibre SAN on the Filer for the first time. It currently has no Fibre connections available as the Onbaord is being used by the shelves and all I have is a HBA 4 Port Shelve card.

This paticular Filer has four LUNS. I need to:

a. Add a 4 Port Fibre card to a slot.

b. Move the Shelve Fibre cable from the onboard Fibre Ports and redirect them to the newly added Fibre card.

c. Use the on Board Fibre cards for SAN switching.

What I am having trouble planning is how do I redirect the shelves to work with the Fibre card I will add? I know its not plug and play" and I dont have much down time.

Any input would be greatly appreciated.

Hiya,

The simple way might be to just use the 4port card for the SAN connections rather than the onboard. Then you wouldn't have to move anything. Regardless, I'm pretty sure you don't have to do anything to connect an new/existing shelf loop while preserving the data to a new FC card. I'd verify that with your SE to be safe though. (software disk ownership enabled, etc)

radek_kubka · ‎2010-03-26

The simple way might be to just use the 4port card for the SAN connections rather than the onboard.

That's not doable - FC HBA expansion cards can be target, or initiator, not both:

X1130A-R6 - Quad-port 4/2/1-Gb FCP target-mode interface (for connecting hosts / FC fabric)

X2054A-R63/X2054B-R6 - Quad Port 4/2/1 Gb Fibre Channel optical (for connecting disk shelves or tape devices)

I'm assuming one of the latter models has been purchased, so it cannot be connected to hosts. Also not every filer supports target-mode cards - e.g. FAS2050 doesn't.

Recabling will be required, but onboard ports can be switched to target mode:

http://now.netapp.com/NOW/knowledge/docs/ontap/rel732_vs/html/ontap/bsag/GUID-9657E3AB-3ACA-4717-B049-F4EA8611CF2E.html

Regards,

Radek

belislesm · ‎2010-03-29

radek.kubka wrote:

The simple way might be to just use the 4port card for the SAN connections rather than the onboard.

That's not doable - FC HBA expansion cards can be target, or initiator, not both:

X1130A-R6 - Quad-port 4/2/1-Gb FCP target-mode interface (for connecting hosts / FC fabric)

X2054A-R63/X2054B-R6 - Quad Port 4/2/1 Gb Fibre Channel optical (for connecting disk shelves or tape devices)

I'm assuming one of the latter models has been purchased, so it cannot be connected to hosts. Also not every filer supports target-mode cards - e.g. FAS2050 doesn't.

Recabling will be required, but onboard ports can be switched to target mode:

http://now.netapp.com/NOW/knowledge/docs/ontap/rel732_vs/html/ontap/bsag/GUID-9657E3AB-3ACA-4717-B049-F4EA8611CF2E.html

Regards,

Radek

Ahh I didn't see what model FC card he had. Guess he could always trade it in for the other one if he didn't feel like moving cables 😃 I was wondering if there is any benefit to keeping drive loops connected to the onboard ports rather than connecting them to external cards? Do the 4 port FC cards (init and/or target) suffer from oversubscription where it would be better to keep the drive loops on the internal cards that can leverage the backplane? We have very few SAN implementations of filers, mostly NAS. We have used onboard for loops, and external cards for tapes via NDMP. Is this even worth doing?

stephenaponte · ‎2010-03-27

The problem is the card I am adding is a X2054B-R6

according to Netapp this is only an initiator and can not be used for Target unless that is incorrect?

rkaramchedu1 · ‎2010-03-28

Yes - this card - X2054B-R6 - is an initiator only card. you can use this card for disk shelves and for tape - but you cannot use this card to present luns to your fabric. As it was previously mentioned, the onboards can be changed to target mode and can be used to present luns.

HTH

amiller_1 · ‎2010-04-18

Just in case you didn't wander through all the docs, the largest note is that this will require one reboot at a minimum to change the onboard parts from "initiator" (for disk) to "target" (for FC clients) mode.

eric_barlier · ‎2010-04-18

So to recap. ONLY onboard FC cards can have their personality changed. here is a step by step procedure we have written and used. use at your own risk of course 😉

Also, this is only a cut and paste so you will have to format it youself.

Cheers,

Eric

High Level Steps:

Check Cluster state

Login to contoller RLM (important as reboot will occur)
Cluster failover to cluster partner
Check configuration
Change configuration
Reboot
Check configuration
Cluster giveback

Check cluster status from controller 01B:
01B> cf status > Look for “Cluster enabled, 1A is up.”

On the partner of the controller that will have its FC port(s) change issue a cluster failover.
01B> cf takeover > wait unil completed

Wait until takover has been completed, check failover is complete:
01B> cf status > Look for message stating failover has completed and in takeovermode

1.1.1 Login to RLM/BMC on 01A

1. SSH to RLM/BMC IP of 01A
2. Login to RLM/BMC
3. Issue “system console” > will send you to controller login
4. Login to controller

You are now logged into to controller over RLM/BMC.

1.1.2 Check FC port personality/configuration

01A> fcadmin config 0c > see below for 0c config.

Local
Adapter Type State Status
---------------------------------------------------
0c initiator UNDEFINED online

1.1.3 Change Port Personality

Take the adapter offline using the "storage disable command"
01A> fcadmin offline 0c

Use Fcadmin command to change the port from initiator to target:

Syntax: fcadmin config -d : Disables the adapters drive -t {target|initiator|unconfigured}

01A> fcadmin config -d -t target 0c

Reboot the controller
01A> reboot

Make sure you are connected to RLM/BMC at this point

1.2 Post-Implementation Steps
1.2.1 Verify Port 0c is a target

Plug port 0c into a SAN (switched fabric).
01A> fcp config
01A> fcp show adapter

1.2.2 Bring adapter back online

01A> fcadmin offline 0c

1.2.3 Check Configuration

01A> fcadmin config

Local
Adapter Type State Status
---------------------------------------------------
0c target configured online

1.2.4 Cluster giveback

Check cluster status from controller 01B:
01B> cf status > Look for message stating cluster is in takeovermode

Issue a cluster giveback on controller 01B
01B> cf giveback

Wait until giveback has completed, check cluster state is normal:
01B> cf status > Look for “Cluster enabled, 1A is up.”