How-to exchange filers' disks and aggrs

[jqLi]

Hi,

We have 2 Fas8020 Running 7-Mode OnTap 8.2.3P3 in HA pair, their aggrs are all SyncMirrored. We'd like to exchange their disks ownership to make A controller works as B controller. And the same to B. How could we do?

I have read some Introductions for replace one node using

disk reassign -s   <old_sysid> -d <new_sysid>

but I am afraid if I use 

disk reassign -s   <a_sysid> -d unowned

disk reassign -s   <b_sysid> -d <a_sysid>

disk reassign -s   unowned -d <b_sysid>

reboot

I think something error may happen during that.

[aborzenkov]

Is it HA pair? In this case both controllers are identical, what is the reason to do it? Or do you have two independent single node filers?

[jqLi]

Yes, It's HA Pair.  The A controller is alway facing a power problem that both controller and shelves are down these days. Without MetroCluster It can be accessible ONLY  taken over by admin before the power loss(I don't want to use cf forcetakeover -d). Unfortunaly, It's not predictable all the time. The A controller is holding main  business we need. It seems B's down is acceptable.

[aborzenkov · Accepted solution]

Well, in this case disable cf before exchanging disks (and reboot controllers after that to make sure they come as single nodes). When swapping disks do not unassign them, rather assign to some dummy sysid not equal to both controllers. This way you will always have overview which disks belong to which controller. After booting enable cf again.

[jqLi]

Thanks a lot.

I have several questions. 

1. Must I try reassign in Maintance Mode? If the disks owner was changed to unowned, then another one own it ,all data lost?

2. Are all configs  writen to vol0? After I exchanged disks, will their ethernet configs exchange? And the port defination?

Thanks again.

[aborzenkov]

If your nodes have symmetrical hardware configuration (which they should have in HA pair) there should be no problem with interfaces. Not sure what "port destination" means. What I'm not sure about is FC target WWPNs; if you have them you may want to record original configuration and manually change WWPNs after reassignment. Yet another reason to have support case open to get support in case something turns out wrong.

Oh, and LUN serial numbers may change as well. Happened in the past during head swap. Same consideration - record original configuration (or have ASUP ready) and compare after reassignment, fix serial numbers if necessary.

As you effectively move system to another place, you may need to get in touch with your infrastructure team to let them update their inventory (what is connected where).

[jqLi]

Or how about just change their NVRAM card?

[aborzenkov]

So you apparently have MetroCluster. In which case takeover after total site outage is exactly what it is intended for. Did you consider tie breaker to automate it? But if you insist ...

FAS8020 does not have separate NVRAM card, you probably confuse FC-VI for it. Part of configuration is stored in volatile /var which is backed up to boot medium and root volume. It is possible to either restore /var backup from new root or simply swap boot media between two nodes before reassigning disks. And yes, disk reassignment must be done in maintenance mode with both controllers down. I'd also clear mailboxes in maintenance mode to be on safe side (see any document on head swap).

You will also need to reinstall licenses from partner node as licenses are associated with system serial numbers and after disk swap each node will see licenses of partner. And I strongly recommend you to open support case and confirm that this will actually work, to be on safe side.

[jqLi]

OK. That means olny reassign disks is not enough. Restore /var of boot media is also needed.

Are the configs  copied both in boot media and root vol? Or I see /etc/rc and /etc/hosts will be loaded from root vol and options will be loaded from /var.

How to restore the /var dir? I'm sorry I have searched the forums but can't find any solution.

[jqLi]

It seems that cables on fiber switches are not the same either side. But I think it should be configured in the switch not the head. The easiest way might be swap head physically. But this method is clumsy.

[aborzenkov]

As I mentioned already, just swapping boot media should be enough to get matching /var content. Otherwise look at boot media replacement document available on support portal for each filer model. It describes how to restore copy from root volume.

[jqLi]

Yes. I'm going to boot it to loader> and use printenv and setenv. They should be same since symmetrical structs.
By the way if I use disk assign <all_disks_in_an_aggr> -s <new_sysid> -f, will they become spare disks or total as an aggr?
Have a nice day!

[aborzenkov]

Reassigned disks remain in the aggregate.