Solved: Moving Storage into different Subnet

onephonede · ‎2012-03-01

Dear Community,

I have a FAS2020 in A/A Mode with each controller having a VIF connected to two switches.

I'm planing to move the Storage out of the public subnet into a separate storage subnet.

The current Subnet is: 10.10.0.0/16

The Storage Controller settings are:

VIF1: 10.10.0.20

VIF2: 10.10.0.21

BMC1: 10.10.0.22

BMC2: 10.10.0.23

In the 10.10.0.0/16 subnet I have the DNS Servers and the MS Domain Controllers. The Storage was integrated into the Active-Directory and DNS during initial base Setup.

I dont use CIFs (its disabled) so I dont need UserAuthentication from the Domain-controllers but I still have a few concerns.

The new storage subnet would be somethings like 192.168.0.0/16

- Will the Storage be degraded in some kind of way if I remove the DNS Domain name, remove the DNS servers and disable DNS ?

Dispite the fact that I need to make sure that the ESXi invironment is adjusted to the new IP's adresses of the Storage:

- Would it be possible to leave both BMCs Console ports in the current 10.10.0.0/16 Network for configuration via NetApp System Manager from my Client and for AutoSupport to reach the MailSMTP ?

- I'm not sure but I'm afraid that the System Manager discovers the Controller via their VIF-IPs and not via their BMC IPs ? or is it possible to connect to the Systemmanager via BMC ?

- What about AutoSupport is it possible to use the BMC interface for outgoing Emails towords the Mailserver ?

further I will have a own subnet for each NFS (192.168.1.0) and iSCSI (192.168.2.0) and I cannot set VLANs on the Storage Interfaces if they are combinded as VIF

From my understanding if the Storage is in a class B subnet 192.168.0.0/16 both NFS and iSCSI protocols should be able to reach the Storage ?

Thanks for you help !

Kind Regards,

Michael

andris · ‎2012-03-02

Yes, to keep things consistent, I would use tagged VLAN's for all 3 subnets on the VIF and configure your network switch accordingly.

You should use interface-specific control for your protocols, too.

https://kb.netapp.com/support/index?page=content&id=3011652

Please don't forget to update your /etc/rc file with all of the interface changes.

Yes, there's no problem with the filer being multi-homed to multiple IP subnets. 2 considerations to remember:

- Your default route should point out the public subnet to that subnet's gateway.

- The NFS and iSCSI subnets don't need the default route, as long as all of your hosts are local to those subnets.

Correct. Bug 446493 is fixed in 7.3.5.1P1 or later. You are still at risk.

View solution in original post

andris · ‎2012-03-01

DNS: It makes things more administratively manual, when resolving any hostnames. I suppose you can always hardcode name/IP mappings in hosts files on all of your components in your environment (filer, hosts). E.g. You might need to do that if you send to support.netapp.com via HTTPS.

BMC: The BMC does not pass any form of TCP/IP traffic to the filer from your client (System Manager, telnet, SSH). Your client would need to directly reach the filer's IP addresses for management. Neither does it provide an outgoing path for AutoSupport message from the storage system. It can provide a pass-through for the serial console, though. The BMC can also independently send its own AutoSupport messages for "filer down" problems via SMTP (it inherits the AutoSupport configuration options from its host filer).

Subnets: If you're specifying /16 for the subnet, you can't really say that you're carving up .1 and .2 subnets inside this /16 space. With your proposed setup, both NFS and iSCSI will be able to talk to the filer, but you would need to connect everything in the same L2 Ethernet domain and configure all hosts with the /16 subnet mask. This also means you don't have L2 separation between NFS and iSCSI. VLAN's would be your solution for sharing the VIF's - are you sure you can't configure them on VIF's?

See: https://kb.netapp.com/support/index?page=content&id=1010910

onephonede · ‎2012-03-02

Hi Andris,

I understand that i would need DNS for certain stuff in the Storage subnet, that means I need a extra DNS server in the storage subnet ?

I understand that BMC cannot perform Systemmanager nor Autosupport tasks, that means that I will always have to make sure that there is some kind of communication between storage subnet and public subnet due to outgoing mails and sysmtemanager. that a bit funny becuase my primary goal was to clearly separate them and hide the storage subnet from the public subnet.

but maybe im following the wrong road: maybe I can leave the storage as is it and just add a 2 VLANs on the VIFS for NFS and ISCSI traffic.

but that doesnt work .. see picture.

or do you think that the VIF IPs defenatly need to be moved out of the public subnet ?

EDIT: i see your link it very usefull.

would you recommend leaving the filer in the public subnet for access to mailserver and for Systemamanger and just addin VLANs for NFS and iSCSI on the existing VIFS ?

that would make things easy 🙂

andris · ‎2012-03-02

Note: Per the KB, you can't have a "-" in the VIF name.

Is it a "public" subnet in the Internet sense? Or public to the rest of your company?

Hard to design in a forum, but if you have a VLAN-capable GbE switching infrastructure, there's no reason you can't put 3 VLAN's on the VIF: Public, NFS, iSCSI. Your default route entry would point out the Public subnet, allowing the filer to reach big "I" and other subnets for AutoSupport, DNS, NTP, SNMP, etc. You might want to try CLI if System Manager doesn't let you do this, for some reason.

BTW, I noticed you're running 7.3.5P1 - You should move off this release (7.3.6 would be ideal) as soon as possible.

Details: http://now.netapp.com/NOW/products/csb/csb1106-03.shtml

onephonede · ‎2012-03-02

Note: Per the KB, you can't have a "-" in the VIF name.

yeah, I'm a bit pissed on that "specialist" who setup the Storage I will have to delete the VIF and recreate it ha ? I should do that via BMC or Console, if I do it via Systemmanager I will lose Connection once deleted.

F*** I didnt want to do a complete base setup of the Filer

After deletion of the VIF is it possible to create a new VIF with out an IP like it is now. I mean that only the VLANs have IP's ?

or does the VIF it self always need an IP ?

Is it a "public" subnet in the Internet sense? Or public to the rest of your company?

not in sense of Internet I mean public to the rest of the company.

Hard to design in a forum, but if you have a VLAN-capable GbE switching infrastructure, there's no reason you can't put 3 VLAN's on the VIF: Public, NFS, iSCSI. Your default route entry would point out the Public subnet, allowing the filer to reach big "I" and other subnets for AutoSupport, DNS, NTP, SNMP, etc. You might want to try CLI if System Manager doesn't let you do this, for some reason.

I have VLAN-capable switches but im not sure if I should leave the basic IP of the Filer (without a VLAN) in the "public" subnet ... sure creating VLANs would separate NDS and ISCSI.

Have other companies also have the basic IP of the filer in the normal subnet and create VLANs for Storage traffic ?

BTW, I noticed you're running 7.3.5P1 - You should move off this release (7.3.6 would be ideal) as soon as possible.

Details: http://now.netapp.com/NOW/products/csb/csb1106-03.shtml

this seems very important to you, the 7.3.5P1 has not yet fixed this Excessive SCSI recovery scans error hasn't it ? the version where it was fixed is 7.3.51P1 or are they the same ?

andris · ‎2012-03-02

Yes, to keep things consistent, I would use tagged VLAN's for all 3 subnets on the VIF and configure your network switch accordingly.

You should use interface-specific control for your protocols, too.

https://kb.netapp.com/support/index?page=content&id=3011652

Please don't forget to update your /etc/rc file with all of the interface changes.

Yes, there's no problem with the filer being multi-homed to multiple IP subnets. 2 considerations to remember:

- Your default route should point out the public subnet to that subnet's gateway.

- The NFS and iSCSI subnets don't need the default route, as long as all of your hosts are local to those subnets.

Correct. Bug 446493 is fixed in 7.3.5.1P1 or later. You are still at risk.

onephonede · ‎2012-03-05

Thanks again Andris for taking some time to answer me again.

do I understand it correct that the iSCSI and NFS Service will be served on all VLANs if I dont use interface specific control ?

in my case an Interface would be a VLAN ? (iSCSI VLAN, NFS VLAN etc..)

is it also correct that i have to disable all not needed protocolls per interface ?

Michael

PS: the NetApp whitepapers dont state anything about interface specific control.. why is this then so important ?

andris · ‎2012-03-05

Protocol services by default would be available on all interfaces (some specific limits are placed on the e0M interface, if you have one on your platform).

Every tagged VLAN on the VIF would be an "interface", yes.

Since you were bothering to separate different protocols with different interface, it sounded like a good idea.

https://kb.netapp.com/support/index?page=content&id=3011652

This TR covers this topic: Best Practices for Secure Configuration of Data ONTAP 7G

http://media.netapp.com/documents/tr-3649.pdf