Disk assign and different size

[l_augeard]

Hello, I have a problem,

I have 9 disk 450go (7 assign and 2 parity)

But a have add 5 disk 275go to my aggregate and I see the total size is not egal to 7x450 + 5x275 but is 12x275 !

I see my error, as is the same aggregate, the disk 450go reduce to 275, it's ok ???

but how to remove my 275 from my aggregate ?

[vreddypalli]

Hi,

Can you provide "sysconfig -r" and "sysconfig -a"?

[l_augeard]

Aggregate aggr0 (online, raid_dp) (block checksums)
  Plex /aggr0/plex0 (online, normal, active, pool0)
    RAID group /aggr0/plex0/rg0 (normal)

      RAID Disk Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks                                                                                                                                )    Phys (MB/blks)
      --------- ------          ------------- ---- ---- ---- ----- -------------                                                                                                                                -    --------------
      dparity   0c.00.7         0c    0   7   SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      parity    0c.00.3         0c    0   3   SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0c.00.5         0c    0   5   SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0c.00.1         0c    0   1   SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0c.00.0         0c    0   0   SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0c.00.9         0c    0   9   SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0c.00.8         0c    0   8   SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0c.00.10        0c    0   10  SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0c.00.11        0c    0   11  SA:A   0  SAS  15000 418000/856064                                                                                                                                000  420584/861357448
      data      0b.16           0b    1   0   FC:A   0  FCAL 15000 272000/557056                                                                                                                                000  280104/573653840
      data      0b.17           0b    1   1   FC:A   0  FCAL 15000 272000/557056                                                                                                                                000  280104/573653840
      data      0b.18           0b    1   2   FC:A   0  FCAL 15000 272000/557056                                                                                                                                000  280104/573653840
      data      0b.19           0b    1   3   FC:A   0  FCAL 15000 272000/557056                                                                                                                                000  280104/573653840
      data      0b.20           0b    1   4   FC:A   0  FCAL 15000 272000/557056                                                                                                                                000  280104/573653840
      data      0b.21           0b    1   5   FC:A   0  FCAL 15000 272000/557056                                                                                                                                000  280104/573653840

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk       Device          HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------          ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block or zoned checksum traditional volumes or aggregates
spare           0b.22           0b    1   6   FC:A   0  FCAL 15000 272000/557056000  280104/573653840

NetApp Release 7.3.3: Thu Mar 11 22:43:52 PST 2010
        System ID: 0135068497 (FAS2020-1); partner ID: 0135068203 (FAS2020-2)
        System Serial Number: 500000018255 (FAS2020-1)
        System Rev: E3
        System Storage Configuration: Single-Path HA
        System ACP Connectivity: NA
        slot 0: System Board 2198 MHz (System Board XIV G1)
                Model Name:         FAS2020
                Part Number:        110-00049
                Revision:           G1
                Serial Number:      850769
                BIOS version:       3.0
                Processors:         1
                Processor ID:       0xf29
                Microcode Version:  0x2f
                Memory Size:        1024 MB
                NVMEM Size:         128 MB of Main Memory Used
                CMOS RAM Status:    OK
                Controller:         A
        slot 0: Dual 10/100/1000 Ethernet Controller G20
                e0a MAC Address:    00:a0:98:13:71:cc (auto-1000t-fd-up)
                e0b MAC Address:    00:a0:98:13:71:cd (auto-unknown-down)
                Device Type:        Rev 6
                        memory mapped I/O base 0xd9600000, size 0x20000
                        memory mapped I/O base 0xd9620000, size 0x20000
                        I/O base 0x0000cfe0, size 0x20
                        memory mapped I/O base 0xd9640000, size 0x20000
                        memory mapped I/O base 0xd9660000, size 0x20000
                        I/O base 0x0000cfc0, size 0x20
                        memory mapped I/O base 0xd8000000, size 0x1000
                        I/O base 0x0000ffe0, size 0x20
        slot 0: Interconnect HBA:   Mellanox IB MT25204
                GUID: 0x2c90285076900   LID: 0xbb51     Remote LID: 0xba2b
                Firmware rev:           1.0.800
                Hardware rev:           160
                Command rev:            1
                        memory mapped I/O base 0xd9000000, size 0x100000
                        prefetchable memory base 0xdb800000, size 0x800000
        slot 0: Fibre Channel Target Host Adapter 0a
                (Dual-channel, QLogic 2432(2462) rev. 2, 64-bit, <ONLINE>)
                Firmware rev:     4.5.2
                Host Port Addr:   011600
                Cacheline size:   16
                SRAM parity:      Yes
                FC Nodename:      50:0a:09:80:88:ec:fb:51 (500a098088ecfb51)
                FC Portname:      50:0a:09:81:88:ec:fb:51 (500a098188ecfb51)
                Connection:       PTP, Fabric
                SFP Vendor Name:  FINISAR CORP.
                SFP Vendor P/N:   FTLF8524P2BNV
                SFP Vendor Rev:   A
                SFP Serial No.:   PH53HHS
                SFP Connector:    LC
                SFP Capabilities: 1, 2, 4 Gbit/Sec
                        I/O base 0x0000ee00, size 0x100
                        memory mapped I/O base 0xd9200000, size 0x4000
        slot 0: FC Host Adapter 0b (QLogic 2432 rev. 2, L-port, <UP>)
                Firmware rev:      4.5.2
                Host Loop Id:      7
                FC Node Name:      5:00a:098100:04142b
                SFP Vendor:        FINISAR CORP.
                SFP Part Number:   FTLF8524P2BNV
                SFP Serial Number: PH548AU
                SFP Capabilities:  1, 2 or 4 Gbit
                Link Data Rate:    4 Gbit
                 16  : NETAPP   X279_S15K7288F15 NA00 272.0GB 520B/sect (3SJ1KXG4000090507RDY)
                 17  : NETAPP   X279_S15K7288F15 NA00 272.0GB 520B/sect (3SJ1R4MC00009052PLQ4)
                 18  : NETAPP   X279_S15K7288F15 NA00 272.0GB 520B/sect (3SJ1R6NL00009052MMR5)
                 19  : NETAPP   X279_S15K7288F15 NA00 272.0GB 520B/sect (3SJ1PGRV00009051L4S5)
                 20  : NETAPP   X279_S15K7288F15 NA00 272.0GB 520B/sect (3SJ1NDT500009050X2WM)
                 21  : NETAPP   X279_S15K7288F15 NA00 272.0GB 520B/sect (3SJ0ZHR900009036ZD6L)
                 22  : NETAPP   X279_S15K7288F15 NA00 272.0GB 520B/sect (3SJ1P2DW00009050VSY0)
                Shelf 1: ESH4  Firmware rev. ESH A: 13  ESH B: 13
                        I/O base 0x0000ef00, size 0x100
                        memory mapped I/O base 0xd9204000, size 0x4000
        slot 0: SAS Host Adapter 0c (LSI Logic 1068E rev. B2, SAS, <UP>)
                Firmware rev:      1.26.03.00
                Base WWN:          f:080cfb:5100000:00
                Phy State:         [0] Enabled, 3.0 Gb/s
                                   [1] Enabled, 3.0 Gb/s
                                   [2] Enabled, 3.0 Gb/s
                                   [3] Enabled, 3.0 Gb/s
                00.0 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMX0R6XC)
                00.1 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMWXAMHC)
                00.2 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMX217AC)
                00.3 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMWRTN2C)
                00.4 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMWRW7WC)
                00.5 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMX07SLC)
                00.6 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMX0Y6DC)
                00.7 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMX12NDC)
                00.8 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMWXA4EC)
                00.9 : NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMX219ZC)
                00.10: NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMWYJDAC)
                00.11: NETAPP   X289_HVPBP420A15 NA00 418.0GB 520B/sect (JMX1ZT0C)
                Shelf   0: ESAS  Firmware rev. SAS A: 05 SAS B: 05
                        I/O base 0x0000df00, size 0x100
                        memory mapped I/O base 0xd9410000, size 0x4000
                        memory mapped I/O base 0xd9400000, size 0x10000
        slot 0: ATA/IDE Adapter 0e (0x000001f0)
                0e.0                 STEC     NACF256MM1U-B11 01/17/07 245MB 512B/sect (STI1MMI409300124224)
                        I/O base 0x0000ffc0, size 0x8
                        I/O base 0x0000ffb8, size 0x4
                        I/O base 0x0000ffc8, size 0x8
                        I/O base 0x0000ffbc, size 0x4
                        I/O base 0x0000ffd0, size 0x10
                        memory mapped I/O base 0xd8001000, size 0x400
        Baseboard Management Controller:
                Firmware Version:   1.2
                IPMI version:       2.0
                DHCP:               on
                BMC MAC address:    00:a0:98:13:71:ce
                IP address:         0.0.0.0
                IP mask:            0.0.0.0
                Gateway IP address: 0.0.0.0
                BMC ARP interval:   10 seconds
                BMC has (1) user:   naroot
                ASUP enabled:       on
                ASUP mailhost:      0.0.0.0
                ASUP from:          FAS2020-1@carrieres-sous-poissy.fr
                ASUP recipients:    l.augeard@carrieres-sous-poissy.fr autosupport@netapp.com
                Uptime:             52 Days, 16:07:52

[vreddypalli]

1. General RAID concept is when you have disks with different sizes in a RAID group it will be rounded off to the disk which is smaller in size.  In your case it 275g.

2. You cannot remove disks from an aggregate. You need to destroy the aggregate and recreate it.

3. While creating aggregate NetApp best practice is use homogenous disks.

Hope the above information helps.

[l_augeard]

Is a problem, In my SAN a haven't any disk to recreate aggregate... who I have this without losing my data ?

[chriskranz]

The only way to fix this is the hard way (very hard).

As you only have 1 single aggregate, and this is the root aggregate, you need to re-initialise the entire filer if you want to get these disks back. Copy all your data off the filer, then boot into maintenance mode and 4a the system. This will zero all the disks and create a new aggregate, then you can start again and create either 2 separate aggregates, or 1 aggregate with 2 RAID groups.

Sorry, I can't see an easy solution for you.

[l_augeard]

Yes but if a who I copy all data ? I have only 3 LUN

[chriskranz]

Sorry, that I don't really understand the question?

You would probably need to copy the data out of the LUNs from the host side, then reconfigure the LUNs and copy the data back afterwards.

[l_augeard]

Yes but in production I'm affraid...

And the vol with root of FAS2020 is in the same aggregate, who do it ?

[chriskranz]

Hi,

Unfortunately there is no way to do this non-disruptively. To reconfigure the aggregate you need to take the system offline.

When you boot the system into maintenance mode and run 4a against the system, you re-initialise the aggregate layout and it recreates the root aggregate. Basically you'll be starting again from scratch. You might want to get a NetApp certified engineer on site to help you with this reconfiguration as it's quite involved if you haven't used a NetApp system before.

Cheers...

[l_augeard]

Yes and they are utility to save all LUN ?

[chriskranz]

No, sorry, you'll need to manually copy the data out of the LUNs from the host side of things.

If you have a clustered NetApp, you may be able to juggle the storage, but you'll still need to copy the data across from the host side.

[l_augeard]

Hello,

I have a solution, my data is transfert to NFS share, Can you tel me who delete aggregate in maintenance mode ?

[nttalapaquette]

hi you need to reinstall your filer .

copy your /etc/rc and the exports  if it's exist.