Simulators and SSDs

ekashpureff · ‎2014-04-17

All -

We used to have students add their own SSDs to the sims in our lab environs.

The procedure has been taken out of the most recent version of the Cluster Mode class.

I thought I'd take some time to document it here.

It's needed to be able to simulate creating hybrid aggrs - Flash Pools.

Step 1 - Unlock the Diag user

Cluster mode:

set diag

security login unlock -username diag -vserver clustername

security login password -username diag -vserver clustername

7-mode:

priv set diag

useradmin diaguser unlock

useradmin diaguser password

Step 2 - Enter system shell

Log in as user diag ...

Cluster mode:

systemshell -node clustername1/2

7-mode:

systemshell

Step 3 - Create SSD Disks

setenv PATH /sim/bin:$PATH

cd /sim/dev

sudo vsim_makedisks -t 35 -a 2 -n 14

exit

Step 4 -

Reboot to recognize new disks.

assign new unowned disks

Enjoy !

I'll update this doc with defs of the different sim disk types as I figure it out ...

I hope this response has been helpful to you.

At your service,

Eugene E. Kashpureff

Independent NetApp Consultant, K&H Research http://www.linkedin.com/in/eugenekashpureff

Senior NetApp Instructor, Unitek Education http://www.unitek.com/training/netapp/

(P.S. I appreciate points for helpful or correct answers.)

shatfield · ‎2014-04-25

Since you have to reboot anyway, you could skip the tour of the systemshell and just rearm vdevinit:

1: Reboot, stopping at the VLOADER> prompt

2: Enter the following:

setenv bootarg.vm.sim.vdevinit "35:14:2"

setenv bootarg.sim.vdevinit "35:14:2"

boot

The values should look familiar. It's Type:Number:Adapter, and it spawns vsim_makedisks with those values during boot.

It can do multiple passes by comma separating the value triplets. For example, "23:14:2,34:14:3"

If you want the list of valid disk types, run vsim_makedisks -h at the systemshell and it will output the current list.

35 is a 500mb ssd. If you're just building flashpools, 34 is a 100mb ssd.

ekashpureff · ‎2014-04-28

Sean -

Awesome response and info, thank you !

Where did you find this documented, or did you just hack it out ?

At your service,

Eugene E. Kashpureff

Independent NetApp Consultant, K&H Research http://www.linkedin.com/in/eugenekashpureff

Senior NetApp Instructor, Unitek Education http://www.unitek.com/training/netapp/

shatfield · ‎2014-04-28

I noticed those values getting were set to false after the first boot, and it didn't take too long to figure out how they work. I've been using them to build custom vsims ever since.

If you catch the vsim on the first boot, that whole process of customizing/maximizing vsims can be reduced to a few of bootargs.

HARRYKAKAROUBAS · ‎2014-08-16

Thanks Eugene, found this post very helpful.

Regards

Harry Kakaroubas

ekashpureff · ‎2014-08-16

Harry -

You're very welcome.

Hope you'll share it with other guys on the team ?

Hoping you all rock on the test !

javierb · ‎2016-04-06

Hello people

just a quick question.

in my vsim-simulator-832-RC1

Following the procedure, everything is alsmot Ok in my case.

After adding some type 34 and type 35 SSD,

If I enter

CDOT-vsim::*> node run -node ClusterSim-01|ClusterSim-02 -command disk show -n

there is no new disk available from either node ClusterSim-01 or 02 to be assigned.

But if i go to systemshell, the disks do appear

ClusterSim::*> node run -node ClusterSim-0* -command disk show -v v2.32 (v2.32 already existed)
2 entries were acted on.

Node: ClusterSim-01
DISK       OWNER                    POOL   SERIAL NUMBER         HOME                     DR HOME                CHKSUM
------------ -------------            ----- -------------         -------------            -------------          --------
v2.32        ClusterSim-01(4082368511)    Pool0 11744813              ClusterSim-01(4082368511)                          Block

Node: ClusterSim-02
DISK       OWNER                    POOL   SERIAL NUMBER         HOME                     DR HOME                CHKSUM
------------ -------------            ----- -------------         -------------            -------------          --------
v2.32        ClusterSim-02(4034389062)    Pool0 62751613              ClusterSim-02(4034389062)                          Block

In systemshell of ClusterSim-01, they do both appear, the existing and the new one (SSD): v2.32 and v2.33

ClusterSim-01% ls /sim/dev/,disks/v2.32:NETAPP__:VD-4000MB-FZ-520:16744813:8248448
/sim/dev/,disks/v2.32:NETAPP__:VD-4000MB-FZ-520:16744813:8248448
ClusterSim-01% ls /sim/dev/,disks/v2.33:NETAPP__:VD-100MB-SS-512_:63004514:261248
/sim/dev/,disks/v2.33:NETAPP__:VD-100MB-SS-512_:63004514:261248

Any idea on how to make the node ClusterSim-01 "see" or "detect" the new added SSD devices created?

thanks

Javier

oscaraock · ‎2016-12-21

Im in the same scenario that javierb, after running the comand in the diag prompt and reboot the cluter doesnt see the disks but with the systemshell is posible see the disk.

After I tried with the environment variables it works to add the disks SSD but it removes the other disks and I must initilize all disk and setup again

How I do to simply add the disks?

oscaraock · ‎2016-12-21

After initialize all disk we can see the SSD disk in the simulator

Now I have problems to create the StoragePool

CDOT::> storage pool create -storage-pool sp1 -simulate true -disk-list NET-1.29 NET-1.30 NET-1.31

Storage pool creation would succeed with allocation unit size 351MB.

CDOT::> storage pool create -storage-pool sp1 -simulate false -disk-list NET-1.29 NET-1.30 NET-1.31
[Job 14] storage pool create job failed. Rolling back partial changes made by jo                                                                                                 [Job 14] storage pool create job failed. Rollback Progress: Waiting for disks to                                                                                                  come back.                                                                                                                                                                      Error: command failed: [Job 14] Job failed: Unable to share disk "NET-1.29". Reason: Not a spare disk.
       Rollback Status: storage pool create rollback for "sp1" completed successfully

CDOT::> disk show
                     Usable           Disk    Container   Container
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
NET-1.1              3.93GB     - 42 FCAL    spare       Pool0     CDOT-01
NET-1.2              3.93GB     - 43 FCAL    spare       Pool0     CDOT-01
NET-1.3              3.93GB     - 44 FCAL    spare       Pool0     CDOT-01
NET-1.4              3.93GB     - 45 FCAL    spare       Pool0     CDOT-01
NET-1.5              3.93GB     - 48 FCAL    spare       Pool0     CDOT-01
NET-1.6              3.93GB     - 16 FCAL    aggregate   aggr0     CDOT-01
NET-1.7              3.93GB     - 17 FCAL    aggregate   aggr0     CDOT-01
NET-1.8              3.93GB     - 18 FCAL    aggregate   aggr0     CDOT-01
NET-1.9              3.93GB     - 19 FCAL    spare       Pool0     CDOT-01
NET-1.10             3.93GB     - 20 FCAL    spare       Pool0     CDOT-01
NET-1.11             3.93GB     - 21 FCAL    spare       Pool0     CDOT-01
NET-1.12             3.93GB     - 22 FCAL    spare       Pool0     CDOT-01
NET-1.13             3.93GB     - 24 FCAL    spare       Pool0     CDOT-01
NET-1.14             3.93GB     - 25 FCAL    spare       Pool0     CDOT-01
NET-1.15             3.93GB     - 26 FCAL    spare       Pool0     CDOT-01
NET-1.16             3.93GB     - 27 FCAL    spare       Pool0     CDOT-01
NET-1.17             3.93GB     - 28 FCAL    spare       Pool0     CDOT-01
NET-1.18             3.93GB     - 29 FCAL    spare       Pool0     CDOT-01
NET-1.19             3.93GB     - 32 FCAL    spare       Pool0     CDOT-01
NET-1.20             3.93GB     - 33 FCAL    spare       Pool0     CDOT-01

                     Usable           Disk    Container   Container
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
NET-1.21             3.93GB     - 34 FCAL    spare       Pool0     CDOT-01
NET-1.22             3.93GB     - 35 FCAL    spare       Pool0     CDOT-01
NET-1.23             3.93GB     - 36 FCAL    spare       Pool0     CDOT-01
NET-1.24             3.93GB     - 37 FCAL    spare       Pool0     CDOT-01
NET-1.25             3.93GB     - 38 FCAL    spare       Pool0     CDOT-01
NET-1.26             3.93GB     - 39 FCAL    spare       Pool0     CDOT-01
NET-1.27             3.93GB     - 40 FCAL    spare       Pool0     CDOT-01
NET-1.28             3.93GB     - 41 FCAL    spare       Pool0     CDOT-01
NET-1.29            520.5MB     - 16 SSD     spare       Pool0     CDOT-01
NET-1.30            520.5MB     - 17 SSD     spare       Pool0     CDOT-01
NET-1.31            520.5MB     - 18 SSD     spare       Pool0     CDOT-01
NET-1.32            520.5MB     - 19 SSD     spare       Pool0     CDOT-01
NET-1.33            520.5MB     - 20 SSD     spare       Pool0     CDOT-01
NET-1.34            520.5MB     - 21 SSD     spare       Pool0     CDOT-01
NET-1.35            520.5MB     - 22 SSD     spare       Pool0     CDOT-01
NET-1.36            520.5MB     - 24 SSD     spare       Pool0     CDOT-01
NET-1.37            520.5MB     - 25 SSD     spare       Pool0     CDOT-01
NET-1.38            520.5MB     - 26 SSD     spare       Pool0     CDOT-01
NET-1.39            520.5MB     - 27 SSD     spare       Pool0     CDOT-01

                     Usable           Disk    Container   Container
Disk                   Size Shelf Bay Type    Type        Name      Owner
---------------- ---------- ----- --- ------- ----------- --------- --------
NET-1.40            520.5MB     - 28 SSD     spare       Pool0     CDOT-01
NET-1.41            520.5MB     - 29 SSD     spare       Pool0     CDOT-01
NET-1.42            520.5MB     - 32 SSD     spare       Pool0     CDOT-01
42 entries were displayed.

Any idea?

SeanHatfield · ‎2017-01-11

After adding disks using the systemshell approach, you must reboot the simulator for the disks to become available for use. There is a way to hot add, but rebooting the simulator is much simpler. Also be sure not to place more than 14 disks on each controller.

Using the bootarg approach to add disks doesn't appear to have been working for a long time, but I can't edit that old post. Just ignore it and follow Eugene's original writeup.

On the storagepool creation issue, this type of simulated disk cannot be shared (partitioned), so creating storage pools will fail. You can still create flashpools by using complete disks.

Chucky · ‎2017-10-10

Hi SeanHatfield,

My Simulator is a 2-node ONTAP 9.1 cluster.

I have my the disks setup and the 4 adapters are already filled up with 14 disks each.

How can i add SSDs in this case please?

DO i have to drop some disks from an adapter in other to make space for a SSD disk??

Thank you

SeanHatfield · ‎2017-10-10

You need to remove some disks first. If they are spare you can run "disk simpull" on the disks you want to remove, then you can add SSD to replace them.

Chucky · ‎2017-10-11

Hi SeanHatfield,

Ok i used the command "storage disk remove DISK_Name" to remove some sparedisks.

The sparedisks were named NET-1.34,NET-1.35 & NET-1.36.

Atter that i added the 3 SSD -type 35 disks and rebooted to -a 1(i.e adapter 1)

I have executed storage disk show -spare and i can see the disks i deleted were readded after reboot , and when i run show disk i don't see the SSD disks

cluster1::> storage disk show -spare
Original Owner: cluster1-01
Checksum Compatibility: block
Usable Physical
Disk HA Shelf Bay Chan Pool Type Class RPM Size Size Owner
--------------- ------------ ---- ------ ------ ----------- ------ -------- -------- --------
NET-1.34 v0 - 27 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-01
NET-1.35 v0 - 28 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-01
NET-1.36 v0 - 29 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-01
NET-1.37 v0 - 32 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-01
Original Owner: cluster1-02
Checksum Compatibility: block
Usable Physical
Disk HA Shelf Bay Chan Pool Type Class RPM Size Size Owner
--------------- ------------ ---- ------ ------ ----------- ------ -------- -------- --------
NET-2.20 v0 - 21 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
NET-2.21 v0 - 22 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
NET-2.22 v0 - 24 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
NET-2.23 v0 - 25 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
NET-2.31 v0 - 26 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
NET-2.33 v0 - 28 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
NET-2.34 v0 - 29 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
NET-2.35 v0 - 32 A Pool0 FCAL performance 15000 3.93GB 3.93GB cluster1-02
12 entries were displayed.

Chucky · ‎2017-10-11

Basically i need help with adding SSD disks to an ONTAP 9.1 2-node simulator.

I have tried removing sparedisks and creating SSD disks and rebooting.

The SSD disks don't show up when i run storage disk show.

What am i missing please?

I am an Oracle DBA new to Netapp

HELP!!

SeanHatfield · ‎2017-10-13

The simulator disks get re-detected when the controller boots up because they are still present in the /sim filesystem.

So we need to "pull" them out of the simulated disk shelf.

First I need to find my spares:

demo::> disk show -spare
Original Owner: test92p1                                              
  Checksum Compatibility: block
                                                                         Usable Physical
    Disk            HA Shelf Bay Chan   Pool   Type Class          RPM     Size     Size Owner
    --------------- ------------ ---- ------ ------ ----------- ------ -------- -------- --------
    NET-1.5         v1     -  20    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.6         v1     -  21    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.7         v1     -  22    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.8         v1     -  24    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.9         v1     -  25    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.10        v1     -  26    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.11        v1     -  27    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.12        v1     -  28    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.13        v1     -  29    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.14        v1     -  32    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.25        v0     -  27    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.26        v0     -  28    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.27        v0     -  29    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.28        v0     -  32    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
14 entries were displayed.

I'll pull out the disks in v0.27,28,29,and 32 to make room for some SSD.

demo::> run local disk simpull v0.27 v0.28 v0.29 v0.32

Now when I show spares they aren't listed:

demo::> disk show -spare                        
Original Owner: test92p1                                              
  Checksum Compatibility: block
                                                                         Usable Physical
    Disk            HA Shelf Bay Chan   Pool   Type Class          RPM     Size     Size Owner
    --------------- ------------ ---- ------ ------ ----------- ------ -------- -------- --------
    NET-1.5         v1     -  20    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.6         v1     -  21    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.7         v1     -  22    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.8         v1     -  24    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.9         v1     -  25    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.10        v1     -  26    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.11        v1     -  27    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.12        v1     -  28    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.13        v1     -  29    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.14        v1     -  32    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
10 entries were displayed.

I can also see which disks have been pulled like this:

demo::> run local disk simpush -l


The following pulled disks are available for pushing:
	 v0.27:NETAPP__:VD-1000MB-FZ-520:14822110:2104448
	 v0.28:NETAPP__:VD-1000MB-FZ-520:14822111:2104448
	 v0.29:NETAPP__:VD-1000MB-FZ-520:14822112:2104448
	 v0.32:NETAPP__:VD-1000MB-FZ-520:14822113:2104448

Now I can make new disks, from the systemshell:

demo::*> systemshell local
  (system node systemshell)
diag@127.0.0.1's password: 

Warning:  The system shell provides access to low-level
diagnostic tools that can cause irreparable damage to
the system if not used properly.  Use this environment
only when directed to do so by support personnel.

test92p1% sudo bash
bash-3.2# cd /sim/dev
bash-3.2# vsim_makedisks -t 35 -a 0 -n 4
Creating ,disks/v0.27:NETAPP__:VD-500MB-SS-520_:17920610:1080448
Creating ,disks/v0.28:NETAPP__:VD-500MB-SS-520_:17920611:1080448
Creating ,disks/v0.29:NETAPP__:VD-500MB-SS-520_:17920612:1080448
Creating ,disks/v0.32:NETAPP__:VD-500MB-SS-520_:17920613:1080448
Shelf file Shelf:DiskShelf14 updated
bash-3.2#

Now I could reboot here, and the new disks would be detected, and all would be good, but here's a tip for skipping the reboot and doing this while the sim cluster is live:

Move the new disk files into the pulled directory:

mv ",disks/v0.27:NETAPP__:VD-500MB-SS-520_:17920610:1080448" ,disks/,pulled
mv ",disks/v0.28:NETAPP__:VD-500MB-SS-520_:17920611:1080448" ,disks/,pulled
mv ",disks/v0.29:NETAPP__:VD-500MB-SS-520_:17920612:1080448" ,disks/,pulled
mv ",disks/v0.32:NETAPP__:VD-500MB-SS-520_:17920613:1080448" ,disks/,pulled

Now exit the systemshell, and look at the list of disks we can push again:

demo::*> run local disk simpush -l


The following pulled disks are available for pushing:
	 v0.27:NETAPP__:VD-1000MB-FZ-520:14822110:2104448
	 v0.28:NETAPP__:VD-1000MB-FZ-520:14822111:2104448
	 v0.29:NETAPP__:VD-1000MB-FZ-520:14822112:2104448
	 v0.32:NETAPP__:VD-1000MB-FZ-520:14822113:2104448
	 v0.27:NETAPP__:VD-500MB-SS-520_:17920610:1080448
	 v0.28:NETAPP__:VD-500MB-SS-520_:17920611:1080448
	 v0.29:NETAPP__:VD-500MB-SS-520_:17920612:1080448
	 v0.32:NETAPP__:VD-500MB-SS-520_:17920613:1080448

See those SSD waiting to be pushed? Lets do that:

demo::*> run local disk simpush v0.27:NETAPP__:VD-500MB-SS-520_:17920610:1080448
demo::*> run local disk simpush v0.28:NETAPP__:VD-500MB-SS-520_:17920611:1080448
demo::*> run local disk simpush v0.29:NETAPP__:VD-500MB-SS-520_:17920612:1080448
demo::*> run local disk simpush v0.32:NETAPP__:VD-500MB-SS-520_:17920613:1080448

And run a disk show and they will appear as unassigned. Wait for autoassign to pick them up or do a disk assign -all true, and then they'll be spares ready for use:

demo::*> disk show -spare                 
Original Owner: test92p1                                              
  Checksum Compatibility: block
                                                                         Usable Physical
    Disk            HA Shelf Bay Chan   Pool   Type Class          RPM     Size     Size Owner
    --------------- ------------ ---- ------ ------ ----------- ------ -------- -------- --------
    NET-1.5         v1     -  20    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.6         v1     -  21    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.7         v1     -  22    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.8         v1     -  24    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.9         v1     -  25    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.10        v1     -  26    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.11        v1     -  27    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.12        v1     -  28    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.13        v1     -  29    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.14        v1     -  32    A  Pool0   FCAL performance  15000   1020MB   1.00GB test92p1
    NET-1.29        v0     -  27    A  Pool0    SSD solid-state      -  520.5MB  527.6MB test92p1
    NET-1.30        v0     -  28    A  Pool0    SSD solid-state      -  520.5MB  527.6MB test92p1
    NET-1.31        v0     -  29    A  Pool0    SSD solid-state      -  520.5MB  527.6MB test92p1
    NET-1.32        v0     -  32    A  Pool0    SSD solid-state      -  520.5MB  527.6MB test92p1
14 entries were displayed.

Now I have SSD sim disks ready for use.

Community

Simulators and SSDs

Simulators and SSDs

Warning!