2015-10-18 08:08 PM
Big Problem with new Sim 8.3.1 format is the high amount of Memory 5 GB usage! 8.2 Sim used 1.6 GB Memory and now 5 GB. Especial if running in VMware Workstation means
for a Cluster 10 GB RAM what is very high. With Windows AD so almost no Memory Left to play around with other VM.
Think Back when 7-Mode used for HA only ~512 MB!
After Installation changing back ot 1.6 GB, Sim 8.3 crash and does not start anymore. When putting back to 5 GB all works again.
Does anybody has an Idea how to setup to 1.6 GB Memory and that is working? Is there any setenv?
2015-10-19 07:19 PM
I seem to recall answering a nearly identical question around 8.3.
On 8.3, I was able to run it on 2gb/node with some tuning. On 8.3.1 it needs more like 2.5gb/node minimum but 3gb is a safer starting point. There are two tunables in the loader environment you can optimize for a low memory vsim. First is the variable that sets how much ram is reserved for the BSD layer, the second sets the size of the Virtual NVRAM. If the sim has already been run on the default 256mb vnvram, it may crash if you resize the vnvram. To avoid that crash we'll discard the vnvram contents.
ONTAP still needs more ram, but for a short term lightweight laptop demo 3gb will usually suffice. Give it 4gb if you have it or just want to keep it alive longer, but the 1gb default root volume will take it out pretty quickly so remediate that right away.
2015-10-20 02:43 AM
I according to your guidance to test, vsim is still panic, prompt physical memory is not enough, the previous 8.3 tgz version is OK, but 8.3.1 ova version seems to be no。
2015-10-20 08:24 AM
I just spun one up with the workstation 8.3.1 OVA build: I set ram to 2560MB, vnvram to 64, low_mem to 512
demo100::> run local sysconfig -v NetApp Release 8.3.1: Mon Aug 31 02:06:44 PDT 2015 System ID: 4082368507 (demo100-01) System Serial Number: 4082368-50-7 (demo100-01) System Storage Configuration: Single Path System ACP Connectivity: NA All-Flash Optimized: false slot 0: System Board 1.6 GHz (NetApp VSim) Model Name: SIMBOX Serial Number: 999999 Loader version: 1.0 Processors: 2 Memory Size: 2559 MB Memory Attributes: None Virtual NVRAM Size: 64 MB CMOS RAM Status: OK slot 0: 10/100/1000 Ethernet Controller V e0a MAC Address: 00:0c:29:bd:2c:67 (auto-1000t-fd-up) e0b MAC Address: 00:0c:29:bd:2c:71 (auto-1000t-fd-up) e0c MAC Address: 00:0c:29:bd:2c:7b (auto-1000t-fd-up) e0d MAC Address: 00:0c:29:bd:2c:85 (auto-1000t-fd-up) Device Type: Rev 1 Firmware Version: 15.15 demo100::>
2015-11-17 05:37 AM
Has anyone managed to get the 8.3.1 simulator working with less memory on ESX?
I have tried setting vnvram to 64, low_mem to 512 and various memory sizes up to 4 GB but it kept panicking. It only booted with 5 GB of ram. Below is the panic message with any ram <= 4 GB
VLOADER> setenv bootarg.vnvram.size 64
VLOADER> setenv bootarg.init.low_mem 512
x86_64/freebsd/image1/kernel data=0x99f0c8+0x41a0e0 syms=[0x8+0x46758+0x8+0x2fbe3]
x86_64/freebsd/image1/platform.ko text=0x1e5edd data=0x485a0+0x42bd0 syms=[0x8+0x21768+0x8+0x175cd]
bootarg.init.low_mem set. This will set the FreeBSD size to 512MB
NetApp Data ONTAP 8.3.1
Copyright (C) 1992-2015 NetApp.
All rights reserved.
* Press Ctrl-C for Boot Menu. *
^CBoot Menu will be available.
Please choose one of the following:
(1) Normal Boot.
(2) Boot without /etc/rc.
(3) Change password.
(4) Clean configuration and initialize all disks.
(5) Maintenance mode boot.
(6) Update flash from backup config.
(7) Install new software first.
(8) Reboot node.
Selection (1-8)? 5
WAFL CPLEDGER is enabled. Checklist = 0x7ff841ff
PANIC : prod/common/wafl/free_cache.c:997: Assertion failure.
version: 8.3.1: Mon Aug 31 02:06:44 PDT 2015
conf : x86_64.sim
cpuid = 0
PANIC: prod/common/wafl/free_cache.c:997: Assertion failure. in SK process wafl_hipri on release 8.3.1 (C) on Tue Nov 17 12:29:03 GMT 2015
version: 8.3.1: Mon Aug 31 02:06:44 PDT 2015
compile flags: x86_64.sim
recursive PANIC: page_t has no physical address
cpuid = 0
The operating system has halted.
Please press any key to reboot.
2015-11-17 10:07 AM - edited 2015-11-17 11:10 AM
Are you setting nvram_discard true whenever you adjust the sizing?
Also, what hypervisor are you running it on?
I've been testing against fusion8 and esx6, but its possible on some versions that you may have to manually place the pci hole when you drop under 4gb. Try adding "pciHole.start = 3072" to the vmx file.
On 8.3.1 I've been able to boot to the setup script without modifying the loader values all the way down to 2560MB of ram.
Setting vnvram=128 gets it to boot down to 2432MB
Setting vnvram=64 boots at 2368MB
So here I'm just stealing from vnvram to give to wafl, and its riding the envelope of the low end. I can push it down a little further with low_mem=512. Stealing from BSD now, and get it to boot with 2112. But that only gets you to the setup script. The RDBs consume ram, so you need to add some ram to the sizing to account for actually getting the cluster services up and running. So 3072 seems like a pretty good number for a laptop instance, depending on what you need it to do and how long you need it to live.
That smallest supported platform for 8.3+ has 6gb of ram, so its probably going to get harder to bring up smaller instances as time goes on.
2015-11-20 03:49 AM
I'm using esxi6 on workstation hardware ( i5@3100GHz / 32 GB ram) as virtual lab.
Haven't set "nvram_discard true" each time I changed ram.
However after your reply (thanks for that!) I have restarted from scratch and managed to get it running using the steps below:
1. Deploy 8.3.1 esx ova (by default it gets 8 GB of ram).
2. Reduce memory allocation to 3072 MB (3 GB).
3. Add two more network cards (for a total of 6) with default settings.
4. Add a serial port set to "Use Network", Direction = "Server", Port URI = "telnet://:12345". This way after power on I am able to connect via putty (telnet to esx host IP address on port 12345) to the virtual serial console making initial setup and management much easier. This will actually start working right after step 6.b below.
5. Power up vm and initially use the vm console to get to VLOADER prompt.
6. VLOADER commands:
a. setenv comconsole_speed 115200
b. setenv console comconsole,vidconsole (at which point we can open the putty telnet session and continue from there)
c.1 setenv bootarg.vm.sim.vdevinit "36:14:0,31:14:1,30:14:2,35:14:3"
c.2 setenv bootarg.sim.vdevinit "36:14:0,31:14:1,30:14:2,35:14:3" (this changes the disk configuration to add a shelf of virtual ssd's)
7. Ctrl + C and choose option 5 to assign 3 specific disks for the root aggregate then halt and reboot.
8. Ctrl + C and choose option 4 to initialize all disks and reset configuration.
After reboot it got to the system setup and I was able to go all the way to create the cluster.
I'm not expecting too much from the simulator performance wise therefore not going to push it however I would need it to be as stable as possible (as in not panicking) for a fairly long run as my lab is normally up 24x7.. so given the steps I've taken can you advise on the below:
1. What LOADER values I should now amend (if any) to achieve this?
2. Would it make a difference if I gave each node 4 GB ram instead of 3?
3. Any other tuning that may help?
2015-11-20 07:36 AM
Looks good. I also favor the telnet console, its just easier to work with, capture output, etc. I would just make sure root landed on either the 4gb or the 9gb drives, if not add enough disks to get the root aggr to about 4gb, expand to root vol as high as it will let you, and disable snapshots on the root volume. If you have the simulator running inside your nested esx host you may be better off moving it out to the physical workstation host. The telnet console still works there, but it requires manual vmx entries. These should do it:
serial0.present = "TRUE" serial0.fileType = "network" serial0.fileName = "telnet://0.0.0.0:2201" serial0.network.endPoint = "server" serial0.startConnected = "TRUE"
The ram really depends on your usage. If you are going to use it to host datastores back to your nested esx hosts and put VMs in them, then more ram is probably a good idea. Mostly because the VHA disk subsystem is really slow, so more ram would mean better cache hits and fewer calls to the simdisks. 3gb is probably enough to get by otherwise. 8.3.2 raises the low end a tad, but still seems to be working on 3gb. Leaving it up 24x7 will generate logs and asup files that you may need to periodically prune from the mroot, but at ~4gb on the root its not usually a problem.
One other thing to check is the vnvram settings. on some builds I've seen it set to full, on some builds I've seen it set to panic. I would set it to full, so the contents to the nvram are persisted to the backing vmdk. That way when microsoft gives a bsod theres a good chance your sim will recover gracefully.
setenv bootarg.vm.vnvram full