Community maintenance is complete. Thank you for your patience!

Simulator Discussions

9.7 simulator root vol0 keeps filling up snap and auto support turned off

mjgratke

Something like logging is filling root vol0 up you can see the creep in used over just a few seconds.   What is running to fill it up?   snap and auto support turned off      Thanks 

 

cluster97-01*> df
Filesystem kbytes used avail capacity Mounted on
/vol/vol0/ 10957620 956916 10000704 9% /vol/vol0/
/vol/vol0/.snapshot 576716 0 576716 0% /vol/vol0/.snapshot
cluster97-01*> df
Filesystem kbytes used avail capacity Mounted on
/vol/vol0/ 10957620 957004 10000616 9% /vol/vol0/
/vol/vol0/.snapshot 576716 0 576716 0% /vol/vol0/.snapshot
cluster97-01*> df
Filesystem kbytes used avail capacity Mounted on
/vol/vol0/ 10957620 957632 9999988 9% /vol/vol0/
/vol/vol0/.snapshot 576716 0 576716 0% /vol/vol0/.snapshot
cluster97-01*>

1 ACCEPTED SOLUTION

JohnChampion

# Expand root volume size
# Add disk(s) to root aggregate
storage aggregate add-disks -aggregate aggr0 -diskcount 1

# Set root volume to new size
volume size -vserver <vserver_name> -volume vol0 -new-size 1.5GB

# Verify
volume show -volume vol0

 

# Disable root snapshots

run local
snap delete -a -f vol0
snap sched vol0 0 0 0
snap autodelete vol0 on
snap autodelete vol0 target_free_space 35
snap autodelete vol0

 

...and have a look at this post - it's a bit old but much still applies and works - especially making more and larger disks.

 

https://community.netapp.com/t5/Developer-Network-Articles-and-Resources/Building-and-Customizing-NetApp-Simulators/ta-p/87157/highlight/true

 

View solution in original post

4 REPLIES 4

JohnChampion

# Expand root volume size
# Add disk(s) to root aggregate
storage aggregate add-disks -aggregate aggr0 -diskcount 1

# Set root volume to new size
volume size -vserver <vserver_name> -volume vol0 -new-size 1.5GB

# Verify
volume show -volume vol0

 

# Disable root snapshots

run local
snap delete -a -f vol0
snap sched vol0 0 0 0
snap autodelete vol0 on
snap autodelete vol0 target_free_space 35
snap autodelete vol0

 

...and have a look at this post - it's a bit old but much still applies and works - especially making more and larger disks.

 

https://community.netapp.com/t5/Developer-Network-Articles-and-Resources/Building-and-Customizing-NetApp-Simulators/ta-p/87157/highlight/true

 

View solution in original post

mjgratke

Thanks for the reply.   I have increased vol0 size and it's going to take longer fill up but something is still constantly consuming free space.  I assume logging but i can not track it down.    Will the simulator manage the growth so vol0 will not run out of space?    

ploiesti

I`ve recovered space from diag mode --> systemshell node run...bla bla...

 

right there on the low level BSD(linux stuff) you can delete log files ( A LOT of log files).

 

cluster2-01% ll
total 40
-rw-r--r-- 1 diag nogroup 1145 Dec 12 15:05 .cshrc
-rw------- 1 diag nogroup 545 Jan 9 09:16 .history
-rw-r--r-- 1 diag nogroup 323 Dec 12 15:05 .login
-rw-r--r-- 1 diag nogroup 91 Dec 12 15:05 .login_conf
-rw------- 1 diag nogroup 301 Dec 12 15:05 .mail_aliases
-rw-r--r-- 1 diag nogroup 267 Dec 12 15:05 .mailrc
-rw-r--r-- 1 diag nogroup 881 Dec 12 15:05 .profile
-rw------- 1 diag nogroup 212 Dec 12 15:05 .rhosts
-rw-r--r-- 1 diag nogroup 782 Dec 12 15:05 .shrc
drwxr-xr-x 2 root nogroup 512 Jan 9 09:21 .ssh-systemshell/
cluster2-01% cd /
cluster2-01% ll
total 38545
-r--r--r-- 1 root wheel 398 Jan 9 2020 .valgrindrc
-rw-r--r-- 1 root wheel 168 Jan 9 2020 BUILD
-r--r--r-- 1 root wheel 63 Jan 9 2020 COMPAT.TXT
-r--r--r-- 1 root wheel 6142 Jan 9 2020 COPYRIGHT
-r-xr-xr-x 1 root wheel 52779 Jan 9 2020 INSTALL*
-r--r--r-- 1 root wheel 1867 Jan 9 2020 README.TXT
-r--r--r-- 1 2087 30 4 Jan 17 2020 VERSION
drwxr-xr-x 2 root wheel 1024 Jan 9 2020 bin/
drwxr-xr-x 6 root wheel 512 Jan 9 2020 boot/
-r--r--r-- 1 root wheel 172318 Jan 9 2020 cap.xml
drwxr-xr-x 1 root wheel 32768 Jan 1 1980 cfcard/
dr-xr-xr-x 2 root wheel 512 Jan 9 09:18 clus/
dr-xr-xr-x 6 root wheel 512 Jan 9 09:18 dev/
drwxr-xr-x 32 root wheel 6656 Jan 17 2020 etc/
drwxr-xr-x 2 root wheel 1024 Jan 9 2020 fw/
drwxr-xr-x 3 root wheel 512 Jan 9 2020 lib/
drwxr-xr-x 3 root wheel 512 Jan 9 2020 libexec/
drwxr-xr-x 6 root wheel 512 Jan 9 2020 mnt/
drwxr-xr-x 8 root wheel 4096 Jan 9 09:26 mroot/
-rw-r--r-- 1 root wheel 4720265 Jan 9 2020 mroot.tgz
drwxr-xr-x 3 root wheel 512 Jan 9 2020 mroot_late/
-rw-r--r-- 1 root wheel 34196628 Jan 9 2020 mroot_late.tgz
-r-xr-xr-x 1 root wheel 868 Jan 9 2020 netapp_nodar_build_check*
drwxr-xr-x 2 root wheel 512 Jan 9 2020 nfsroot/
drwxr-xr-x 3 root wheel 512 Jan 9 2020 ontap/
drwxr-xr-x 2 root wheel 512 Jan 9 2020 ovl/
drwxr-xr-x 2 root wheel 512 Jan 9 2020 partner/
drwxr-xr-x 5 root wheel 512 Jan 9 2020 platform/
dr-xr-xr-x 1 root wheel 0 Jan 9 09:26 proc/
drwxr-xr-x 3 root wheel 512 Jan 9 2020 root/
drwxr-xr-x 2 root wheel 3584 Jan 17 2020 sbin/
drwxr-xr-x 8 root wheel 512 Dec 12 15:03 sim/
drwxr-xr-x 19 root wheel 512 Jan 9 2020 sldiag/
drwxrwxrwt 7 root wheel 1024 Jan 9 09:26 tmp/
drwxr-xr-x 12 root wheel 512 Jan 9 2020 usr/
drwxr-xr-x 24 root wheel 1024 Jan 9 09:26 var/
-rw-r--r-- 1 root wheel 40178 Jan 17 2020 varfs.tgz
-r--r--r-- 1 root wheel 37495 Jan 9 2020 vs_conf_files.tgz
drwxr-xr-x 3 root wheel 512 Jan 9 2020 webjail/
cluster2-01% cd /v
var/ varfs.tgz vs_conf_files.tgz
cluster2-01% cd /var/log
cluster2-01% ls
apache_logs netapp_fpolicy.log
audit.log.00000000000000000028 netapp_frulog.log
audit.log.00000000000000000029 netapp_mdns.log
bcomd.log.00000000000000000135 netapp_mes.log
bcomd.log.00000000000000000136 netapp_mgmt.log
cm-daemon.log.00000000000000000149 netsetup.log
cm-daemon.log.00000000000000000150 notifyd.log.00000000000000002259
cro_proxy.log.00000000000000000024 notifyd.log.00000000000000002260
cro_proxy.log.00000000000000000025 perfstatd.log.00000000000000000050
crs.log.00000000000000000218 perfstatd.log.00000000000000000051
crs.log.00000000000000000219 php-fpm.access.log.00000000000000001119
debug.log.00000000000000000043 php-fpm.access.log.00000000000000001120
debug.log.00000000000000000044 php-fpm.error.log.00000000000000000001
fpolicy.log.00000000000000000029 php-fpm.error.log.00000000000000000002
fpolicy.log.00000000000000000030 php.log.00000000000000000001
hashd.log.00000000000000000005 secd.log.00000000000000000124
hashd.log.00000000000000000006 secd.log.00000000000000000125
kmip2_client.log.00000000000000000080 servprocd.log.00000000000000000248
kmip2_client.log.00000000000000000081 servprocd.log.00000000000000000249
licensed.log.00000000000000000027 spmd.log.00000000000000000077
licensed.log.00000000000000000028 spmd.log.00000000000000000078
memsnap.log syslog_CA_info
messages.log.00000000000000000063 syslog_dest_info
messages.log.00000000000000000064 sysmgr.log.00000000000000012746
mgwd-start.log sysmgr.log.00000000000000012747
mgwd-start.log.1 utx.lastlogin
mgwd.log.00000000000000000556 utx.log
mgwd.log.00000000000000000557 vifmgr.log.00000000000000000240
mlogd-start.log vifmgr.log.00000000000000000241
mlogd-start.log.1 vldb.log.00000000000000000171
mlogd_stats.0.log vldb.log.00000000000000000172
mlogd_stats.1.log vserverdr.log.00000000000000000742
mlogd_stats.2.log vserverdr.log.00000000000000000743
mount_spin.log ypbind.log.00000000000000000001
ndmpd.log.00000000000000000003 ypbind.log.00000000000000000002
ndmpd.log.00000000000000000004

mikesmit

Deleting files is a good temporary solution. I dug a little deeper and found out that the performance archiver was filling up vol0 on my simulator also. I found this kb article which describes symptoms similar to what I was seeing (KB article 162421 . The following commands dramatically reduced the space consumed in vol0 on my simulator and the added side benefit was that I didn't have to continually increase the size of vol0. 
Reduce percentage of vol0 disk space that is used for perf archive from 5% (Default) to 2% (Absolute minimum):
statistics archive config modify -datastore-max-percent 2
Reduce the number of days to retain the perf archive data for to 5 from 15 (default):
statistics archive config modify -datastore-max-retention 5

This is what the output looks like after the changes are made:

windy::*> statistics archive config show

Is the Performance Archive Enabled?: true
Base Path of Performance Archive Datastore: /mroot/etc/log/stats/ccma
Days to Retain Performance Archive Data: 5
Maximum Percentage of Root Volume Used for Performance Archive Data: 2%
Maximum Size of Performance Archive Data on Disk: 36.62MB
Minutes Between Performance Archive Data File Rotations: 2
Minutes Between Perfstat Collections: 0
Date Perfstat Collection Will Be Turned off: -
Minutes Between OPM Data File Rotations: 5
Minutes Between ASUP Data File Rotations: 30
Days to Retain ASUP Archive Data: 28
Data ONTAP Version of Installed Configuration: 9.7.0
Enabled Performance Preset Names: asup-hourly, asup_cfg,
copy_offload, css, default,
mcc_performance, nas, opm,
ps, san, statit, sysstat_all,
sysstat_mp, waffinity_stats,
wafl_bufstats, wafl_cpm,
wafl_spinhi, wafl_susp

 

Announcements
NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner
Public