I am running FreeNAS 0.7 (rev. 3514) since several weeks as my 'production' server at home on a Mini-ITX Atom based system (see FreeNAS 0.7 on a Intel D945GCLF).
I've done some tuning of FreeBSD and ZFS with good experiences (good performance, no panics, etc.).
Here is what I've done...
First of all it is important to 'tune' ZFS. I've seen some panics of my systems without using this parameters. It is necessary to use lots of RAM for ZFS. I have 2GB in my little server...
I've followed the FreeBSD ZFSTuning Guide and added the following lines to the /boot/loader.conf file
vm.kmem_size_max="1073741824"
vm.kmem_size="1073741824"
vfs.zfs.prefetch_disable=1
I am not sure about the vfs.zfs.prefetch_disable=1, but as I said, my experiences are good with this settings.
Editing of the /boot/loader.conf file is easy :-) Use your Web-GUI, go to -> Advanced -> Edit File -> add /boot/loader.conf to the File path and hit Load. Add the lines to that file and hit the Save Button.
Please be aware that these changes are not saved to the 'XML system configuration' (-> System -> Backup/Restore -> Download configuration).
FreeBSD tuning... I've simply used the 'Tuning' of FreeNAS. You can find that also on your Web-GUI.
Go to -> System -> Advanced -> mark the Tuning Box and hit Save.
And last but not least there are some nice tuning variables to be found on the TCP Tuning Guide for FreeBSD (here http://acs.lbl.gov/TCP-tuning/FreeBSD.html or here http://fasterdata.es.net/TCP-tuning/FreeBSD.html)
Add this variables via -> System -> Advanced -> sysctl.conf
Here are the variables in detail:
# ups spinup time for drive recognition
hw.ata.to=15
# System tuning - Original -> 2097152
kern.ipc.maxsockbuf=16777216
# System tuning
kern.ipc.nmbclusters=32768
# System tuning
kern.ipc.somaxconn=8192
# System tuning
kern.maxfiles=65536
# System tuning
kern.maxfilesperproc=32768
# System tuning
net.inet.tcp.delayed_ack=0
# System tuning
net.inet.tcp.inflight.enable=0
# System tuning
net.inet.tcp.path_mtu_discovery=0
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html
net.inet.tcp.recvbuf_auto=1
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html
net.inet.tcp.recvbuf_inc=16384
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html
net.inet.tcp.recvbuf_max=16777216
# System tuning
net.inet.tcp.recvspace=65536
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html
net.inet.tcp.rfc1323=1
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html
net.inet.tcp.sendbuf_auto=1
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html
net.inet.tcp.sendbuf_inc =8192
# System tuning
net.inet.tcp.sendspace=65536
# System tuning
net.inet.udp.maxdgram=57344
# System tuning
net.inet.udp.recvspace=65536
# System tuning
net.local.stream.recvspace=65536
# System tuning
net.local.stream.sendspace=65536
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html
net.inet.tcp.sendbuf_max=16777216
I would really appreciate any comments about this tuning variables!
15 comments:
Here's a question for you though:
Do I really want to be running the 0.7 Alpha build and putting data I care about on it?
As far as I can tell you, I am running my server at home since serveral weeks on 0.7 and I am really happy with it. But... I have also a proper backup :-)
The line "vfs.zfs.prefetch_disable=1" must probably refer to a somewhat strange behavior in ZFS.
When the host issues a read of size smaller than 16k, it is expanded by ZFS to a read size of 64k, in an (horrible) attempt to prefetch some data.
This "feature" has been corrected now at least in OpenSolaris, with ZFS prefetching only metadata reads, but i don't really know if it's been corrected in OpenBSD. So this must be the reason behing this parameter.
Hi Harry,
thank you for sharing your experience.
Would you mind posting the advanced system tuning entries in text, additionally? Would save me a lot of time (just copy&paste)
Cu
Hi Falk,
I've added the entries in the blog-post.
Regards, harryd
Where are the amd64 0.7 builds you are using? they are not in nightly builds any more ...
Thanks
You are right! At the moment there is only a i386 version of build 3953. You might ask Voler or Olivier for this version...
Regards, harryd
Hi,
Nice job.
How do you change /boot/loader.conf parameters?
I've got mine running in compact flash, so loader.conf is in
/cf/boot/loader.conf
i have to unmount /cf make the changes and then reboot. I have done this for my raid controller.
But when i placed
vm.kmem_size_max="1073741824"
vm.kmem_size="1073741824"
vfs.zfs.prefetch_disable=1
in my loader.conf then my system cannot reboot.
it's giving me a bunch of on screen error messages.
Any ideas?
Teo
If you use the embedded installation, you are right, you need to add the lines to /cf/boot/loader.conf. You can do this like I've described it for the /boot/loader.conf. It is not necessary to umount /cf...
Can you tell me more about the error-message? Maybe you can open a new thread in the FreeNAS-Forum...
Thanks for the helpful tips. I am running 0.7RC2 Khasadar (revision 4856) , booting from a CD and saving config to a USB drive. I can't seem to find the loader.conf. Any clues?
freenas:/boot# ls /boot
boot boot0 boot1 defaults kernel mbr zfs
freenas:/# find / -name "*.conf" -ls | grep load
freenas:/# find / -name "*.conf" -ls | grep light
3431 8 -rw-r--r-- 1 root wheel 2261 Aug 11 03:35 /var/etc/lighttpd.conf
freenas:/#
Please have a look at /cf/boot ;-) (You have a embedded installation?)
I'm curious how much of a difference any of these changes made. I know you've been running ok without any problems, what I'm curious about is what sort of performance change did you see as a result of your changes. Do you have any before and after numbers?
As this post is older, I have no numbers. Sry... But it is a good hint for the future :-)
Hi!
Thanks for some nice info regarding FreeNAS. I would just like to point out a small typo in your text. Though it may seem obvious to many, some may fail because of it.
The line reads "/boof/loader.conf" just above your first picture in the post.
Thanks, and hope your NAS is running fine! :)
Hi Thomas, thanks for this. I've changed this typo ;-)
Post a Comment