harryd

ZFS Pool Versions

2011-05-09T20:56:00.010+02:00

I was looking for an overview of the available ZFS Pool Versions. The Opensolaris XWiki Page of the ZFS Group show on page for every page, but not an complete overview.

History of FreeBSD releases with ZFS is as follows:

7.0+ - original ZFS import, ZFS v6; requires significant tuning for stable operation (no longer supported)

7.2 - still ZFS v6, improved memory handling, amd64 may need no memory tuning (no longer supported)

7.3+ - backport of new ZFS v13 code, similar to the 8.0 code

8.0 - new ZFS v13 code, lots of bug fixes - recommended over all past versions. (no longer supported)

8.1+ - ZFS v14

8.2+ - ZFS v15

9.0+ - ZFS v28

Here I list of all current Versions until today 9-May-2011

ZFS Pool Version 31

This version includes support for improved 'zfs list' performance.

Pool version 31 is available in this release:

Nevada, build 150

The change record for the version 31 change is:

6980669 zfs rename -r is slow, causes appliance UI to hang

ZFS Pool Version 30

This version includes support for ZFS encryption.

Pool version 30 is available in this release:

Nevada, build 149

The change record for the version 30 change is:

4854202 ZFS data set encryption

ZFS Pool Version 29

This version includes support for the RAID-Z/mirror hybrid allocator.

Pool version 29 is available in this release:

Nevada, build 148

The change record for the version 29 change is:

6977913 RAID-Z/mirror hybrid allocator

ZFS Pool Version 28

This version includes support for multiple virtual device replacements.

Pool version 28 is available in this release:

Nevada, build 147

The change record for the version 28 change is:

6782540 zpool cannot replace a replacing device

ZFS Pool Version 27

This version includes support for improved snapshot creation performance.

Pool version 27 is available in this release:

Nevada, build 145

The change record for the version 27 change is:

6844896 recursive snapshots take a long time

ZFS Pool Version 26

This version includes support for improved snapshot deletion performance.

Pool version 26 is available in this release:

Nevada, build 141

The change record for the version 26 change is:

6948890 snapshot deletion can induce pathologically long spa_sync() times

ZFS Pool Version 25

This version includes support for improved pool scrubbing and resilvering statistics.

Pool version 25 is available in this release:

Nevada, build 140

The change record for the version 25 change is:

6391915 provide interval arg to zpool status to monitor resilvering

ZFS Pool Version 24

This version includes support for system attributes.

Pool version 24 is available in this release:

Nevada, build 137

The change records for the version 24 change are:

6716117 ZFS needs native system attribute infrastructure

6516171 zpl symlinks should have their own object type

ZFS Pool Version 23

This version includes support for the slim ZIL.

Pool version 23 is available in this release:

Nevada, build 135

The change record for the version 23 change is:

6595532 ZIL is too talkative

ZFS Pool Version 22

This version includes support for zfs receive properties.

Pool version 22 is available in this release:

Nevada, build 128

The PSARC case for the version 22 change is:

PSARC/2009/510 ZFS Received Properties

ZFS Pool Version 21

This version includes support for ZFS deduplication properties.

Pool version 21 is available in this release:

Nevada, build 128

The PSARC case for the version 21 change is:

PSARC/2009/571 ZFS Deduplication Properties

ZFS Pool Version 20

This version includes the zle compression algorithm that is needed to support the ZFS deduplication properties in ZFS pool version 21. Both pool versions are available in this release:

Nevada, build 128

The PSARC case for the version 20 change is:

PSARC/2009/571 ZFS Deduplication Properties

ZFS Pool Version 19

This version includes support for the following feature:

ZFS log device removal

This feature is available in:

Nevada, build 125

The related change record for the version 19 change is:

6574286 removing a slog doesn't work

ZFS Pool Version 18

This version includes support for the following feature:

ZFS snapshot holds

This feature is available in:

Nevada, build 121

The related change record for the version 18 change is:

6803121 want user-settable refcounts on snapshots

ZFS Pool Version 17

This version includes support for the following feature:

triple-parity RAID-Z

This feature is available in:

Nevada, build 120

The related change record for the version 17 change is:

6854612 triple-parity RAID-Z

ZFS Pool Version 16

This version includes support for the following feature:

STMF property support

This feature is available in:

Nevada, build 116

The related bug for the version 16 change is:

6736004 ZFS volumes need an additional property for COMSTAR support

ZFS Pool Version 15

This version includes support for the following features:

userused@... and groupused@... properties

userquota@... and groupquota@... properties

These features are available in:

Nevada, build 114

The related bug and PSARC case for version 15 changes are:

6501037 want user/group quotas on ZFS

PSARC/2009/204 ZFS user/group quotas and space accounting

ZFS Pool Version 14

This version includes support for the following feature:

passthrough-x aclinherit property support

This feature is available in:

OpenSolaris 2009.06, Nevada, build 103

The related bug and PSARC case for the version 14 change are:

6765166 Need to provide mechanism to optionally inherit ACE_EXECUTE

PSARC 2008/659 New ZFS "passthrough-x" ACL inheritance rules

ZFS Pool Version 13

This version includes support for the following features:

usedbysnapshots property

usedbychildren property

usedbyrefreservation property

usedbydataset property

These features are available in:

OpenSolaris 2008.11, Nevada, build 98

The related bug and PSARC case for version 13 change are:

6730799 want user properties on snapshots

PSARC/2008/518 ZFS space accounting enhancements

ZFS Pool Version 12

This version includes support for the following feature:

Snapshot properties

This feature is available in:

OpenSolaris 2008.11, Nevada build 96

The related bug for the version 12 change is:

6701797 want user properties on snapshot

ZFS Pool Version 11

This version includes support for the following feature:

Improved zpool scrub / resilver performance

This feature is available in:

OpenSolaris 2008.11, Nevada, build 94

The related bug for the version 11 change is:

6343667 scrub/resilver has to start over when a snapshot is taken

Note: this bug is fixed when using build 94 even with older pool versions. However, upgrading the pool can improve scrub performance when there are many file systems, snapshots, and clones.

ZFS Pool Version 10

This version includes support for the following feature:

Devices can be added to a storage pool as "cache devices." These devices provide an additional layer of caching between main memory and disk. Using cache devices provides the greatest performance improvement for random read-workloads of mostly static content.

This feature is available in Nevada, build 78.

The Solaris 10 10/08 release includes ZFS pool version 10, but support for cache devices is not included in this Solaris release.

The related bug for the version 10 change is:

6536054 second tier ("external") ARC

ZFS Pool Version 9

This version includes support for the following features:

In addition to the existing ZFS quota and reservation features, this release includes dataset quotas and reservations that do not include descendent datasets, such as snapshots and clones, in the space consumption. ("zfs set refquota" and "zfs set refreservation".)

A reservation is automatically set when a non-sparse ZFS volume is created that matches the size of the volume. This release provides an immediate reservation feature so that you set a reservation on a non-sparse volume with enough space to take snapshots and modify the contents of the volume.

CIFS server support

These features are available in Nevada, build 77.

The related bugs for version 9 changes are:

6431277 want filesystem-only quotas

6483677 need immediate reservation

6617183 CIFS Service - PSARC 2006/715

ZFS Pool Version 8

This version now supports the ability to delegate zfs(1M) administrative tasks to ordinary users.

This feature is available in:

Nevada, build 69

Solaris 10 10/08 release

The related bug for the version 8 change is:

6349470 investigate non-root restore/backup

ZFS Pool Version 7

This version includes support for the following feature:

The ZFS Intent Log (ZIL) satisfies the need of some applications to know the data they changed is on stable storage on return from a system call. The Intent Log holds records of those system calls and they are replayed if the system power fails or panics if they have not been committed to the main pool. When the Intent Log is allocated from the main pool, it allocates blocks that chain through the pool. This version adds the capability to specify a separate Intent Log device or devices.

This feature is available in:

Nevada, build 68

Solaris 10 10/08 release

The related bug for the version 7 change is:

6339640 Make ZIL use NVRAM when available

ZFS Pool Version 6

This version includes support for the following feature:

'bootfs' pool property

This feature is available in:

Nevada, build 62

Solaris 10 10/08 release

The related bugs for version 6 changes are as follows:

4929890 ZFS boot support for the x86 platform

6479807 pools need properties

ZFS Pool Version 5

This version includes support for the following feature:

gzip compression for ZFS datasets

This feature is available in:

Nevada, build 62

Solaris 10 10/08 release

The related bug for the version 5 changes is:

6536606 gzip compression for ZFS

ZFS Pool Version 4

This version includes support for the following feature:

zpool history

This feature is available in:

Nevada, build 62

Solaris 10 8/07 release

The related bugs for version 4 changes are as follows:

6529406 zpool history needs to bump the on-disk version

6343741 want to store a command history on disk

ZFS Pool Version 3

This version includes support for the following features:

Hot spares

Double-parity RAID-Z (raidz2)

Improved RAID-Z accounting

These features are available in:

Nevada, build 42

Solaris 10 11/06 release, (build 3)

The related bugs for version 3 changes are as follows:

6405966 Hot Spare support in ZFS

6417978 double parity RAID-Z a.k.a. RAID6

6288488 du reports misleading size on RAID-Z

ZFS Pool Version 2

This version includes support for "Ditto Blocks", or replicated metadata. Due to the tree-like structure of the ZFS on-disk format, an uncorrectable error in a leaf block may be relatively benign, while an uncorrectable error in pool metadata can result in an unopenable pool. This feature introduces automatic replication of metadata (up to 3 copies of each block) independent of any underlying pool-wide redundancy. For example, on a pool with a single mirror, the most critical metadata will appear three times on each side of the mirror, for a total of six copies. This ensures that while user data may be lost due to corruption, all data in the pool will be discoverable and the pool will still be usable. This will be expanded in the future to allow user data replication on a per-dataset basis.

This feature integrated on 4/10/06 with the following bug fix:

6410698 ZFS metadata needs to be more highly replicated (ditto blocks)

This feature is available in:

Nevada, build 38

Solaris 10 10/06 release (build 09)

ZFS Pool Version 1

This is the initial ZFS on-disk format as integrated on 10/31/05. During the next six months of internal use, there were a few on-disk format changes that did not result in a version number change, but resulted in a flag day since earlier versions could not read the newer changes. The first official releases supporting this version are:

Nevada, build 36

Solaris 10 6/06 release

Earlier releases may not support this version, despite being formatted with the same on-disk number. This is due to:

6389368 fat zap should use 16k blocks (with backwards compatibility)

6390677 version number checking makes upgrades challenging

Where is harryd? -- follow-up

2011-05-07T07:42:00.000+02:00

Nearly one year ago I've written that we build our own house. Now after a lot work is behind us, we are in and I have more time than I had in the past year. Currently I am preparing a blogpost. If you think that might has something to do with FreeNAS 8 and a benchmark, than you might be right. So... stay tuned...

A Closer Look at ZFS, Vdevs and Performance

2010-06-07T17:07:00.001+02:00

Constantin gives here an great introduction into ZFS, vdevs and performance.

FreeNAS vs Nexentastor

2010-05-31T14:36:00.005+02:00

I've tested the performance of FreeNAS 0.7.1 stable (Build 5127) and Nexentastor Community Edition 3.0.2. I will not dig into the different features or the details of both WebGUIs.

For the tests I've used the following hardware

Server

CPU - Intel Core2Duo E6300 - 1.8 GHz - 2 MB Cache

RAM - 2 GByte

Mainboard - Intel DG965WH

Harddisks - 3x WesternDigital WD3200 (320 GByte connected to the onboard SATA2)

Client

My good old MacBook Pro (late 2008), 4 GByte RAM, Mac OS X 10.6.3

Software
iSCSI initiator - Global SAN iSCSI initiator (4.0.0.204)
Benchmark Software - XBench

Benchmark Software - Helios LAN Test

Benchmark Software - iozone (MacPorts)

Protocols

CIFS
NFS
AFP (currently only available for FreeNAS)
iSCSI

Benchmarks
Xbench - The best out of 5 results
Helios LAN Test - 10 tests, two times, best result
iozone - options -e -i0 -i1 -i2 -+n -r 256k -s2g -t4 -c -x

Tuning

No tuning on the Nexentastor. For FreeNAS I've enabled in the WebGUI System | Advanced | Tuning (enable tuning of some kernel variables)

CIFS
Keep in mind, I've tested CIFS with the built-in smb client of OSX. A native Windows OS might provide different results.

Tuning for FreeNAS (via WebGUI)

Services | CIFS/SMB | Settings

-> Send and Receive Buffer Size to 65535

-> Enable large read/write

-> Auxiliary parameters : max xmit = 65535

-> Auxiliary parameters : socket options = TCP_NODELAY SO_SNDBUF=65535 SO_RCVBUF=65535

If you compare the CIFS results you will see that NexentaStor is slightly faster than FreeNAS

NFS
I wasn't able to get XBench to run on a FreeNAS NFS mountpoint. So you wont see any results on the XBench charts.

With the missing XBench results for FreeNAS I would say NexentaStor is here the winner. It provides slightly higher throughput rates.

AFP
NexentaStor doesn't provide this protocol. As I am a Mac user it is very useful to have a system serving data via AFP. Especially the support of FreeNAS for time machine is IMHO excellent (see my blogposts about backup and restore using FreeNAS and time machine)

FreeNAS is in this disciplin the clear winner.

iSCSI
There was also an issue to get iozone to run on a FreeNAS iSCSI target.

The iSCSI protocol shows several advantages compared to the other. It shows high throughput rates with a low latency. But it is not usable to share the LUN to several systems at the same time (it is possible with special Filesystems, I know, but I think this is not very common). With the lack of the iozone results for FreeNAS I would also say, Nexentastor is here the winner.

Overall comparisson

Conclusion
FreeNAS and Nexenta is on the performance side very much comparable. NexentaStor is slitghly faster in most of the disciplines, but the lack of AFP make them both overall equal.
FreeNAS is easy to setup and has a very clear and structured WebGUI. Nexenta has more features (e.g. built in snapshots, deduplication, etc.)

At the moment I definitely will stay with FreeNAS as the performance is comparable and I really like the integration of Time Machine via AFP.

P.S.: If you are interested in a detailed Excel-Sheet with more details, please drop me a mail harryd71 at users dot sourceforge dot net

Flattr

2010-05-30T17:51:00.006+02:00

I've stumbled upon flattr :-)

Flattr is a social micropayment platform that lets you show love for the things you like.

As I've always looked for a service like this to show other people my respect and want to give them something back, I think this might be the right platform to do this. I've added a button to each of my blogposts and if you like what you read, you can press this button.

Of course, you need to be a user of this service, so please register for a beta invite or ask me at harryd71 at users dot sourceforge dot net or send me a direct message on twitter.

Many thanks for your support!

P.S.: Nico showed on this blogpost how to add a flattr button to blogger. Many thanks, Nico (of course, I've flattr'd you :-) )

Where is harryd?

2010-05-28T08:57:00.001+02:00

Maybe you've recognized that I am not very active here or at the FreeNAS forum.

The reason behind this... I build together with my wife, our own house. She is the architect and there is a lot of work to do. To be honest... If you see our piece of land, you won't see too much, as the craftsmen just begun. But there is a lot of work to do behind the scenes.

The next few weeks / months will be very exciting, as we can see how our dream is built in stone and wood (and all the other materials). There is a lot of work to do for me, as I will do the whole electricity (and a few other things).

I've prepared another blogpost and I hope I'm able to finish it soon. So, please stay tuned and thanks for all your feedback!

Mac OS X system restore using time machine and FreeNAS

2010-05-26T23:19:00.019+02:00

I'd like to show you in this post how you can recover your Mac from a time machine backup using FreeNAS. If you've set up time machine as I've described here, you should be able to do a complete system restore.

I've used a virtual machine using VirtualBox 3.2.0 on my MacMini to demonstrate this. VirtualBox supports now to install Mac OS X guests (maybe not for a very long time, AFAIK this is not allowed by Apple.)

1. If you have an issue with your system and you want to do a complete restore (or disaster recovery), please boot from the original boot DVD

2. Choose 'Restore System From Backup...' the 'Utilities' menu

3. Click 'Continue'. The next screen will show you the available sources.

4. After entering your login credentials you should see the backup source for your system

5. The next screen shows you the available backups of your system. Select the one you've want to restore

6. Select the destination disk and click 'restore' read the warning and 'continue'

7. And finally restart the system

Lots of information about ZFS and storage

2010-05-04T10:50:00.002+02:00

Seeding FreeNAS 0.7.1 stable (5127)

2010-05-01T14:57:00.001+02:00

You will find the .torrent files here -> TBP...

Please help to seed

Seeding FreeNAS 0.7 stable (4919)

2010-01-12T10:21:00.000+01:00

You will find the .torrent files here -> TBP...

Please help to seed!

Episode about the future of FreeNAS on bsdtalk

2009-12-16T21:20:00.002+01:00

You will find here an excellent talk with Josh Paetzel from iXsystems about the future of FreeNAS. Many thanks for this information!

The NEW Future of FreeNAS...

2009-12-05T06:51:00.002+01:00

Olivier left me here a comment...

The FreeNAS project will continue to use FreeBSD in the future! This sounds excellent! Please read the original message on the blog and forum post.

Here is a copy of the message:

Hi all,

FreeNAS needs some big modification for removing its present limitation (one of the biggest is the non support of easly users add-ons).
We think that a full-rewriting of the FreeNAS base is needed. From this idea, we will take 2 differents paths:

Volker will create a new project called "'OpenMediaVault" based on a GNU/Linux using all its experience acquired with all its nights and week-ends spent to improve FreeNAS during the last 2 years. He still continue to work on FreeNAS (and try to share its time with this 2 projects).
And, a great surprise: iXsystems, a company specialized in professional FreeBSD offers to take FreeNAS under their wings as an open source community driven project. This mean that they will involve their professionals FreeBSD developers to FreeNAS! Their manpower will permit to do a full-rewriting of FreeNAS.

Personally, I come back to actively work in FreeNAS and begin to upgrade it to FreeBSD 8.0 (that is "production ready" for ZFS).

So we will see 'more' developers working on FreeNAS. Excellent!

Mac OSX Time Machine and FreeNAS 0.7

2009-11-23T22:38:00.019+01:00

Since FreeNAS 0.7 it is easy to configure TimeMachine to use a AFP share (like Apple's Time Capsule)... Here is a short howto :-)

First step: Configure FreeNAS

-> Enable AFP

-> Configure the share

Automatic disk discovery - Enable automatic disk dicovery
Automatic disk discovery mode - Time Machine

Second Step: Configure OSX Time Machine

-> Select System Preferences -> Time Machine

-> Select Backup Disk
-> and Authenticate
Thats it!

Your FreeNAS will work now similar than a TimeCapsule. Enjoy! Details can be found here...

I've combined this with ZFS. With two clients and regular backups since two months, I have here a compression (gzip) enabled volume with 238 GByte. The compression ratio (compressratio) is at 1.24x

freenas:~# zfs get compressratio data0/timemachine
NAME PROPERTY VALUE SOURCE
data0/timemachine compressratio 1.24x -

Instead of 295 GByte, only 238 GBytes are used. IMHO this is great :-)

If you want to see how to do a complete restore of your system, please see my new blogpost "Mac OS X system restore using time machine and FreeNAS"

Overview of NAS Operation Systems

2009-11-22T22:13:00.009+01:00

CryptoNAS (http://cryptonas.org/) - Free
EON (http://sites.google.com/site/eonstorage/, Blog http://eonstorage.blogspot.com/) - Free
EuroNAS (http://euronas.de/web/index.php) - Commercial
FreeNAS (http://www.freenas.org/) - Free
Napp-it (http://www.napp-it.org/) - Free
NASLite (http://www.serverelements.com/naslite.php) - Commercial
NexentaStor (http://www.nexenta.com/) - Commercial (Developer Edition is restricted to 4TB)
Open-E (http://www.open-e.com/) - Commercial (Lite Version is restricted to 2TB)
OpenFiler (http://www.openfiler.com/) - Free (Commercial support is available)
PulsarOS (http://pulsaros.digitalplayground.at/) - Free (?), not released yet
UnRAID (http://www.lime-technology.com) - Commercial (Basic Version is restricted to 3 Drives)

Do you have any experiences with one of those?

The Future of FreeNAS...

2009-11-20T23:00:00.006+01:00

Please read here to get the latest news...

Read through this forum thread to see the actual discussion. In short...

FreeNAS 0.8 will be based on Debian GNU/Linux! Volker (the core developer) started an intermediate project called CoreNAS. FreeNAS 0.8 will be based on that.

Here is a short list of pros by Volker:

- Text and graphical installer that can be customized. This means no hand written install scripts anymore which causes some problems in FreeNAS

- WOL works in Linux

- lmsensor - A WORKING sensor framework which is a really needed feature in FreeNAS to check the CPU/MB temps and fan speeds

- Better Samba performance

- Ability to implement HA features

- System can be updated via 'apt-get' or any other deb package manager

- Better driver support

- Maybe 'ZFS' over FUSE (there is already one commercial product available that uses this feature)

- NFS4

- ...

What does this mean to me? I definetly want to have an OS which supports native ZFS (zfs on fuse is not an option). I don't need an extra small footprint or all of these 'special' features that some users of FreeNAS frequently requested (boinc, printserver...). I really appreciated the wonderful WebGUI of FreeNAS but if it is necessary, I'm not bound to it. As I have quite a lot experience with solaris, I will switch to Opensolaris. It has all I need except the AFP support. So I need to get more experience with Netatalk.

I am also really looking forward for the new features of ZFS like deduplication and crypto. If you are intersted about my experiences, you will find some here...

Nevertheless I would like to say many, many thanks for this wonderful piece of software!

FreeNAS 0.7 - Samba tuning

2009-10-21T21:45:00.003+02:00

Here is a nice blogpost from learnedbyerror about tuning samba (and other...). Especially the samba/cifs tweaks should give you a performance boost.

I don't recommend to use the old zfs tuning settings as the latest FreeNAS Versions are based on FreeBSD 7.2 (see http://wiki.freebsd.org/ZFSTuningGuide).

Anyway, good blogpost!

FreeNAS 0.7, ZFS snapshots and scrubbing

2009-10-09T21:20:00.007+02:00

I've described here how I am doing snapshots. Cron and a little script works perfect, but I also scrub from time to time the zpools. But the scrub never finishes :-(

The reason behind this is described here. Scrubbing or resilvering a pool starts over when a snapshot is taken.

So it is necessary to change the script that creates the snapshots

snapshot_hourly.sh

#!/bin/sh

# If a scrub is running, don't do a snapshot (otherwise scrub will restarts at 0%)

pools=$(zpool list -H -o name)

for pool in $pools;

if zpool status -v $pool | grep -q "scrub in progress"; then

exit

done

# Destroy the old snapshot and create the new

zfs destroy $1@hourly.`date "+%H"` > /dev/null 2>&1

zfs snapshot $1@hourly.`date "+%H"`

snapshot_daily.sh

#!/bin/sh

# If a scrub is running, don't do a snapshot (otherwise scrub will restarts at $

pools=$(zpool list -H -o name)

for pool in $pools;

if zpool status -v $pool | grep -q "scrub in progress"; then

exit

done

# Destroy the old snapshot and create the new

zfs destroy $1@daily.`date "+%a"` > /dev/null 2>&1

zfs snapshot $1@daily.`date "+%a"`

and snapshot_weekly.sh (please be aware that I use this script to keep snapshots of the last twelve weeks. If necessary, you need to change this...)

#!/bin/sh

# If a scrub is running, don't do a snapshot (otherwise scrub will restarts at $

pools=$(zpool list -H -o name)

for pool in $pools;

if zpool status -v $pool | grep -q "scrub in progress"; then

exit

done

# Destroy the oldest snapshot, rotate the other and create the new

zfs destroy $1@weekly.12 > /dev/null 2>&1

zfs rename $1@weekly.11 @weekly.12 > /dev/null 2>&1

zfs rename $1@weekly.10 @weekly.11 > /dev/null 2>&1

zfs rename $1@weekly.9 @weekly.10 > /dev/null 2>&1

zfs rename $1@weekly.8 @weekly.9 > /dev/null 2>&1

zfs rename $1@weekly.7 @weekly.8 > /dev/null 2>&1

zfs rename $1@weekly.6 @weekly.7 > /dev/null 2>&1

zfs rename $1@weekly.5 @weekly.6 > /dev/null 2>&1

zfs rename $1@weekly.4 @weekly.5 > /dev/null 2>&1

zfs rename $1@weekly.3 @weekly.4 > /dev/null 2>&1

zfs rename $1@weekly.2 @weekly.3 > /dev/null 2>&1

zfs rename $1@weekly.1 @weekly.2 > /dev/null 2>&1

zfs snapshot $1@weekly.1

I've found a script from http://hype-o-thetic.com to run a scrub regularly. See his blogpost here...

Just for backup reasons, I post this script below...

#!/bin/bash

#VERSION: 0.2
#AUTHOR: gimpe
#EMAIL: gimpe [at] hype-o-thetic.com
#WEBSITE: http://hype-o-thetic.com
#DESCRIPTION: Created on FreeNAS 0.7RC1 (Sardaukar)
# This script will start a scrub on each ZFS pool (one at a time) and
# will send an e-mail or display the result when everyting is completed.

#CHANGELOG
# 0.2: 2009-08-27 Code clean up
# 0.1: 2009-08-25 Make it work

#SOURCES:
# http://aspiringsysadmin.com/blog/2007/06/07/scrub-your-zfs-file-systems-regularly/
# http://www.sun.com/bigadmin/scripts/sunScripts/zfs_completion.bash.txt
# http://www.packetwatch.net/documents/guides/2009073001.php

# e-mail variables
FROM=from@devnull.com
TO=to@devnull.com
SUBJECT="$0 results"
BODY=""

# arguments
VERBOSE=0
SENDEMAIL=1
args=("$@")
for arg in $args; do
case $arg in
"-v" | "--verbose")
VERBOSE=1
;;
"-n" | "--noemail")
SENDEMAIL=0
;;
"-a" | "--author")
echo "by gimpe at hype-o-thetic.com"
exit
;;
"-h" | "--help" | *)
echo "
usage: $0 [-v --verbose|-n --noemail]
-v --verbose output display
-n --noemail don't send an e-mail with result
-a --author display author info (by gimpe at hype-o-thetic.com)
-h --help display this help
"
exit
;;
esac
done

# work variables
ERROR=0
SEP=" - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "
RUNNING=1

# commands & configuration
ZPOOL=/sbin/zpool
PRINTF=/usr/bin/printf
MSMTP=/usr/local/bin/msmtp
MSMTPCONF=/var/etc/msmtp.conf

# print a message
function _log {
DATE="`date +"%Y-%m-%d %H:%M:%S"`"
# add message to e-mail body
BODY="${BODY}$DATE: $1\n"

# output to console if verbose mode
if [ $VERBOSE = 1 ]; then
echo "$DATE: $1"
fi
}

# find all pools
pools=$($ZPOOL list -H -o name)

# for each pool
for pool in $pools; do
# start scrub for $pool
_log "starting scrub on $pool"
zpool scrub $pool
RUNNING=1
# wait until scrub for $pool has finished running
while [ $RUNNING = 1 ]; do
# still running?
if $ZPOOL status -v $pool | grep -q "scrub in progress"; then
sleep 60
# not running
else
# finished with this pool, exit
_log "scrub ended on $pool"
_log "`$ZPOOL status -v $pool`"
_log "$SEP"
RUNNING=0
# check for errors
if ! $ZPOOL status -v $pool | grep -q "No known data errors"; then
_log "data errors detected on $pool"
ERROR=1
fi
fi
done
done

# change e-mail subject if there was error
if [ $ERROR = 1 ]; then
SUBJECT="${SUBJECT}: ERROR(S) DETECTED"
fi

# send e-mail
if [ $SENDEMAIL = 1 ]; then
$PRINTF "From:$FROM\nTo:$TO\nSubject:$SUBJECT\n\n$BODY" | $MSMTP --file=$MSMTPCONF -t
fi

M0n0wall and SixXS IPv6 - My first time...

2009-07-17T22:24:00.003+02:00

I have at home a little Ffirewall running on a VIA EPIA 5000 Mini-ITX board. It is the m0n0wall which runs since more than one year on this board without any issues.

As I want to join the IPv6 community, I've searched which is the best way to get 'connected'. And I've found sixxs.net.

The latest Beta-Release (1.3b16) supports AICCU of SixXS. The upgrade from the stable version worked flawless...

After I've created a user account at sixxs.net I was able to request a tunnel. As I have here a little router (Speedport W 701V from T-Online) in front of the firewall, it was not possible to get the tunnel to work in both ways. It was possible to connect my firewall to sixxs.net, but it was not possible to send a ping6 to my firewall. So... no alive packet reached my firewall.

Currently I've enabled the PPPOE-Passthrough on this router (and disabled the router function by deleting the login information).

Everything works perfect (I can ping IPv6 addresses from my firewall). Right no, I wait for more ISKs to request my own IPv6 subnet.

Stay tuned...

Benchmark of FreeNAS 0.7 and a single SSD

2009-06-25T13:27:00.023+02:00

As you can read in my previous blog post, I have a SuperTalent Ultra Drive ME MLC 64GB. I am interested about the performance of this little drives. So here is what I've found out...

First of all I've installed this SSD into a server for doing tests.

Server hardware specs

CPU Intel Core2Quad Q6000 (2.4 GHz)
8 GB RAM
Mainboard Intel DG965WH
Network Card Intel Pro/1000 PT

FreeNAS 0.7 RC2 (AMD64, rev. 4744)

I've tuned the system a little bit :-)

-> Network|Interfaces|LAN -> Advanced Configuration -> MTU 9000

-> System|Advanced ->Tuning -> Enable tuning of some kernel variables

And -> System|Advanced|sysctl.conf

hw.ata.to = 15
# ATA disk timeout vis-a-vis power-saving

kern.coredump = 0
# Disable core dump

kern.ipc.maxsockbuf = 16777216
# System tuning - Original -> 2097152

kern.ipc.nmbclusters = 32768
# System tuning

kern.ipc.somaxconn = 8192
# System tuning

kern.maxfiles = 65536
# System tuning

kern.maxfilesperproc = 32768
# System tuning

net.inet.tcp.delayed_ack = 0
# System tuning

net.inet.tcp.inflight.enable = 0
# System tuning

net.inet.tcp.path_mtu_discovery = 0
# System tuning

net.inet.tcp.recvbuf_auto = 1
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html

net.inet.tcp.recvbuf_inc = 524288
# http://fasterdata.es.net/TCP-tuning/FreeBSD.html

net.inet.tcp.recvbuf_max = 16777216
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html

net.inet.tcp.recvspace = 65536
# System tuning

net.inet.tcp.rfc1323 = 1
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html

net.inet.tcp.sendbuf_auto = 1
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html

net.inet.tcp.sendbuf_inc = 16384
# http://fasterdata.es.net/TCP-tuning/FreeBSD.html

net.inet.tcp.sendbuf_max = 16777216
# http://acs.lbl.gov/TCP-tuning/FreeBSD.html

net.inet.tcp.sendspace = 65536
# System tuning

net.inet.udp.maxdgram = 57344
# System tuning

net.inet.udp.recvspace = 65536
# System tuning

net.local.stream.recvspace = 65536
# System tuning

net.local.stream.sendspace = 65536
# System tuning

net.inet.tcp.hostcache.expire = 1
# http://fasterdata.es.net/TCP-tuning/FreeBSD.html

Protocol, Client and benchmark programm

I've used AFP (Apple File Protocol), my new MacBook Pro (15", Core2Due 2.4 GHz, 4GB RAM, Unibody late 2008, I've lost my old one) and IOZONE.

IOZONE (Version 3.323 from macports)

Iozone is a w

onderful tool to benchmark filesystems, but it is also a feature monster. After some investigation I've found some useful options for my tests.

iozone -e -i0 -i1 -i2 -+n -r 64k -s8g -t2 -c -x

-e -- Include flush (fsync,fflush) in the timing calculations
-i0 -- Test write/rewrite
-i1 -- Test read/reread
-i2 -- Test random read/write
-+n -- Disable retests
-r 64k -- Record or block size
-s8g -- Size of the file to test (2x RAM of the client)
-t2 -- number of threads
-c -- Include close() in timing calculations
-x -- Turn of stone-walling (I've received slightly better results with turned this off, see iozone manpage for details)

First impression

The first tests didn't showed me the results I've expected! Diskinfo showed good resutls (140 MB/s read, low latency) but over the GBE-Network I was not able to get mor than 60 MB/s read and 80 MB/s write!

Troubleshooting

I've identified that the network switch I have here (Longshine LCS-GS8208A) was not able to deliver more speed. This is the reason why I've connected the MBP direct to the FreeNAS Server.

Screenshots

Write Performance - Traffic graph

A peak traffic of 890 Mbps isn't too bad for a single HDD

Read performance - Traffic graph

The read performance shows nearly constant bandwidth of more that 910 Mbps

IOZONE results

iozone -e -i0 -i1 -i2 -+n -r 64k -s8g -t2 -c -x

Iozone: Performance Test of File I/O
Version $Revision: 3.323 $
Compiled for 32 bit mode.
Build: macosx

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.

Run began: Tue Jun 23 15:47:17 2009

Include fsync in write timing
No retest option selected
Record Size 64 KB
File size set to 8388608 KB
Include close in write timing
Stonewall disabled
Command line used: iozone -e -i0 -i1 -i2 -+n -r 64k -s8g -t2 -c -x
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 2 processes
Each process writes a 8388608 Kbyte file in 64 Kbyte records

Children see throughput for 2 initial writers = 79800.84 KB/sec
Parent sees throughput for 2 initial writers = 78177.66 KB/sec
Min throughput per process = 39088.91 KB/sec
Max throughput per process = 40711.93 KB/sec
Avg throughput per process = 39900.42 KB/sec
Min xfer = 8388608.00 KB

Children see throughput for 2 readers = 107705.98 KB/sec
Parent sees throughput for 2 readers = 107705.48 KB/sec
Min throughput per process = 53852.83 KB/sec
Max throughput per process = 53853.15 KB/sec
Avg throughput per process = 53852.99 KB/sec
Min xfer = 8388608.00 KB

Children see throughput for 2 random readers = 56735.84 KB/sec
Parent sees throughput for 2 random readers = 56735.20 KB/sec
Min throughput per process = 28367.65 KB/sec
Max throughput per process = 28368.19 KB/sec
Avg throughput per process = 28367.92 KB/sec
Min xfer = 8388608.00 KB

Children see throughput for 2 random writers = 24004.56 KB/sec
Parent sees throughput for 2 random writers = 23958.08 KB/sec
Min throughput per process = 11979.05 KB/sec
Max throughput per process = 12025.51 KB/sec Avg throughput per process = 12002.28 KB/sec
Min xfer = 8388608.00 KB

iozone test complete.

Conclusion

You can see in the results a constant bandwidth of 77 MByte/s write and 105 MByte/s read performance which is IMHO quite good for a single HDD. I am not sure about the random read write performance. I need to digg a bit deeper into this :-)

P.S.
Maybe you want to know 'Why AFP?' First of all, I use Apple Computers at home and it showed me the best throughput compared to NFS or Samba

FreeNAS 0.7 - diskinfo of a SuperTalent UltraDrive ME MLC 64GB

2009-06-12T22:37:00.009+02:00

I've bought a SuperTalent UltraDrive ME MLC 64GB for playing around with SSD's. So I will provide here my experiences

For the FreeNAS 0.7 ZFS tests I've installed this SSD into a PC with a Intel Core2Quad Q6000 (2.4 GHz), 8 GB RAM. Mainboard Intel DG965WH.

My first test is a quick diskinfo -ct

Here we go...

freenas:~# diskinfo -ct /dev/ad6

/dev/ad6

512 # sectorsize

64023257088 # mediasize in bytes (60G)

125045424 # mediasize in sectors

124053 # Cylinders according to firmware.

16 # Heads according to firmware.

63 # Sectors according to firmware.

ad:P565045-BDIX-6029019 # Disk ident.

I/O command overhead:

time to read 10MB block 0.068941 sec = 0.003 msec/sector

time to read 20480 sectors 1.169508 sec = 0.057 msec/sector

calculated command overhead = 0.054 msec/sector

Seek times:

Full stroke: 250 iter in 0.024970 sec = 0.100 msec

Half stroke: 250 iter in 0.025027 sec = 0.100 msec

Quarter stroke: 500 iter in 0.049870 sec = 0.100 msec

Short forward: 400 iter in 0.039449 sec = 0.099 msec

Short backward: 400 iter in 0.039230 sec = 0.098 msec

Seq outer: 2048 iter in 0.114818 sec = 0.056 msec

Seq inner: 2048 iter in 0.123779 sec = 0.060 msec

Transfer rates:

outside: 102400 kbytes in 0.696010 sec = 147124 kbytes/sec

middle: 102400 kbytes in 0.618375 sec = 165595 kbytes/sec

inside: 102400 kbytes in 0.623304 sec = 164286 kbytes/sec

As comparison... here is the same command with a WD20EADS 2TB

quake:~# diskinfo -ct /dev/ad4

/dev/ad4

512 # sectorsize

2000398934016 # mediasize in bytes (1.8T)

3907029168 # mediasize in sectors

3876021 # Cylinders according to firmware.

16 # Heads according to firmware.

63 # Sectors according to firmware.

I/O command overhead:

time to read 10MB block 0.072554 sec = 0.004 msec/sector

time to read 20480 sectors 2.515795 sec = 0.123 msec/sector

calculated command overhead = 0.119 msec/sector

Seek times:

Full stroke: 250 iter in 6.150143 sec = 24.601 msec

Half stroke: 250 iter in 4.960021 sec = 19.840 msec

Quarter stroke: 500 iter in 8.214024 sec = 16.428 msec

Short forward: 400 iter in 2.523203 sec = 6.308 msec

Short backward: 400 iter in 2.008208 sec = 5.021 msec

Seq outer: 2048 iter in 0.298850 sec = 0.146 msec

Seq inner: 2048 iter in 0.379041 sec = 0.185 msec

Transfer rates:

outside: 102400 kbytes in 1.077248 sec = 95057 kbytes/sec

middle: 102400 kbytes in 1.378671 sec = 74274 kbytes/sec

inside: 102400 kbytes in 2.267264 sec = 45165 kbytes/sec

As you can see... the seek times of the SSD are impressive!

Goodbye MacBook Pro :-(

2009-05-28T23:45:00.002+02:00

I was on vacation last week. We stayed in France near Paris (Melun, La Rochette). On the campground La Belle Etoile.

In the night to Friday (22-May-2009) there was a robbery on the campcround.

Thieves stolen my backpack and my camera bag out of a VW-Bus while my friends stayed inside.

It is unbelievable but there was my beloved MacBook Pro, my Ipod, an external disk, two objectives for my camera and lots of other equipment in this bags :-( All of this is gone...

So if any of you find a MBP with the serial number W8809S9BYJK it belongs to me. Please contact me harryd71 at users dot sourceforge dot net

Seeding FreeNAS 0.7 4661...

2009-05-28T23:43:00.001+02:00

You will find the .torrent files (AMD64 & i386) here -> TBP

Please help me to seed!

Seeding FreeNAS 0.7 4621...

2009-05-12T19:33:00.002+02:00

You will find the .torrent files (AMD64 & i386) here -> TBP

Please help me to seed!

Seeding FreeNAS 0.7 4583...

2009-04-29T23:59:00.000+02:00