Monday, May 9, 2011

ZFS Pool Versions

I was looking for an overview of the available ZFS Pool Versions. The Opensolaris XWiki Page of the ZFS Group show on page for every page, but not an complete overview.


7.0+ - original ZFS import, ZFS v6; requires significant tuning for stable operation (no longer supported)
7.2 - still ZFS v6, improved memory handling, amd64 may need no memory tuning (no longer supported)
7.3+ - backport of new ZFS v13 code, similar to the 8.0 code
8.0 - new ZFS v13 code, lots of bug fixes - recommended over all past versions. (no longer supported)
8.1+ - ZFS v14
8.2+ - ZFS v15
9.0+ - ZFS v28


Here I list of all current Versions until today 9-May-2011



This version includes support for improved 'zfs list' performance.

Pool version 31 is available in this release:

Nevada, build 150
The change record for the version 31 change is:

6980669 zfs rename -r is slow, causes appliance UI to hang



This version includes support for ZFS encryption.

Pool version 30 is available in this release:

Nevada, build 149
The change record for the version 30 change is:

4854202 ZFS data set encryption



This version includes support for the RAID-Z/mirror hybrid allocator.

Pool version 29 is available in this release:

Nevada, build 148
The change record for the version 29 change is:

6977913 RAID-Z/mirror hybrid allocator



This version includes support for multiple virtual device replacements.

Pool version 28 is available in this release:

Nevada, build 147
The change record for the version 28 change is:

6782540 zpool cannot replace a replacing device



This version includes support for improved snapshot creation performance.

Pool version 27 is available in this release:

Nevada, build 145
The change record for the version 27 change is:

6844896 recursive snapshots take a long time



This version includes support for improved snapshot deletion performance.

Pool version 26 is available in this release:

Nevada, build 141
The change record for the version 26 change is:

6948890 snapshot deletion can induce pathologically long spa_sync() times



This version includes support for improved pool scrubbing and resilvering statistics.

Pool version 25 is available in this release:

Nevada, build 140
The change record for the version 25 change is:

6391915 provide interval arg to zpool status to monitor resilvering



This version includes support for system attributes.

Pool version 24 is available in this release:

Nevada, build 137
The change records for the version 24 change are:

6716117 ZFS needs native system attribute infrastructure
6516171 zpl symlinks should have their own object type



This version includes support for the slim ZIL.

Pool version 23 is available in this release:

Nevada, build 135
The change record for the version 23 change is:

6595532 ZIL is too talkative



This version includes support for zfs receive properties.

Pool version 22 is available in this release:

Nevada, build 128
The PSARC case for the version 22 change is:

PSARC/2009/510 ZFS Received Properties



This version includes support for ZFS deduplication properties.

Pool version 21 is available in this release:

Nevada, build 128
The PSARC case for the version 21 change is:

PSARC/2009/571 ZFS Deduplication Properties



This version includes the zle compression algorithm that is needed to support the ZFS deduplication properties in ZFS pool version 21. Both pool versions are available in this release:

Nevada, build 128
The PSARC case for the version 20 change is:

PSARC/2009/571 ZFS Deduplication Properties



This version includes support for the following feature:

ZFS log device removal
This feature is available in:

Nevada, build 125
The related change record for the version 19 change is:

6574286 removing a slog doesn't work



This version includes support for the following feature:

ZFS snapshot holds
This feature is available in:

Nevada, build 121
The related change record for the version 18 change is:

6803121 want user-settable refcounts on snapshots



This version includes support for the following feature:

triple-parity RAID-Z
This feature is available in:

Nevada, build 120
The related change record for the version 17 change is:

6854612 triple-parity RAID-Z



This version includes support for the following feature:

STMF property support
This feature is available in:

Nevada, build 116
The related bug for the version 16 change is:

6736004 ZFS volumes need an additional property for COMSTAR support



This version includes support for the following features:

userused@... and groupused@... properties
userquota@... and groupquota@... properties
These features are available in:

Nevada, build 114
The related bug and PSARC case for version 15 changes are:

6501037 want user/group quotas on ZFS
PSARC/2009/204 ZFS user/group quotas and space accounting



This version includes support for the following feature:

passthrough-x aclinherit property support
This feature is available in:

OpenSolaris 2009.06, Nevada, build 103
The related bug and PSARC case for the version 14 change are:

6765166 Need to provide mechanism to optionally inherit ACE_EXECUTE
PSARC 2008/659 New ZFS "passthrough-x" ACL inheritance rules



This version includes support for the following features:

usedbysnapshots property
usedbychildren property
usedbyrefreservation property
usedbydataset property
These features are available in:

OpenSolaris 2008.11, Nevada, build 98
The related bug and PSARC case for version 13 change are:

6730799 want user properties on snapshots
PSARC/2008/518 ZFS space accounting enhancements



This version includes support for the following feature:

Snapshot properties
This feature is available in:

OpenSolaris 2008.11, Nevada build 96
The related bug for the version 12 change is:

6701797 want user properties on snapshot



This version includes support for the following feature:

Improved zpool scrub / resilver performance
This feature is available in:

OpenSolaris 2008.11, Nevada, build 94
The related bug for the version 11 change is:

6343667 scrub/resilver has to start over when a snapshot is taken
Note: this bug is fixed when using build 94 even with older pool versions. However, upgrading the pool can improve scrub performance when there are many file systems, snapshots, and clones.



This version includes support for the following feature:

Devices can be added to a storage pool as "cache devices." These devices provide an additional layer of caching between main memory and disk. Using cache devices provides the greatest performance improvement for random read-workloads of mostly static content.
This feature is available in Nevada, build 78.

The Solaris 10 10/08 release includes ZFS pool version 10, but support for cache devices is not included in this Solaris release.

The related bug for the version 10 change is:

6536054 second tier ("external") ARC



This version includes support for the following features:

In addition to the existing ZFS quota and reservation features, this release includes dataset quotas and reservations that do not include descendent datasets, such as snapshots and clones, in the space consumption. ("zfs set refquota" and "zfs set refreservation".)
A reservation is automatically set when a non-sparse ZFS volume is created that matches the size of the volume. This release provides an immediate reservation feature so that you set a reservation on a non-sparse volume with enough space to take snapshots and modify the contents of the volume.
CIFS server support
These features are available in Nevada, build 77.

The related bugs for version 9 changes are:

6431277 want filesystem-only quotas
6483677 need immediate reservation
6617183 CIFS Service - PSARC 2006/715



This version now supports the ability to delegate zfs(1M) administrative tasks to ordinary users.

This feature is available in:

Nevada, build 69
Solaris 10 10/08 release
The related bug for the version 8 change is:

6349470 investigate non-root restore/backup



This version includes support for the following feature:

The ZFS Intent Log (ZIL) satisfies the need of some applications to know the data they changed is on stable storage on return from a system call. The Intent Log holds records of those system calls and they are replayed if the system power fails or panics if they have not been committed to the main pool. When the Intent Log is allocated from the main pool, it allocates blocks that chain through the pool. This version adds the capability to specify a separate Intent Log device or devices.

This feature is available in:

Nevada, build 68
Solaris 10 10/08 release
The related bug for the version 7 change is:

6339640 Make ZIL use NVRAM when available



This version includes support for the following feature:

'bootfs' pool property
This feature is available in:

Nevada, build 62
Solaris 10 10/08 release
The related bugs for version 6 changes are as follows:

4929890 ZFS boot support for the x86 platform
6479807 pools need properties



This version includes support for the following feature:

gzip compression for ZFS datasets
This feature is available in:

Nevada, build 62
Solaris 10 10/08 release
The related bug for the version 5 changes is:

6536606 gzip compression for ZFS



This version includes support for the following feature:

zpool history
This feature is available in:

Nevada, build 62
Solaris 10 8/07 release
The related bugs for version 4 changes are as follows:

6529406 zpool history needs to bump the on-disk version
6343741 want to store a command history on disk



This version includes support for the following features:

Hot spares
Double-parity RAID-Z (raidz2)
Improved RAID-Z accounting
These features are available in:

Nevada, build 42
Solaris 10 11/06 release, (build 3)
The related bugs for version 3 changes are as follows:

6405966 Hot Spare support in ZFS
6417978 double parity RAID-Z a.k.a. RAID6
6288488 du reports misleading size on RAID-Z



This version includes support for "Ditto Blocks", or replicated metadata. Due to the tree-like structure of the ZFS on-disk format, an uncorrectable error in a leaf block may be relatively benign, while an uncorrectable error in pool metadata can result in an unopenable pool. This feature introduces automatic replication of metadata (up to 3 copies of each block) independent of any underlying pool-wide redundancy. For example, on a pool with a single mirror, the most critical metadata will appear three times on each side of the mirror, for a total of six copies. This ensures that while user data may be lost due to corruption, all data in the pool will be discoverable and the pool will still be usable. This will be expanded in the future to allow user data replication on a per-dataset basis.

This feature integrated on 4/10/06 with the following bug fix:

6410698 ZFS metadata needs to be more highly replicated (ditto blocks)

This feature is available in:

Nevada, build 38
Solaris 10 10/06 release (build 09)



This is the initial ZFS on-disk format as integrated on 10/31/05. During the next six months of internal use, there were a few on-disk format changes that did not result in a version number change, but resulted in a flag day since earlier versions could not read the newer changes. The first official releases supporting this version are:

Nevada, build 36
Solaris 10 6/06 release
Earlier releases may not support this version, despite being formatted with the same on-disk number. This is due to:

6389368 fat zap should use 16k blocks (with backwards compatibility)
6390677 version number checking makes upgrades challenging

Saturday, May 7, 2011

Where is harryd? -- follow-up

Nearly one year ago I've written that we build our own house. Now after a lot work is behind us, we are in and I have more time than I had in the past year. Currently I am preparing a blogpost. If you think that might has something to do with FreeNAS 8 and a benchmark, than you might be right. So... stay tuned...

Monday, June 7, 2010

A Closer Look at ZFS, Vdevs and Performance

Constantin gives here an great introduction into ZFS, vdevs and performance.

Monday, May 31, 2010

FreeNAS vs Nexentastor

I've tested the performance of FreeNAS 0.7.1 stable (Build 5127) and Nexentastor Community Edition 3.0.2. I will not dig into the different features or the details of both WebGUIs.

For the tests I've used the following hardware

Server
CPU - Intel Core2Duo E6300 - 1.8 GHz - 2 MB Cache
RAM - 2 GByte
Mainboard - Intel DG965WH
Harddisks - 3x WesternDigital WD3200 (320 GByte connected to the onboard SATA2)

Client
My good old MacBook Pro (late 2008), 4 GByte RAM, Mac OS X 10.6.3

Software
iSCSI initiator - Global SAN iSCSI initiator (4.0.0.204)
Benchmark Software - XBench
Benchmark Software - Helios LAN Test
Benchmark Software - iozone (MacPorts)

Protocols
CIFS
NFS
AFP (currently only available for FreeNAS)
iSCSI

Benchmarks
Xbench - The best out of 5 results

Helios LAN Test - 10 tests, two times, best result
iozone - options -e -i0 -i1 -i2 -+n -r 256k -s2g -t4 -c -x

Tuning
No tuning on the Nexentastor. For FreeNAS I've enabled in the WebGUI System | Advanced | Tuning (enable tuning of some kernel variables)

CIFS
Keep in mind, I've tested CIFS with the built-in smb client of OSX. A native Windows OS might provide different results.

Tuning for FreeNAS (via WebGUI)
Services | CIFS/SMB | Settings
-> Send and Receive Buffer Size to 65535
-> Enable large read/write
-> Auxiliary parameters : max xmit = 65535
-> Auxiliary parameters : socket options = TCP_NODELAY SO_SNDBUF=65535 SO_RCVBUF=65535


















If you compare the CIFS results you will see that NexentaStor is slightly faster than FreeNAS

NFS
I wasn't able to get XBench to run on a FreeNAS NFS mountpoint. So you wont see any results on the XBench charts.




With the missing XBench results for FreeNAS I would say NexentaStor is here the winner. It provides slightly higher throughput rates.

AFP
NexentaStor doesn't provide this protocol. As I am a Mac user it is very useful to have a system serving data via AFP. Especially the support of FreeNAS for time machine is IMHO excellent (see my blogposts about backup and restore using FreeNAS and time machine)




FreeNAS is in this disciplin the clear winner.

iSCSI
There was also an issue to get iozone to run on a FreeNAS iSCSI target.





The iSCSI protocol shows several advantages compared to the other. It shows high throughput rates with a low latency. But it is not usable to share the LUN to several systems at the same time (it is possible with special Filesystems, I know, but I think this is not very common). With the lack of the iozone results for FreeNAS I would also say, Nexentastor is here the winner.

Overall comparisson











Conclusion
FreeNAS and Nexenta is on the performance side very much comparable. NexentaStor is slitghly faster in most of the disciplines, but the lack of AFP make them both overall equal.
FreeNAS is easy to setup and has a very clear and structured WebGUI. Nexenta has more features (e.g. built in snapshots, deduplication, etc.)
At the moment I definitely will stay with FreeNAS as the performance is comparable and I really like the integration of Time Machine via AFP.

P.S.: If you are interested in a detailed Excel-Sheet with more details, please drop me a mail harryd71 at users dot sourceforge dot net

Sunday, May 30, 2010

Flattr

I've stumbled upon flattr :-)
Flattr is a social micropayment platform that lets you show love for the things you like.
As I've always looked for a service like this to show other people my respect and want to give them something back, I think this might be the right platform to do this. I've added a button to each of my blogposts and if you like what you read, you can press this button.
Of course, you need to be a user of this service, so please register for a beta invite or ask me at harryd71 at users dot sourceforge dot net or send me a direct message on twitter.

Many thanks for your support!

P.S.: Nico showed on this blogpost how to add a flattr button to blogger. Many thanks, Nico (of course, I've flattr'd you :-) )

Friday, May 28, 2010

Where is harryd?

Maybe you've recognized that I am not very active here or at the FreeNAS forum.

The reason behind this... I build together with my wife, our own house. She is the architect and there is a lot of work to do. To be honest... If you see our piece of land, you won't see too much, as the craftsmen just begun. But there is a lot of work to do behind the scenes.

The next few weeks / months will be very exciting, as we can see how our dream is built in stone and wood (and all the other materials). There is a lot of work to do for me, as I will do the whole electricity (and a few other things).

I've prepared another blogpost and I hope I'm able to finish it soon. So, please stay tuned and thanks for all your feedback!

Wednesday, May 26, 2010

Mac OS X system restore using time machine and FreeNAS

I'd like to show you in this post how you can recover your Mac from a time machine backup using FreeNAS. If you've set up time machine as I've described here, you should be able to do a complete system restore.

I've used a virtual machine using VirtualBox 3.2.0 on my MacMini to demonstrate this. VirtualBox supports now to install Mac OS X guests (maybe not for a very long time, AFAIK this is not allowed by Apple.)

1. If you have an issue with your system and you want to do a complete restore (or disaster recovery), please boot from the original boot DVD


2. Choose 'Restore System From Backup...' the 'Utilities' menu


3. Click 'Continue'. The next screen will show you the available sources.


4. After entering your login credentials you should see the backup source for your system


5. The next screen shows you the available backups of your system. Select the one you've want to restore


6. Select the destination disk and click 'restore' read the warning and 'continue'


7. And finally restart the system