Friday, August 15, 2008

FreeNAS 0.7 and ZFS snapshots

Currently it is not possible to use the ZFS snapshot functionality via the Web-GUI. But you can use a script and cron to schedule snapshots.
You have to think about a schedule for the snapshots. I want to save my home-directories every hour (and have this snapshots available for 24 hours). Also I'd like to have a daily and a weekly snapshot (the daily snapshot will be available for 7 days the weeklys are stored for 4 weeks). All of the snapshots will be overwritten after that time.

First you need a place where you can store the script(s) that will create the snapshots.

I'm using a ZFS dataset called 'datapool/opt'. Under the directory 'bin' I'll have three scripts:


snapshot_hourly.sh
#!/bin/sh
zfs destroy $1@hourly.`date "+%H"` > /dev/null 2>&1
zfs snapshot $1@hourly.`date "+%H"`

snapshot_daily.sh
#!/bin/sh
zfs destroy $1@daily.`date "+%a"` > /dev/null 2>&1
zfs snapshot $1@daily.`date "+%a"`

snapshot_weekly.sh

#!/bin/sh
zfs destroy $1@weekly.4 > /dev/null 2>&1
zfs rename $1@weekly.3 @weekly.4 > /dev/null 2>&1
zfs rename $1@weekly.2 @weekly.3 > /dev/null 2>&1
zfs rename $1@weekly.1 @weekly.2 > /dev/null 2>&1
zfs snapshot $1@weekly.1

Be aware that these scripts should be executable!


freenas:/mnt/datapool/opt/bin# chmod 744 snapshot_*

Now you have to schedule the cron-jobs. You should do this via the Web-GUI. Go to -> System -> Advanced -> Cron

Command: /mnt/datapool/opt/bin/snapshot_hourly.sh datapool/home
User: root
Descritpion: Hourly snapshot of datapool/home
Schedule time:

minutes -> 0
hours -> All
days -> All
months -> All
week days -> All

Command: /mnt/datapool/opt/bin/snapshot_daily.sh datapool/home
User: root
Descritpion: Daily snapshot of datapool/home @ 20:00
Schedule time:
minutes -> 0
hours -> 20
days -> All
months -> All
week days -> All

Command: /mnt/datapool/opt/bin/snapshot_weekly.sh datapool/home
User: root
Descritpion: Weekly snapshot of datapool/home Sun @ 20:00
Schedule time:
minutes -> 0
hours -> 20
days -> All
months -> All
week days -> Sunday

After some time you will see somthing like this

freenas:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
...
datapool/home 14.9G 266G 21K /mnt/datapool/home
datapool/home@daily.Sat 0 - 21K -
datapool/home@weekly.2 0 - 21K -
datapool/home@daily.Fri 0 - 21K -
datapool/home@daily.Sun 0 - 21K -
datapool/home@weekly.1 0 - 21K -
datapool/home@daily.Mon 0 - 21K -
datapool/home@daily.Tue 0 - 21K -
datapool/home@daily.Wed 0 - 21K -
datapool/home@hourly.14 0 - 21K -
datapool/home@hourly.15 0 - 21K -
datapool/home@hourly.16 0 - 21K -
datapool/home@hourly.17 0 - 21K -
datapool/home@hourly.18 0 - 21K -
datapool/home@hourly.19 0 - 21K -
datapool/home@hourly.20 0 - 21K -
datapool/home@daily.Thu 0 - 21K -
datapool/home@hourly.21 0 - 21K -
datapool/home@hourly.22 0 - 21K -
datapool/home@hourly.23 0 - 21K -
datapool/home@hourly.00 0 - 21K -
datapool/home@hourly.01 0 - 21K -
datapool/home@hourly.02 0 - 21K -
datapool/home@hourly.03 0 - 21K -
datapool/home@hourly.04 0 - 21K -
datapool/home@hourly.05 0 - 21K -
datapool/home@hourly.06 0 - 21K -
datapool/home@hourly.07 0 - 21K -
datapool/home@hourly.08 0 - 21K -
datapool/home@hourly.09 0 - 21K -
datapool/home@hourly.10 0 - 21K -
datapool/home@hourly.11 0 - 21K -
datapool/home@hourly.12 0 - 21K -
datapool/home@hourly.13 0 - 21K -
...

How it works...

I've created a 10MB testfile

freenas:/mnt/datapool/home# dd if=/dev/zero of=testfile bs=1024k count=10
10+0 records in
10+0 records out
10485760 bytes transferred in 0.159408 secs (65779346 bytes/sec)
freenas:/mnt/datapool/home# ls -l
total 10237
-rw-r--r-- 1 root wheel 10485760 Aug 15 15:13 testfile
freenas:/mnt/datapool/home# df -h
Filesystem Size Used Avail Capacity Mounted on
...
datapool/home 5.3G 10M 5.3G 0% /mnt/datapool/home

Now there will be a snapshot created... (either via cron or the script -> freenas:/mnt/datapool/home# /mnt/datapool/opt/bin/snapshot_hourly.sh datapool/home)

freenas:/mnt/datapool/home# zfs list
NAME USED AVAIL REFER MOUNTPOINT
...
datapool/home 10.0M 5.34G 10.0M /mnt/datapool/home
datapool/home@hourly.15 0 - 10.0M -
...

If the file will be deleted...

freenas:/mnt/datapool/home# rm testfile
freenas:/mnt/datapool/home# zfs list
NAME USED AVAIL REFER MOUNTPOINT
...
datapool/home 10.0M 5.34G 26.9K /mnt/datapool/home
datapool/home@hourly.15 10.0M - 10.0M -

As you can see, the snapshot growed by 10MByte. But the file is gone...

freenas:/mnt/datapool/home# ls -al
total 4
drwxr-xr-x 2 root wheel 2 Aug 15 15:16 .
drwxrwxrwx 5 root wheel 5 Aug 15 13:58 ..

But how can I access the snapshot? Change to the .zfs/snapshot directory. And there you can see the snapshots.

freenas:/mnt/datapool/home# cd .zfs/snapshot
freenas:/mnt/datapool/home/.zfs/snapshot# ls
...
hourly.15
...

freenas:/mnt/datapool/home/.zfs/snapshot# cd hourly.15
freenas:/mnt/datapool/home/.zfs/snapshot/hourly.15# ls -l
total 10237
-rw-r--r-- 1 root wheel 10485760 Aug 15 15:13 testfile

Be aware! Snapshots are Read-Only. So you can copy this file back to the origin, or wherever you want.

9 comments:

Gary Sims (aka The Hungry Penguin) said...

Harryd,

This is a great post. I would like to reproduce it as an article on my LearnFreeNAS.com web site. Please contact me to talk about this.

Again, great post!

Gary (gary dot sims [at] gmail dot com)

tr said...

Thanks for this post. I'm curious about something though... What's the point of ever destroying a snapshot? They don't take any extra room.... Do they?!

harryd said...

A snapshot will consume space if you delete or change a file.
Look for the "Solaris ZFS Administration Guide", Chapter "Snapshot Space Accounting" -> http://docs.sun.com

Thomas said...

This script dosent work. I have made all the things based on yout tutorial - but it keeps saying "snapshot_day.sh: Command not found."

What to do now?

with regards
Thomas

harryd said...

Hi thomas,

sorry for the delay... First of all, the name of the script is 'snapshot_daily.sh'.

Do you've changed the permissions of the script? ( chmod 744 snapshot_* )

And you must be root to execute the script...

Can you please explain in more detail your problem?

Regards,

harryd

Ultravox said...

Thanks for the tips and for posting a comment on my blog pointing me to it.

I'll give it a try!

Stefan said...

Thanks again, harry. Just tried snapshots and (after some fiddling) they work as shown.

Problem was: the cron jobs couldn't be executed.

I have changed two little things:
1.) First line of the shell scripts to:
#!/bin/bash

2.) The cron jobs command to:
bash /root/snapshot_hourly.sh zroot/dataset

Please don't ask why - I'm not that used to shell scripting. But now it works fine.

Maybe that is (also) because I use the "full OS" installation? If so, maybe it points to an explanation why my pool is running after reboot and your pool disappeares??

Manuel Moeller said...

Hi harryd,

thanks a lot for the post. I was always dreaming of such a setup at home!

However, at least on my machine I found that you shell script source code contains a small mistake. The second argument of "zfs rename" also needs the name of the dataset. Thus, the script has to look like this:

#!/bin/sh
zfs destroy $1@weekly.4 > /dev/null 2>/dev/null
zfs rename $1@weekly.3 $1@weekly.4 > /dev/null 2>/dev/null
zfs rename $1@weekly.2 $1@weekly.3 > /dev/null 2>/dev/null
zfs rename $1@weekly.1 $1@weekly.2 > /dev/null 2>/dev/null
zfs snapshot $1@weekly.1

Best regards and thank you again for the great description on how to set this up!

Manuel

harryd said...

Manuel,

thanks for your comment! I don't know why, but it works for me as I've posted it here. Can you please do me a favor and post here the error-message or send it to harryd71 [at] users dot sourceforge dot net. Thx!