As you can read here How to resize ZFS, it is possible to resize a zpool if you replace each disk of the pool with a bigger one. In my first post I used a virtual machine to test this. And... I've replaced one disk (eg. da0) with a different device (eg. da4).
In the real world I would like to replace one device with the same (but bigger one). I was able to get some very old SCSI Disks in an external SCSI enclosure. These disks are perfect for testing this...
Here we go...
My testsystem is a Sempron 2600+ with 1 GB RAM and several SCSI Disks connected via a DAWICONTROL DC-2976 UW controller. The FreeNAS Version I've used for this test is 0.7 Sardaukar (revision 4390)
I've created a testpool with three 2.1 GB SCSI-Disks (yes, it's true, just 2.1 GB ;-) ). And I've also created two filesystems (testfs1, compressed & testfs2), and some testfiles.
crusher:~# zpool list testpool0
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
testpool0 5.91G 2.81G 3.09G 47% ONLINE -
As you can see, this pool has a size of 5.91 GB (remember this)
crusher:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
bigpool 100M 364G 25.3K /mnt/bigpool
bigpool/temp 100M 364G 100M /mnt/bigpool/temp
testpool0 1.87G 2.00G 28.0K /mnt/testpool0
testpool0/testfs1 640M 2.00G 640M /mnt/testpool0/testfs1
testpool0/testfs2 1.25G 2.00G 1.25G /mnt/testpool0/testfs2
crusher:~# zpool status testpool0
pool: testpool0
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
testpool0 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
First, I'd like to start with a scrub to be sure that the data is safe on these disks...
crusher:~# zpool scrub testpool
This will take a while. Check the status with:
crusher:~# zpool status testpool0
pool: testpool0
state: ONLINE
scrub: scrub in progress, 93.34% done, 0h0m to go
config:
NAME STATE READ WRITE CKSUM
testpool0 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
crusher:~# zpool status testpool0
pool: testpool0
state: ONLINE
scrub: scrub completed with 0 errors on Fri Feb 27 21:40:16 2009
config:
NAME STATE READ WRITE CKSUM
testpool0 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
Everything looks perfect... so lets start with the replace. This SCSI enclosure supports Hot-Swapping of the disks (more or less).
Ok, lets replace the first disk (da0)
Offline the first disk
crusher:~# zpool offline testpool0 da0
Bringing device da0 offline
crusher:~# zpool status testpool0
pool: testpool0
state: DEGRADED
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scrub: scrub completed with 0 errors on Fri Feb 27 21:40:16 2009
config:
NAME STATE READ WRITE CKSUM
testpool0 DEGRADED 0 0 0
raidz1 DEGRADED 0 0 0
da0 OFFLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
Replace the first disk (da0)
First physicaly ;-)
And then do the replace command
crusher:~# zpool replace testpool0 da0
crusher:~# zpool status testpool0
pool: testpool0
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 14.79% done, 0h2m to go
config:
NAME STATE READ WRITE CKSUM
testpool0 DEGRADED 0 0 0
raidz1 DEGRADED 0 0 0
replacing DEGRADED 0 0 0
da0/old OFFLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
After a while, the resivering has finished
crusher:~# zpool status testpool
pool: testpool
state: ONLINE
scrub: resilver completed with 0 errors on Fri Feb 27 20:05:52 2009
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
Lets replace the second and third disk...
crusher:~# zpool status testpool0
pool: testpool0
state: ONLINE
scrub: scrub completed with 0 errors on Fri Feb 27 22:57:19 2009
config:
NAME STATE READ WRITE CKSUM
testpool0 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
crusher:~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
bigpool 556G 150M 556G 0% ONLINE -
testpool0 5.91G 2.82G 3.09G 47% ONLINE -
crusher:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
bigpool 100M 364G 25.3K /mnt/bigpool
bigpool/temp 100M 364G 100M /mnt/bigpool/temp
testpool0 1.87G 2.00G 28.0K /mnt/testpool0
testpool0/testfs1 640M 2.00G 640M /mnt/testpool0/testfs1
testpool0/testfs2 1.25G 2.00G 1.25G /mnt/testpool0/testfs2
As you can see the size of the testpool hasn't changed. After exporting and importing the pool, the additional space is available!
crusher:~# zpool export testpool0
crusher:~# zpool import testpool0
crusher:~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
bigpool 556G 150M 556G 0% ONLINE -
testpool0 12.0G 2.82G 9.15G 23% ONLINE -
The size has been changed to 12.0 GB...
