The answer is a littlebit complicated...
If you want to change the 'geometry' of the ZPOOL (for example: change from a mirrored pool to a raidz, or simply add a disk to a raidz, or change from raidz to raidz2) then the answer is no.
But it is possible to change the disks of a pool with bigger ones and use the space.
Here is what I've tested with a FreeNAS 0.7 (rev 3514) installed as a Virtual Machine.
I've used four 1 GByte HDs and four 2 GByte HDs. My mission was to get a raidz from 4 GByte (usable 3 GByte) to around 8 GByte (usable 6 GByte). The initial setup was one raidz with the four 1 GByte-HDs called 'datapool'. The disks da0, da1, da2, da3 are the 1 GByte-Drives. The disks da4, da5, da6, da7 are the 2 GByte-Drives.
Replace the first disk:
freenas:~# zpool replace datapool da0 da4
Check the status:
freenas:~# zpool status -v
pool: datapool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 16.44% done, 0h0m to go
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
replacing ONLINE 0 0 0
da0 ONLINE 0 0 0
da4 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
errors: No known data errors
It will take a while until the pool is completly 'resilvered'
freenas:~# zpool status -v
pool: datapool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Aug 12 16:03:34 2008
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da4 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
errors: No known data errors
Proceed with the next disk...
freenas:~# zpool replace datapool da1 da5
freenas:~# zpool status -v
pool: datapool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 7.86% done, 0h0m to go
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da4 ONLINE 0 0 0
replacing ONLINE 0 0 0
da1 ONLINE 0 0 0
da5 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
errors: No known data errors
freenas:~# zpool status -v
pool: datapool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Aug 12 16:05:34 2008
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
errors: No known data errors
Next one...
freenas:~# zpool replace datapool da2 da6
freenas:~# zpool status -v
pool: datapool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 6.01% done, 0h0m to go
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
replacing ONLINE 0 0 0
da2 ONLINE 0 0 0
da6 ONLINE 0 0 0
da3 ONLINE 0 0 0
errors: No known data errors
You can also monitor the status via this command:
freenas:~# zpool iostat -v 5
capacity operations bandwidth
pool used avail read write read write
------------- ----- ----- ----- ----- ----- -----
datapool 715M 3.27G 16 13 1.94M 912K
raidz1 715M 3.27G 16 13 1.94M 912K
da4 - - 11 11 968K 752K
da5 - - 5 20 440K 1.40M
replacing - - 0 45 0 1.56M
da2 - - 7 7 597K 399K
da6 - - 0 23 3.79K 1.60M
da3 - - 8 6 688K 313K
------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool used avail read write read write
------------- ----- ----- ----- ----- ----- -----
datapool 715M 3.27G 62 5 7.82M 19.6K
raidz1 715M 3.27G 62 5 7.82M 19.6K
da4 - - 10 0 893K 6.13K
da5 - - 23 1 1.87M 28.3K
replacing - - 0 67 0 2.62M
da2 - - 0 17 0 1.40M
da6 - - 0 33 0 2.64M
da3 - - 16 1 1.29M 26.2K
------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool used avail read write read write
------------- ----- ----- ----- ----- ----- -----
datapool 715M 3.27G 31 0 3.82M 2.29K
raidz1 715M 3.27G 31 0 3.82M 2.29K
da4 - - 38 1 3.09M 44.2K
da5 - - 26 1 2.09M 22.9K
replacing - - 0 31 0 1.28M
da2 - - 0 24 0 1.87M
da6 - - 0 17 0 1.30M
da3 - - 32 1 2.68M 22.9K
------------- ----- ----- ----- ----- ----- -----
freenas:~# zpool status -v
pool: datapool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Aug 12 16:07:31 2008
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
da6 ONLINE 0 0 0
da3 ONLINE 0 0 0
errors: No known data errors
And this is the last one...
freenas:~# zpool replace datapool da3 da7
freenas:~# zpool status -v
pool: datapool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress, 3.02% done, 0h0m to go
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
da6 ONLINE 0 0 0
replacing ONLINE 0 0 0
da3 ONLINE 0 0 0
da7 ONLINE 0 0 0
errors: No known data errors
freenas:~# zpool iostat -v 5
capacity operations bandwidth
pool used avail read write read write
------------- ----- ----- ----- ----- ----- -----
datapool 715M 3.27G 14 0 1.75M 0
raidz1 715M 3.27G 14 0 1.75M 0
da4 - - 13 0 1.12M 0
da5 - - 14 0 1.19M 510
da6 - - 13 0 1.13M 0
replacing - - 0 14 0 599K
da3 - - 0 19 0 1.58M
da7 - - 0 7 0 599K
------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool used avail read write read write
------------- ----- ----- ----- ----- ----- -----
datapool 715M 3.27G 68 0 8.51M 4.49K
raidz1 715M 3.27G 68 0 8.51M 4.49K
da4 - - 15 0 1.26M 1.70K
da5 - - 6 0 546K 1.20K
da6 - - 18 0 1.51M 1.40K
replacing - - 0 68 0 2.84M
da3 - - 0 21 0 1.75M
da7 - - 0 35 0 2.84M
------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool used avail read write read write
---------- ----- ----- ----- ----- ----- -----
datapool 715M 3.27G 21 11 2.62M 696K
raidz1 715M 3.27G 21 11 2.62M 696K
da4 - - 15 7 1.22M 430K
da5 - - 13 9 1.11M 583K
da6 - - 9 12 825K 834K
da7 - - 0 21 1007 1.59M
---------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool used avail read write read write
---------- ----- ----- ----- ----- ----- -----
datapool 715M 3.27G 0 0 0 0
raidz1 715M 3.27G 0 0 0 0
da4 - - 0 0 0 0
da5 - - 0 0 0 0
da6 - - 0 0 0 0
da7 - - 0 0 0 0
---------- ----- ----- ----- ----- ----- -----
freenas:~# zpool status -v
pool: datapool
state: ONLINE
scrub: resilver completed with 0 errors on Tue Aug 12 16:09:45 2008
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
errors: No known data errors
Finally it is necessary to reboot
freenas:~# reboot
And here you can see the result. It is possible to resize a ZPOOL if you have bigger disks...
freenas:~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
datapool 7.97G 715M 7.27G 8% ONLINE -
6 comments:
Yes, fine, resize worked flawless. But I wonder 2 things:
1) how to correct the vdev definition after the resize is finished?
After a resize also the pool/commands regarding disks don't work because they are not updated afterwards (original pool had da1+da2, resized pool has da3+da4, but commands still bleat "cannot offline da1: no such device in pool").
I assume the vdev definitions and devices that build a pool can only be corrected by hand in the XML file!??
2) is there a way to get the resized pools capacity updated without a reboot? Have no idea so far.
Is there a smart way to monitor actual ram usage of freenas (I want it all second or so to be logged into a file... or added to the status/graph ;)?
I'd like to stress the memory usage that is told to be "at least" 512MB, but me did never have stress with 256MB. I want some more clarity about ram usage when and with how many and what size of pools of what kind and therefore testing that but dont know how to log the ram usage.
--
Stefan
Thanks for your comment!
My intention about this post was more to show and test the capability to grow a zpool. To be honest, I didn't digged into the XML file to correct the vdev definition.
I was not able to get the new size of resized pool online without a reboot.
About your other questions... Maybe you are able to open a thread in the FreeNAS forum.
You don't have to wait for each resilver to complete, there is no problem to issue all replace commands at once.
Of course, I can't guarantee you'd have some issues if there's a power failure or some other unforeseeable problem while doing the replace, but I assume ZFS is capable to deal with this and still use the old disks as long as the replace is not finished.
I've posted a new post regarding resizing ZFS -> How to resize ZFS - Part 2 (the real world)
Great articles! I linked to them from Gestalt IT (http://gestaltit.com) with some commentary.
Your next assignment is to figure out how to shrink a zpool! Hahaha!
@Stephen: Thanks :-D
I don't think about shrinking my zpools. My data is getting more and more... (lots of pics and videos from my HDV-Camera)
Post a Comment