This is an old revision of the document!


Expanding a ZFS RAID System

This page documents my experiences in converting a 3-way RAIDZ1 pool into a 6-way RAIDZ2 pool (adding 3 new disks to the 3 original disk), with minimal downtime. The system boots off a UFS gmirror and this remains unchanged (though I have allocated equivalent space on the new disks to allow potential future conversion to ZFS).

Original Configuration

server# gpart show
=>        34  1953522988  ada0  GPT  (932G)
          34          94     1  freebsd-boot  (47K)
         128     6291456     2  freebsd-ufs  (3.0G)
     6291584     6291456     3  freebsd-swap  (3.0G)
    12583040     4194304     4  freebsd-ufs  (2.0G)
    16777344  1936745678     5  freebsd-zfs  (924G)

=>        34  1953522988  ada1  GPT  (932G)
          34          94     1  freebsd-boot  (47K)
         128     6291456     2  freebsd-ufs  (3.0G)
     6291584     6291456     3  freebsd-swap  (3.0G)
    12583040     4194304     4  freebsd-ufs  (2.0G)
    16777344  1936745678     5  freebsd-zfs  (924G)

=>        34  1953525101  ada2  GPT  (932G)
          34          94     1  freebsd-boot  (47K)
         128     6291456     2  freebsd-ufs  (3.0G)
     6291584     6291456     3  freebsd-swap  (3.0G)
    12583040     4194304     4  freebsd-ufs  (2.0G)
    16777344  1936747791     5  freebsd-zfs  (924G)

server# zpool status -v
  pool: tank
 state: ONLINE
status: The pool is formatted using an older on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on older software versions.
 scrub: scrub completed after 14h22m with 0 errors on Thu Oct 28 18:22:28 2010
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            ada0p5  ONLINE       0     0     0
            ada1p5  ONLINE       0     0     0
            ada2p5  ONLINE       0     0     0

errors: No known data errors

Final Configuration

server% gpart show
=>        34  1953525101  da0  GPT  (932G)
          34          94    1  freebsd-boot  (47K)
         128    10485760    2  freebsd-zfs  (5.0G)
    10485888     6291456    3  freebsd-swap  (3.0G)
    16777344  1936747791    5  freebsd-zfs  (924G)

=>        34  1953525101  da1  GPT  (932G)
          34          94    1  freebsd-boot  (47K)
         128    10485760    2  freebsd-zfs  (5.0G)
    10485888     6291456    3  freebsd-swap  (3.0G)
    16777344  1936747791    5  freebsd-zfs  (924G)

=>        34  1953522988  ada0  GPT  (932G)
          34          94     1  freebsd-boot  (47K)
         128     6291456     2  freebsd-ufs  (3.0G)
     6291584     6291456     3  freebsd-swap  (3.0G)
    12583040     4194304     4  freebsd-ufs  (2.0G)
    16777344  1936745678     5  freebsd-zfs  (924G)

=>        34  1953522988  ada1  GPT  (932G)
          34          94     1  freebsd-boot  (47K)
         128     6291456     2  freebsd-ufs  (3.0G)
     6291584     6291456     3  freebsd-swap  (3.0G)
    12583040     4194304     4  freebsd-ufs  (2.0G)
    16777344  1936745678     5  freebsd-zfs  (924G)

=>        34  1953525101  ada2  GPT  (932G)
          34          94     1  freebsd-boot  (47K)
         128     6291456     2  freebsd-ufs  (3.0G)
     6291584     6291456     3  freebsd-swap  (3.0G)
    12583040     4194304     4  freebsd-ufs  (2.0G)
    16777344  1936747791     5  freebsd-zfs  (924G)

=>        34  1953525101  ada3  GPT  (932G)
          34          94     1  freebsd-boot  (47K)
         128    10485760     2  freebsd-zfs  (5.0G)
    10485888     6291456     3  freebsd-swap  (3.0G)
    16777344  1936747791     5  freebsd-zfs  (924G)

server% zpool status
  pool: tank
 state: ONLINE
 scrub: scrub completed after 3h54m with 0 errors on Sat Nov  6 16:35:22 2010
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz2    ONLINE       0     0     0
            ada0p5  ONLINE       0     0     0
            ada1p5  ONLINE       0     0     0
            ada2p5  ONLINE       0     0     0
            ada3p5  ONLINE       0     0     0
            da0p5   ONLINE       0     0     0
            da1p5   ONLINE       0     0     0

errors: No known data errors

Procedure

Partition up new disks

In my case, I have root and swap on the same disks, so I needed to carve out space for that. Even if you use the disks solely for ZFS, it's probably a good idea to partition a couple of MB off the disks in case a replacement disk is slight smaller.

zfsraid.1289421029.txt.gz · Last modified: 2010/11/10 20:30 by peterjeremy
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki