Wednesday, February 01, 2006

ZFS - recovering destroyed pools

If you created zfs pool with filesystems on it - it could happen that you destroyed that pool not really wanting it. Fortunatelly when you destroy a pool all that ZFS does is it marks that pool as 'destroyed' but no data or config on disks are erased. Making some really simple changes to ZFS utils (and to ZFS fs itself) you can easly recover previously destroyed pools with all filesystems and data on them.
When you do 'zfs import' it lists you available pools in a system skipping destroyed pools. After applying below changes you will also see destroyed pools which are marked '(DESTROYED)' in a state property. Using 'zpool import -f ' you can import again such pool.


bash-3.00# zpool create backup c2t0d0p0
bash-3.00# zfs create backup/d1
bash-3.00# zfs create backup/d2
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
backup 376K 37.0G 100K /backup
backup/d1 98.5K 37.0G 98.5K /backup/d1
backup/d2 98.5K 37.0G 98.5K /backup/d2
bash-3.00#

bash-3.00# df -h -F zfs
Filesystem size used avail capacity Mounted on
backup 37G 99K 37G 1% /backup
backup/d1 37G 98K 37G 1% /backup/d1
backup/d2 37G 98K 37G 1% /backup/d2
bash-3.00#

bash-3.00# cp -rp /usr/kernel/ /backup/d1/
bash-3.00# cp -rp /usr/platform/ /backup/d2/
bash-3.00#
bash-3.00# df -h -F zfs
Filesystem size used avail capacity Mounted on
backup 37G 99K 37G 1% /backup
backup/d1 37G 3.4M 37G 1% /backup/d1
backup/d2 37G 849K 37G 1% /backup/d2
bash-3.00#


bash-3.00# zpool destroy backup
bash-3.00# zpool list
no pools available
bash-3.00#

bash-3.00# zpool import
pool: backup
id: 6753094033596765985
state: ONLINE (DESTROYED)
action: The pool can be imported using its name or numeric identifier. The
pool was destroyed, but can be imported using the '-f' flag.
config:

backup ONLINE
c2t0d0p0 ONLINE
bash-3.00#

bash-3.00# zpool import -f backup
bash-3.00#
bash-3.00# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
backup 37.2G 4.45M 37.2G 0% ONLINE -
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
backup 4.42M 37.0G 100K /backup
backup/d1 3.38M 37.0G 3.38M /backup/d1
backup/d2 849K 37.0G 849K /backup/d2
bash-3.00# df -h -F zfs
Filesystem size used avail capacity Mounted on
backup 37G 99K 37G 1% /backup
backup/d1 37G 3.4M 37G 1% /backup/d1
backup/d2 37G 849K 37G 1% /backup/d2
bash-3.00#


############################
#Changes to snv_29 source tree.#
############################

bash-3.00$ diff -u lib/libzfs/common/libzfs_import.c.orig lib/libzfs/common/libzfs_import.c
--- lib/libzfs/common/libzfs_import.c.orig Tue Jan 31 23:51:06 2006
+++ lib/libzfs/common/libzfs_import.c Tue Jan 31 23:51:58 2006
@@ -547,7 +547,7 @@
}

if (nvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_STATE,
- &state) != 0 || state > POOL_STATE_EXPORTED) {
+ &state) != 0 || state > POOL_STATE_DESTROYED) {
nvlist_free(config);
continue;
}
bash-3.00$


bash-3.00$ diff -u uts/common/fs/zfs/vdev.c.orig uts/common/fs/zfs/vdev.c
--- uts/common/fs/zfs/vdev.c.orig Sat Dec 17 06:15:08 2005
+++ uts/common/fs/zfs/vdev.c Wed Feb 1 01:24:58 2006
@@ -1147,7 +1147,7 @@
}

if (state != POOL_STATE_ACTIVE &&
- (!import || state != POOL_STATE_EXPORTED)) {
+ (!import || (state != POOL_STATE_EXPORTED && state != POOL_STATE_DESTROYED))) {
dprintf("pool state not active (%llu)\n", state);
nvlist_free(label);
return (EBADF);
bash-3.00$


bash-3.00$ diff -u cmd/zpool/zpool_main.c.orig cmd/zpool/zpool_main.c
--- cmd/zpool/zpool_main.c.orig Sat Dec 17 06:16:00 2005
+++ cmd/zpool/zpool_main.c Wed Feb 1 01:52:58 2006
@@ -799,7 +799,10 @@

(void) printf(" pool: %s\n", name);
(void) printf(" id: %llu\n", guid);
- (void) printf(" state: %s\n", health);
+ (void) printf(" state: %s", health);
+ if (pool_state == POOL_STATE_DESTROYED)
+ (void) printf(" (DESTROYED)");
+ (void) printf("\n");

switch (reason) {
case ZPOOL_STATUS_MISSING_DEV_R:
@@ -832,7 +835,10 @@
if (strcmp(health, gettext("ONLINE")) == 0) {
(void) printf(gettext("action: The pool can be imported"
" using its name or numeric identifier."));
- if (pool_state != POOL_STATE_EXPORTED)
+ if (pool_state == POOL_STATE_DESTROYED)
+ (void) printf(gettext(" The\n\tpool was destroyed, "
+ "but can be imported using the '-f' flag.\n"));
+ else if (pool_state != POOL_STATE_EXPORTED)
(void) printf(gettext(" The\n\tpool may be active on "
"on another system, but can be imported using\n\t"
"the '-f' flag.\n"));
@@ -842,7 +848,10 @@
(void) printf(gettext("action: The pool can be imported "
"despite missing or damaged devices. The\n\tfault "
"tolerance of the pool may be compromised if imported."));
- if (pool_state != POOL_STATE_EXPORTED)
+ if (pool_state == POOL_STATE_DESTROYED)
+ (void) printf(gettext(" The\n\tpool was destroyed, "
+ "but can be imported using the '-f' flag.\n"));
+ else if (pool_state != POOL_STATE_EXPORTED)
(void) printf(gettext(" The\n\tpool may be active on "
"on another system, but can be imported using\n\t"
"the '-f' flag.\n"));
bash-3.00$

2 comments:

vijayscsa said...

Hi Milek
Thanks for your wonderful blog. I'm struggling to get a script, which is to generate the User home directory usage in a NFS server, which is mounted on different NAS servers, in NIS
can you give me some input,where can i find some sample script...
Thanks a lot
best regards
VJ

Anonymous said...

You saved my data. Can't thank you enough.