• new module: Archive::MultiSplit

    From Devin Reade@21:1/5 to All on Tue May 31 05:29:11 2016
    This is actually for a set of three related modules:
    Archive::MultiSplit
    Archive::MultiSplit::Interactive
    Archive::MultiSplit::ZfsDR
    Most of the functionality is in Archive::MultiSplit

    The problem that I'm trying to solve is the ability to get split(1) and
    cat(1) like behavior, except that the data stream can be much larger
    than the size of the largest physical disks currently available. So it
    is a multi-volume (or multi-filesystem) split and cat equivalent.

    I've created the Archive::MultiSplit module that takes a file handle and
    does split-like behavior, except that it has a callback that gets invoked
    when any of the following are triggered:

    1. The percent free space on the filesystem (ignoring any superuser
    reservation) exceeds a threshold
    2. The absolute amount of free space in the filesystem drops below
    a threshold
    3. If configured, the absolute amount of data written by the module
    exceeds a threshold (useful for testing, probably not otherwise)

    Archive::MultiSplit is also responsible for orchestrating the reverse operation, analogous to cat(1), but using the previously created volumes.

    Archive::MultiSplit::Interactive provides the hooks required to get user interaction via /dev/tty; the user is prompted when it is time to update
    or replace the destination filesystem (for splitting) or source filesystem
    (for concatenation).

    Archive::MultiSplit::ZfsDR (short for ZFS disaster recovery) builds on Archive::MultiSplit::Interactive and orchestrates tar(1) or zfs
    send/receive, snapshots, and other ZFS-specific behavior in order to
    create copy of a ZFS dataset (eg: filesystem) and stream it to a set
    of disks for offline disaster recovery purposes, plus being able to reconstitute the DR data set after the fact. It's similar in concept
    to the multi-volume tar behavior of GNU tar, except that it doesn't
    need a character-special device. I'm also writing this for smaller
    sites that may not have tape drives, but can handle removable disks.

    The multisplit(1) and zfsdr(1) perl scripts are front ends to their
    respective modules and as such are pretty minimal.

    zfsdr is targeted at FreeNAS and FreeBSD, while not using any perl
    modules that are not available on a base FreeNAS system; it would
    likely work on Linux.

    multisplit is targeted at any UNIX-like OS.

    AFAICT, this doesn't duplicate anything, particularly WRT the ability
    to span DR disks. Comments? Is it the appropriate namespace for
    the module?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)