• Clonezilla question

    From Ruk@21:1/5 to All on Tue Nov 16 08:12:21 2021
    I've been using this program for years to back up my various Ubuntu and
    Win based PCs without incident. To keep things simple and fast, I
    simply mirror my current hard drives to the same size back up drive,
    then if anything happens, I can just switch out the bad drive for the
    back up and pick up right where I left off. However, since backing up
    the entire drive each time does take time with the larger hard drives,
    I'm wondering if there's a way I can use the program to just back up any changes and/or additions I made since prior back up, that way the
    process would go a lot faster. Thank you!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marco Moock@21:1/5 to All on Tue Nov 16 18:05:02 2021
    If you decide to do create full bit-for-bit copy of your drive it takes
    a long time.
    If you don't want that, create an incremental backup containing just
    files.
    That should go much faster.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paulo da Silva@21:1/5 to All on Tue Nov 16 18:05:23 2021
    Às 17:55 de 16/11/21, Ruk escreveu:
    On 11/16/21 12:05 PM, Marco Moock wrote:
    If you decide to do create full bit-for-bit copy of your drive it takes
    a long time.
    If you don't want that, create an incremental backup containing just
    files.
    That should go much faster.


    True, but can it be done in Clonezilla?
    That cannot be possible.
    For that the program must:
    1. Read a block from source;
    2. Read the corresponding block from dest.;
    3. Compare;
    4. Rewrite the read source block on destination if they are different.

    Compare this with just copying:
    1. Read source block;
    2. Write it to dest.

    As you can see, there is no advantage in first process unless write
    operations are too much slower than read ones which is in general not true.

    Regards.
    Paulo

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Aragorn@21:1/5 to All on Tue Nov 16 19:06:47 2021
    On 16.11.2021 at 18:58, Marco Moock scribbled:

    Am Tue, 16 Nov 2021 12:55:51 -0500
    schrieb Ruk <rthekeeper@zed.net>:

    On 11/16/21 12:05 PM, Marco Moock wrote:
    If you don't want that, create an incremental backup containing
    just files.
    That should go much faster.
    True, but can it be done in Clonezilla?

    I don't know that it supports such a feature, it is a tool for cloning
    disks, partitions and file systems, but I don't think it supports an incremental backup of files.

    When the only tool you have is a hammer, everything in need of fixing
    starts looking like a nail. ;)


    --
    With respect,
    = Aragorn =

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marco Moock@21:1/5 to All on Tue Nov 16 18:58:24 2021
    Am Tue, 16 Nov 2021 12:55:51 -0500
    schrieb Ruk <rthekeeper@zed.net>:

    On 11/16/21 12:05 PM, Marco Moock wrote:
    If you don't want that, create an incremental backup containing just
    files.
    That should go much faster.
    True, but can it be done in Clonezilla?

    I don't know that it supports such a feature, it is a tool for cloning
    disks, partitions and file systems, but I don't think it supports an incremental backup of files.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ruk@21:1/5 to Marco Moock on Tue Nov 16 12:55:51 2021
    On 11/16/21 12:05 PM, Marco Moock wrote:
    If you decide to do create full bit-for-bit copy of your drive it takes
    a long time.
    If you don't want that, create an incremental backup containing just
    files.
    That should go much faster.


    True, but can it be done in Clonezilla?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ray Hughes@21:1/5 to All on Tue Nov 16 18:12:48 2021
    Try Deja Dup. It does incremental.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marco Moock@21:1/5 to All on Tue Nov 16 20:51:26 2021
    Am Tue, 16 Nov 2021 18:12:48 +0000
    schrieb Ray Hughes <rhughes@rays-dell.i-did-not-set--mail-host-address--so-tickle-me>:

    Try Deja Dup. It does incremental.

    I also use it. It is really easy to set up.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Aragorn@21:1/5 to All on Tue Nov 16 21:21:15 2021
    On 16.11.2021 at 20:51, Marco Moock scribbled:

    Am Tue, 16 Nov 2021 18:12:48 +0000
    schrieb Ray Hughes <rhughes@rays-dell.i-did-not-set--mail-host-address--so-tickle-me>:

    Try Deja Dup. It does incremental.

    I also use it. It is really easy to set up.

    Timeshift does incremental backups too, can be used both from the
    command line and by way of a GUI, and can use either rsync or btrfs
    snapshots. It can be run manually or via cron.

    By default, it does not include the home directories — the developer
    insists that it wasn't intended for that — but you can easily make it
    do so, just you can equally easily add or omit directories. If you
    enable the backing up of home directories, then it will by default
    skip ~/.cache, but you may wish to also make it skip /var/cache, and
    perhaps even /var/log.

    It's pretty adaptable, and it hasn't failed me yet so far. It comes preinstalled by default in Manjaro, and it's the backup strategy most
    used in said distribution. Deja Dup is the number two choice.

    --
    With respect,
    = Aragorn =

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Gordon@21:1/5 to Ruk on Tue Nov 16 23:37:44 2021
    On 2021-11-16, Ruk <rthekeeper@zed.net> wrote:
    I've been using this program for years to back up my various Ubuntu and
    Win based PCs without incident. To keep things simple and fast, I
    simply mirror my current hard drives to the same size back up drive,
    then if anything happens, I can just switch out the bad drive for the
    back up and pick up right where I left off. However, since backing up
    the entire drive each time does take time with the larger hard drives,
    I'm wondering if there's a way I can use the program to just back up any changes and/or additions I made since prior back up, that way the
    process would go a lot faster. Thank you!

    When you say imaging a drive I think you are cloning it to the second drive
    as you can just swap drives after the disaster hits.

    Clonezilla images a drive/partition by reading only the blocks in use. The unused space is not read.This saves some time.

    As others have hinted there probably are better ways to achieve your end
    goal without using clonezilla. Remember the Unix philosphy is to have a programme which does one thing, and does it well.

    From the Clonezilla website

    Limitations:

    The destination partition must be equal or larger than the source one.

    Differential/incremental backup is not implemented yet.

    Online imaging/cloning is not implemented yet. The partition to be
    imaged or cloned has to be unmounted.

    Due to the image format limitation, the image can not be explored or mounted. You can _NOT_ recovery single file from the image. However, you
    still have workaround to make it, read this.

    Recovery Clonezilla live with multiple CDs or DVDs is not implemented
    yet. Now all the files have to be in one CD or DVD if you choose to
    create the recovery iso file.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to Ruk on Tue Nov 16 18:34:04 2021
    On 11/16/2021 8:12 AM, Ruk wrote:
    I've been using this program for years to back up my various Ubuntu > and Win based PCs without incident.  To keep things simple and fast, I
    simply mirror my current hard drives to the same size back up drive,
    then if anything happens, I can just switch out the bad drive for the
    back up and pick up right where I left off.  However, since backing
    up the entire drive each time does take time with the larger hard
    drives, I'm wondering if there's a way I can use the program to just
    back up any changes and/or additions I made since prior back up,
    that way the process would go a lot faster.  Thank you!

    There are a couple ways people can write the software.

    They can use scripting, and script the execution of existing
    tools. If a person uses a "progs" package for manipulating
    disk contents, there would be a tendency for the project
    to have the limits imposed by the features of the
    individual progs used.

    If a person sits down and writes source code and compiles
    their own executable, the feature set can be much
    broader, such as supporting Incremental.

    As far as I know, Clonezilla is one of the scripting type
    setups. And can only be as good, as the progs it reuses
    for the purpose.

    *******

    When you use Incrementals, the pattern looks like this.

    Full----Incr---Incr----Incr
    Sunday Monday Tuesday Wednesday

    If you want to restore to the Wednesday state, all
    four files must be readable. This means that eventually,
    if you do enough Incrementals in a row, the "reliability"
    at restoration time drops. All it would take, is for
    one of the four files to be damaged, and the restoration
    to Wednesday would no longer work.

    This means you should truncate the sequence once in a while.

    Full----Incr---Incr----Incr Full-----Incr---Incr
    Sunday Monday Tuesday Wednesday Thursday Friday Saturday

    That way, if the Wednesday file is damaged, I can still
    restore to Thursday, Friday, or Saturday state. Thursday,
    Friday, Saturday, do not depend on the health of Wednesday now.

    Incrementals can save a lot of space, but don't get carried
    away. That's the message here.

    I've had several rather large archives, damaged when the
    RAM was bad on the computer and I did not know it. This is
    why I give the warning about corrupted archives, so you
    don't fall into a similar trap.

    It helps if each file has an internal checksum, so the
    integrity of each archive file can be verified.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to Gordon on Wed Nov 17 01:11:22 2021
    On 11/16/2021 6:37 PM, Gordon wrote:
    On 2021-11-16, Ruk <rthekeeper@zed.net> wrote:
    I've been using this program for years to back up my various Ubuntu and
    Win based PCs without incident. To keep things simple and fast, I
    simply mirror my current hard drives to the same size back up drive,
    then if anything happens, I can just switch out the bad drive for the
    back up and pick up right where I left off. However, since backing up
    the entire drive each time does take time with the larger hard drives,
    I'm wondering if there's a way I can use the program to just back up any
    changes and/or additions I made since prior back up, that way the
    process would go a lot faster. Thank you!

    When you say imaging a drive I think you are cloning it to the second drive as you can just swap drives after the disaster hits.

    Clonezilla images a drive/partition by reading only the blocks in use. The unused space is not read.This saves some time.

    As others have hinted there probably are better ways to achieve your end
    goal without using clonezilla. Remember the Unix philosphy is to have a programme which does one thing, and does it well.

    From the Clonezilla website

    Limitations:

    The destination partition must be equal or larger than the source one.

    Differential/incremental backup is not implemented yet.

    Online imaging/cloning is not implemented yet. The partition to be imaged or cloned has to be unmounted.

    Due to the image format limitation, the image can not be explored or mounted. You can _NOT_ recovery single file from the image. However, you still have workaround to make it, read this.

    Recovery Clonezilla live with multiple CDs or DVDs is not implemented yet. Now all the files have to be in one CD or DVD if you choose to
    create the recovery iso file.


    I would think you could mount a partition inside a flat, dd-like, image.
    It would take a loopback mount, plus an offset parameter.

    And if you need hints, you can try

    sudo disktype ~/Downloads/some.img

    and see what it can tell you. Some simple arithmetic
    with the numbers it provides, can allow working out
    the offset value. A loopback mount accepts a 64-bit
    offset value. I like to enter those in hex, as it's
    cleaner looking.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From jjb@21:1/5 to Ruk on Wed Nov 17 14:57:50 2021
    On 16-11-2021 14:12, Ruk wrote:
    I've been using this program for years to back up my various Ubuntu and
    Win based PCs without incident.  To keep things simple and fast, I
    simply mirror my current hard drives to the same size back up drive,
    then if anything happens, I can just switch out the bad drive for the
    back up and pick up right where I left off.  However, since backing up
    the entire drive each time does take time with the larger hard drives,
    I'm wondering if there's a way I can use the program to just back up any changes and/or additions I made since prior back up, that way the
    process would go a lot faster.  Thank you!

    Two step process:
    Make a Clonezilla full backup say once a month;
    Make backups of the file system (say each day) with rsync in a way that
    only changed/new data is processed (search "jjb rsync" with DDG).
    This way you have only once a month a process that takes quite some
    time, and each day a short time complete backup of your filesystem.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paulo da Silva@21:1/5 to All on Wed Nov 17 18:14:24 2021
    Às 18:05 de 16/11/21, Paulo da Silva escreveu:
    Às 17:55 de 16/11/21, Ruk escreveu:
    On 11/16/21 12:05 PM, Marco Moock wrote:
    If you decide to do create full bit-for-bit copy of your drive it takes
    a long time.
    If you don't want that, create an incremental backup containing just
    files.
    That should go much faster.


    True, but can it be done in Clonezilla?
    That cannot be possible.
    For that the program must:
    1. Read a block from source;
    2. Read the corresponding block from dest.;
    3. Compare;
    4. Rewrite the read source block on destination if they are different.

    Compare this with just copying:
    1. Read source block;
    2. Write it to dest.

    As you can see, there is no advantage in first process unless write operations are too much slower than read ones which is in general not true.

    Sorry, but I confused Clonezilla with another utility.
    Please forget this post.
    BTW, Clonezilla, as per its site, "Differential/incremental backup is
    not implemented yet".

    Paulo

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to jjb on Wed Nov 17 15:20:06 2021
    On 11/17/2021 8:57 AM, jjb wrote:
    On 16-11-2021 14:12, Ruk wrote:
    I've been using this program for years to back up my various Ubuntu and
    Win based PCs without incident.  To keep things simple and fast, I
    simply mirror my current hard drives to the same size back up drive,
    then if anything happens, I can just switch out the bad drive for the
    back up and pick up right where I left off.  However, since backing up
    the entire drive each time does take time with the larger hard drives,
    I'm wondering if there's a way I can use the program to just back up any
    changes and/or additions I made since prior back up, that way the
    process would go a lot faster.  Thank you!

    Two step process:
    Make a Clonezilla full backup say once a month;
    Make backups of the file system (say each day) with rsync in a way that
    only changed/new data is processed (search "jjb rsync" with DDG).
    This way you have only once a month a process that takes quite some
    time, and each day a short time complete backup of your filesystem.


    How about (for extn) ...

    https://wiki.archlinux.org/title/disk_cloning

    # The output is sparse, so it nicely saves space and time for us.
    # And the image can still be "dd" transferred back onto a disk,
    # in place of the damaged partition number.

    cd /media/distro/backups
    sudo e2image -ra -p /dev/sdd1 myimage.img # copy only occupied storage areas
    # -r equals RAW (metadata at same offsets)
    # -a equals include file data
    # -p progress indicator during op

    ls -al myimage.img # lists as 64GB file size
    du myimage.img # but space-on-disk is 10GB (size of files in archive)
    # Transfer speed is decent.

    Best done from a USB pen drive, so the underlying
    file system to be copied, is "at rest". e2image warns
    if you are trying to do this to a "live slash partition".

    To randomly access the backup image later...

    sudo mkdir /mnt/muchlater
    sudo mount myimage.img /mnt/muchlater
    cd /mnt/muchlater/home/username/Downloads
    ls -al

    Since we're using the USB stick, the account shows
    as 1000:1000 instead of username:username. But the file
    is there in the list and can be opened.

    [Picture]

    https://i.postimg.cc/d0yPqQkZ/e2image.gif

    If you boot from the regular HDD slash partition
    and mount /mnt/muchlater, the ls -al then will
    look perfectly normal.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)