• Are files overwritten in place by sftp?

    From bp@www.zefox.net@21:1/5 to All on Sun Nov 17 17:28:40 2024
    In the event one copies a file or directory over an existing
    file or directory is the duplicate written to the same physical
    blocks in storage, or are new physical blocks written with the
    old blocks marked free?

    The situation in mind is using get -r to copy a remote directory,
    then changing or adding a few files to the remote directory followed
    by using get -r to get the directory a second time. Clearly, changed
    files will require writing new blocks on the destination, but what
    about files with unchanged paths and contents?

    The question is motivated by concerns about limited-life storage
    media such as flash.

    Thanks for reading,

    bob prohaska

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Steven G. Kargl@21:1/5 to All on Sun Nov 17 19:50:59 2024
    On Sun, 17 Nov 2024 17:28:40 +0000, bp wrote:

    In the event one copies a file or directory over an existing
    file or directory is the duplicate written to the same physical
    blocks in storage, or are new physical blocks written with the
    old blocks marked free?

    The situation in mind is using get -r to copy a remote directory,
    then changing or adding a few files to the remote directory followed
    by using get -r to get the directory a second time. Clearly, changed
    files will require writing new blocks on the destination, but what
    about files with unchanged paths and contents?

    The question is motivated by concerns about limited-life storage
    media such as flash.

    From the description, it seems that you are looking for
    the net/rsync port. The first time it is executed, it
    will copy the directory. Additional invocations of rsync
    will only copy modified files. rsync also allows you
    to exclude files from being copied such as *.o which can
    change quite often.

    --
    steve

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From bp@www.zefox.net@21:1/5 to Steven G. Kargl on Sun Nov 17 23:35:56 2024
    Steven G. Kargl <sgk@removetroutmask.apl.washington.edu> wrote:
    On Sun, 17 Nov 2024 17:28:40 +0000, bp wrote:

    In the event one copies a file or directory over an existing
    file or directory is the duplicate written to the same physical
    blocks in storage, or are new physical blocks written with the
    old blocks marked free?

    The situation in mind is using get -r to copy a remote directory,
    then changing or adding a few files to the remote directory followed
    by using get -r to get the directory a second time. Clearly, changed
    files will require writing new blocks on the destination, but what
    about files with unchanged paths and contents?

    The question is motivated by concerns about limited-life storage
    media such as flash.

    From the description, it seems that you are looking for
    the net/rsync port. The first time it is executed, it
    will copy the directory. Additional invocations of rsync
    will only copy modified files. rsync also allows you
    to exclude files from being copied such as *.o which can
    change quite often.


    Your surmise is correct, but I wanted to use sftp out of familiarity.
    I'm compiling net/rsync now to have a look at it, but I'm still
    curious how file overwrites, whether across the net or locally, are
    handled.

    Thanks for writing!

    bob prohaska

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Steven G. Kargl@21:1/5 to All on Mon Nov 18 02:19:48 2024
    On Sun, 17 Nov 2024 23:35:56 +0000, bp wrote:

    Steven G. Kargl <sgk@removetroutmask.apl.washington.edu> wrote:
    On Sun, 17 Nov 2024 17:28:40 +0000, bp wrote:

    In the event one copies a file or directory over an existing
    file or directory is the duplicate written to the same physical
    blocks in storage, or are new physical blocks written with the
    old blocks marked free?

    The situation in mind is using get -r to copy a remote directory,
    then changing or adding a few files to the remote directory followed
    by using get -r to get the directory a second time. Clearly, changed
    files will require writing new blocks on the destination, but what
    about files with unchanged paths and contents?

    The question is motivated by concerns about limited-life storage
    media such as flash.

    From the description, it seems that you are looking for
    the net/rsync port. The first time it is executed, it
    will copy the directory. Additional invocations of rsync
    will only copy modified files. rsync also allows you
    to exclude files from being copied such as *.o which can
    change quite often.


    Your surmise is correct, but I wanted to use sftp out of familiarity.
    I'm compiling net/rsync now to have a look at it, but I'm still
    curious how file overwrites, whether across the net or locally, are
    handled.

    It likely depends on the underlying filesystem, but I suspect in
    general that if a file has changed on the system from which you
    are doing the copy, the file on the destination will be written
    to storage without recycling older parts from a previous transfer.

    --
    steve

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)