• src/sbbs3/nopen.c

    From Rob Swindell@1:103/705 to Git commit to sbbs/master on Wed Sep 9 02:22:17 2020
    https://gitlab.synchro.net/sbbs/sbbs/-/commit/914790f5931e27c30f94e204
    Modified Files:
    src/sbbs3/nopen.c
    Log Message:
    Improve file copy (fcopy()) performance by more than an order of magnitude.Using a 256KB read buffer for copying files (rather than one byte at a time).Apparently calling fread() is not the same as a bunch of calls to fgetc()after all. Or maybe it was the many calls to fputc() being replaced withfwrite(). Or maybe it was both. Anyway, decreased the time to copy a 1GBfile from and to a Samba share over a Gb Ethernet network from 13 minutesto less than a minute. This matters when sbbs is backing up your data/mailbase and the files are big. The mail base is locked while being backed upand the longer it takes to back up, the longer the mail base is locked andno mail can be received, read or sent during that time.
    --- SBBSecho 3.11-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Wilfred van Velzen@2:280/464 to Rob Swindell on Wed Sep 9 12:49:55 2020
    Hi Rob,

    On 2020-09-09 02:22:17, you wrote to Git commit to sbbs/master:

    https://gitlab.synchro.net/sbbs/sbbs/-/commit/914790f5931e27c30f94e204 Modified Files:
    src/sbbs3/nopen.c
    Log Message:
    Improve file copy (fcopy()) performance by more than an order of magnitude.Using a 256KB read buffer for copying files (rather than one
    byte
    at a time).Apparently calling fread() is not the same as a bunch of calls to fgetc()after all. Or maybe it was the many calls to fputc() being replaced withfwrite(). Or maybe it was both. Anyway, decreased the time
    to
    copy a 1GBfile from and to a Samba share over a Gb Ethernet network from
    13
    minutesto less than a minute. This matters when sbbs is backing up your data/mailbase and the files are big. The mail base is locked while being backed upand the longer it takes to back up, the longer the mail base is locked andno mail can be received, read or sent during that time.

    Maybe this can even be improved upon:

    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.h https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.c


    Bye, Wilfred.

    --- FMail-lnx64 2.1.0.18-B20170815
    * Origin: FMail development HQ (2:280/464)
  • From Digital Man@1:103/705 to Wilfred van Velzen on Wed Sep 9 09:09:30 2020
    Re: Re: src/sbbs3/nopen.c
    By: Wilfred van Velzen to Rob Swindell on Wed Sep 09 2020 12:49 pm

    Hi Rob,

    On 2020-09-09 02:22:17, you wrote to Git commit to sbbs/master:

    https://gitlab.synchro.net/sbbs/sbbs/-/commit/914790f5931e27c30f94e204 Modified Files:
    src/sbbs3/nopen.c
    Log Message:
    Improve file copy (fcopy()) performance by more than an order of magnitude.Using a 256KB read buffer for copying files (rather than one byte
    at a time).Apparently calling fread() is not the same as a bunch of calls to fgetc()after all. Or maybe it was the many calls to fputc() being replaced withfwrite(). Or maybe it was both. Anyway, decreased the time to copy a 1GBfile from and to a Samba share over a Gb
    Ethernet
    network from 13
    minutesto less than a minute. This matters when sbbs is backing up
    your
    data/mailbase and the files are big. The mail base is locked while being backed upand the longer it takes to back up, the longer the mail base is locked andno mail can be received, read or sent during that time.

    Maybe this can even be improved upon:

    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.h https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.c

    Um... thanks? I was getting 400+Mbps, so I'm pretty sastified with that, and it's portable code. But I may experiment with your Linux-specific code at some point.

    digital man

    Sling Blade quote #24:
    Karl: Kaiser blade. I hit my mother upside the head with it. Mmm... Killed her.
    Norco, CA WX: 75.5øF, 25.0% humidity, 0 mph WSW wind, 0.00 inches rain/24hrs --- SBBSecho 3.11-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Wilfred van Velzen@2:280/464 to Digital Man on Wed Sep 9 19:17:00 2020
    Hi Digital,

    On 2020-09-09 09:09:30, you wrote to me:

    Maybe this can even be improved upon:

    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.h
    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.c

    Um... thanks? I was getting 400+Mbps, so I'm pretty sastified with that, and it's portable code. But I may experiment with your Linux-specific
    code
    at some point.

    And on windows just use the CopyFile() OS function...

    Bye, Wilfred.

    --- FMail-lnx64 2.1.0.18-B20170815
    * Origin: FMail development HQ (2:280/464)
  • From Digital Man@1:103/705 to Wilfred van Velzen on Wed Sep 9 13:23:52 2020
    Re: Re: src/sbbs3/nopen.c
    By: Wilfred van Velzen to Digital Man on Wed Sep 09 2020 07:17 pm

    Hi Digital,

    On 2020-09-09 09:09:30, you wrote to me:

    Maybe this can even be improved upon:

    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.h
    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.c

    Um... thanks? I was getting 400+Mbps, so I'm pretty sastified with that, and it's portable code. But I may experiment with your Linux-specific code at some point.

    And on windows just use the CopyFile() OS function...

    Now *that* was a good tip! Thanks!

    digital man

    Synchronet/BBS Terminology Definition #62:
    SAUCE = Standard Architecture for Universal Comment Extensions (ACiD)
    Norco, CA WX: 87.8øF, 24.0% humidity, 5 mph ESE wind, 0.00 inches rain/24hrs --- SBBSecho 3.11-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Wilfred van Velzen@2:280/464 to Digital Man on Wed Sep 9 22:59:02 2020
    Hi Digital,

    On 2020-09-09 13:23:52, you wrote to me:

    Maybe this can even be improved upon:

    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.h
    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.c

    Um... thanks? I was getting 400+Mbps, so I'm pretty sastified with
    that, and it's portable code. But I may experiment with your
    Linux-specific code at some point.

    And on windows just use the CopyFile() OS function...

    Now *that* was a good tip! Thanks!

    It's in my source code, but you have to notice it. ;-)

    I'm interested if you did the speed messurement again, and if you could detect a difference? ;)

    Bye, Wilfred.

    --- FMail-lnx64 2.1.0.18-B20170815
    * Origin: FMail development HQ (2:280/464)
  • From Digital Man@1:103/705 to Wilfred van Velzen on Wed Sep 9 17:48:23 2020
    Re: Re: src/sbbs3/nopen.c
    By: Wilfred van Velzen to Digital Man on Wed Sep 09 2020 10:59 pm

    Hi Digital,

    On 2020-09-09 13:23:52, you wrote to me:

    Maybe this can even be improved upon:

    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.h
    https://sourceforge.net/p/fmail/code/ci/linux/tree/copyFile.c

    Um... thanks? I was getting 400+Mbps, so I'm pretty sastified with
    that, and it's portable code. But I may experiment with your
    Linux-specific code at some point.

    And on windows just use the CopyFile() OS function...

    Now *that* was a good tip! Thanks!

    It's in my source code, but you have to notice it. ;-)

    I'm interested if you did the speed messurement again, and if you could detect a difference? ;)

    I did and CopyFile() was both faster (reduced from 37 to 5 seconds) and had lower CPU utilization. Nice!

    digital man

    Sling Blade quote #17:
    Charles Bushman: A shovel just makes too goddamned much racket.
    Norco, CA WX: 81.2øF, 44.0% humidity, 12 mph NE wind, 0.00 inches rain/24hrs --- SBBSecho 3.11-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Wilfred van Velzen@2:280/464 to Digital Man on Thu Sep 10 11:07:02 2020
    Hi Digital,

    On 2020-09-09 17:48:23, you wrote to me:

    I'm interested if you did the speed messurement again, and if you could
    detect a difference? ;)

    I did and CopyFile() was both faster (reduced from 37 to 5 seconds) and
    had
    lower CPU utilization. Nice!

    That's much bigger improvement, then I would have expected! Nice indeed!

    Bye, Wilfred.

    --- FMail-lnx64 2.1.0.18-B20170815
    * Origin: FMail development HQ (2:280/464)
  • From Digital Man@1:103/705 to Wilfred van Velzen on Thu Sep 10 11:35:39 2020
    Re: Re: src/sbbs3/nopen.c
    By: Wilfred van Velzen to Digital Man on Thu Sep 10 2020 11:07 am

    Hi Digital,

    On 2020-09-09 17:48:23, you wrote to me:

    I'm interested if you did the speed messurement again, and if you could
    detect a difference? ;)

    I did and CopyFile() was both faster (reduced from 37 to 5 seconds)
    and
    had
    lower CPU utilization. Nice!

    That's much bigger improvement, then I would have expected! Nice indeed!

    Yeah, me too. I didn't know that Win32 function existed, so nice tip. Thanks again,

    digital man

    Synchronet "Real Fact" #105:
    Synchronet channel: http://www.youtube.com/channel/UCsQ8iXU5yvrybyoEgo__97A Norco, CA WX: 84.1øF, 27.0% humidity, 1 mph WNW wind, 0.00 inches rain/24hrs --- SBBSecho 3.11-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell@1:103/705 to Git commit to main/sbbs/master on Thu Apr 14 17:30:07 2022
    https://gitlab.synchro.net/main/sbbs/-/commit/1080ab0d8a0d5dca15dcbaa2
    Modified Files:
    src/sbbs3/nopen.c
    Log Message:
    Fix new function: fopenlog()ftello() returns 0 after fnopen(..., O_APPEND). Use filelength() instead.
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)