• msgpack

    From deon@21:2/116.1 to g00r00 on Sun Dec 23 00:29:24 2018
    Hi g00r00,

    A while I posted some suggested fixes for Mystic, which I'm not sure if
    you've seen (I havent seen any comments on it).

    I'll have to dig it out to make sure you see it, but one thing I wanted you
    to consider was to use "move" instead of "copy" in message base packing.
    (Well anywhere really, but msg base packing is the obvious one).

    I run my Mystic in docker, so "temp" and "data" are on 2 different devices - and I'm sure you know you cannot rename across devices.

    I do know also though there are some who are not docker users, but have had their data on a different device (I recall somebody had their data on a USB stick) and thus rename is a problem for them too.

    ...deon

    _--_|\ | Deon George
    / \ | Chinwag BBS - A BBS on a PI in Docker!
    \_.__.*/ |
    V | Coming from the 'burbs of Melbourne, Australia

    --- Mystic BBS v1.12 A39 2018/04/21 (Raspberry Pi/32)
    * Origin: Chinwag | MysticBBS in Docker on a Pi! (21:2/116.1)
  • From g00r00@21:1/112 to deon on Sat Dec 22 22:20:28 2018
    I'll have to dig it out to make sure you see it, but one thing I wanted you to consider was to use "move" instead of "copy" in message base packing. (Well anywhere really, but msg base packing is the obvious one).

    Are you saying these backwards? Mystic uses rename already, so I am assuming you're saying that you'd need it to use copy instead?

    --- Mystic BBS v1.12 A39 2018/04/21 (Windows/32)
    * Origin: Black Flag <ACiD Telnet HQ> blackflagbbs.com (21:1/112)
  • From deon@21:2/116.1 to g00r00 on Sun Dec 23 05:52:54 2018
    On 12/22/18, g00r00 said the following...
    Are you saying these backwards? Mystic uses rename already, so I am assuming you're saying that you'd need it to use copy instead?

    Bah, you got me.

    Yes, can you use "move" instead of "rename".

    ...deon

    _--_|\ | Deon George
    / \ | Chinwag BBS - A BBS on a PI in Docker!
    \_.__.*/ |
    V | Coming from the 'burbs of Melbourne, Australia

    --- Mystic BBS v1.12 A39 2018/04/21 (Raspberry Pi/32)
    * Origin: Chinwag | MysticBBS in Docker on a Pi! (21:2/116.1)
  • From apam@21:1/125 to deon on Sun Dec 23 15:58:12 2018
    On 12/22/18, g00r00 said the following...
    Are you saying these backwards? Mystic uses rename already, so I assuming you're saying that you'd need it to use copy instead?

    Bah, you got me.

    Yes, can you use "move" instead of "rename".

    Move = rename

    You want copy and delete the original i think, to work across devices.

    Andrew

    --- MagickaBBS v0.12alpha (Linux/x86_64)
    * Origin: The Fat Sandwich - sandwich.hopto.org:2023 (21:1/125)
  • From deon@21:2/116.1 to apam on Sun Dec 23 09:30:40 2018
    On 12/23/18, apam said the following...
    Yes, can you use "move" instead of "rename".

    Move = rename

    Well depends...

    "mv" across devices is different to "mv" on the same device.

    In Pascal terms, if I recall, "rename" just updates the inode, whereas "move" is as you say, copy/delete.

    ...deon

    _--_|\ | Deon George
    / \ | Chinwag BBS - A BBS on a PI in Docker!
    \_.__.*/ |
    V | Coming from the 'burbs of Melbourne, Australia

    --- Mystic BBS v1.12 A39 2018/04/21 (Raspberry Pi/32)
    * Origin: Chinwag | MysticBBS in Docker on a Pi! (21:2/116.1)
  • From vk3jed@21:1/109.1 to apam on Sun Dec 23 09:25:18 2018
    On 24-Dec-2018 02:58, apam wrote to deon <=-

    Move = rename

    You want copy and delete the original i think, to work across devices.

    Some OSs (e.g. Linux) may use the same command for both, which is why a "mv" of
    a large file to the same device is very quick, while the "mv" of the same file
    to a different device takes a long time, because the file has to be physically
    moved instead of simply renamed.

    But technically there is a difference between a "move" and a "rename" operation
    at a low level.


    ... Marriage: the price men pay for sex. Sex: the price women pay for marriag --- MultiMail/Win
    * Origin: Bush Track BBS (21:1/109.1)
  • From apam@21:1/125 to vk3jed on Sun Dec 23 20:04:14 2018
    On 24-Dec-2018 02:58, apam wrote to deon <=-

    Move = rename

    You want copy and delete the original i think, to work across dev

    Some OSs (e.g. Linux) may use the same command for both, which is why
    a "mv" of a large file to the same device is very quick, while the "m
    same file to a different device takes a long time, because the file ha physically moved instead of simply renamed.

    Yeah, but I wasn't referring to commands, rather the programming
    interface.

    But technically there is a difference between a "move" and a "rename" operation at a low level.

    I think you'll find at the syscall level (or even in glibc) there is no
    move operation (there's no copy operation either you have to open the
    file read the data and write it to another file, then close the files.)
    The "mv" command will rename on the same device or copy the file then
    delete it across devices as you say.

    If there is a move operation in pascal, (I wouldn't know) it would be implemented in the standard library, and not a system call feature of the kernel.

    Andrew

    --- MagickaBBS v0.12alpha (Linux/x86_64)
    * Origin: The Fat Sandwich - sandwich.hopto.org:2023 (21:1/125)
  • From vk3jed@21:1/109.1 to apam on Sun Dec 23 11:22:54 2018
    On 24-Dec-2018 07:04, apam wrote to vk3jed <=-

    Yeah, but I wasn't referring to commands, rather the programming interface.

    But technically there is a difference between a "move" and a "rename" operation at a low level.

    I think you'll find at the syscall level (or even in glibc) there is no move operation (there's no copy operation either you have to open the
    file read the data and write it to another file, then close the files.) The "mv" command will rename on the same device or copy the file then delete it across devices as you say.

    A lot of stuff is often abstracted, it's only by being observant or doing low level tracing that you find out what's going on. :)

    If there is a move operation in pascal, (I wouldn't know) it would be implemented in the standard library, and not a system call feature of
    the kernel.

    I don't know, it's been years since I've done serious work with Pascal. I'd like to re-acquaint myself with Pascal, but I can't see me getting my head in the right place anytime soon. I'm too restless these days to do anything like coding. :D


    ... When you're a little rabbit, carry a big gun.
    --- MultiMail/Win
    * Origin: Bush Track BBS (21:1/109.1)
  • From g00r00@21:1/112 to deon on Sun Dec 23 10:52:14 2018
    Are you saying these backwards? Mystic uses rename already, so I am assuming you're saying that you'd need it to use copy instead?

    Bah, you got me.

    Yes, can you use "move" instead of "rename".

    Okay. I think I may have to experiment with this a little bit, because if I don't rename, then I believe I have to copy while will be significantly
    slower.

    Worst case scenario I can make it an option to copy or rename in the .ini
    file but maybe I could also name the temp files in the same directory as the message base files (instead of using the temp directories) and that would
    allow me to rename I think.

    I have to think about how I can test this situation and get something set up
    so I can see it happening. I'll have to identify other areas where this may happen.

    --- Mystic BBS v1.12 A39 2018/04/21 (Windows/32)
    * Origin: Black Flag <ACiD Telnet HQ> blackflagbbs.com (21:1/112)
  • From g00r00@21:1/112 to deon on Sun Dec 23 10:59:28 2018
    "mv" across devices is different to "mv" on the same device.

    In Pascal terms, if I recall, "rename" just updates the inode, whereas "move" is as you say, copy/delete.

    There is no Move in FPC that I know of, just a rename and copy. Although I don't use the Free Pascal RTL for Mystic - everything is written by me... so I will have to look at the various OS APIs to see what my options are.

    I want to avoid using copy as much as I can since its obviously much slower.

    --- Mystic BBS v1.12 A39 2018/04/21 (Windows/32)
    * Origin: Black Flag <ACiD Telnet HQ> blackflagbbs.com (21:1/112)
  • From g00r00@21:1/112 to deon on Sun Dec 23 15:08:10 2018
    I do know also though there are some who are not docker users, but have had their data on a different device (I recall somebody had their data
    on a USB stick) and thus rename is a problem for them too.

    Okay, so I have updated the msgpack function so that it now creates any temp files it uses in the same directory that the message base data is stored, so that if they are stored on a different device Mystic can still quickly rename them.

    Hopefully this solves this particular issue. If you notice other places
    where this is a concern please let me know!

    Thanks for the report!

    --- Mystic BBS v1.12 A39 2018/04/21 (Windows/32)
    * Origin: Black Flag <ACiD Telnet HQ> blackflagbbs.com (21:1/112)
  • From deon@21:2/116.1 to g00r00 on Sun Dec 23 21:26:58 2018
    On 12/23/18, g00r00 said the following...
    Okay. I think I may have to experiment with this a little bit, because
    if I don't rename, then I believe I have to copy while will be significantly slower.

    At the moment, you must be copying into temp to start the message base. But yes, you should stop inbound operations while it is occuring ;)

    Worst case scenario I can make it an option to copy or rename in the .ini file but maybe I could also name the temp files in the same directory as the message base files (instead of using the temp directories) and that would allow me to rename I think.

    I think this would work better. Create a temp dir in the message bases, do
    the stuff, and rename back and delete the temp dir. (If the tempdir fails to delete, something went wrong.)

    But still should stop inbound mail (which I'm not sure if you do).

    ...deon

    _--_|\ | Deon George
    / \ | Chinwag BBS - A BBS on a PI in Docker!
    \_.__.*/ |
    V | Coming from the 'burbs of Melbourne, Australia

    --- Mystic BBS v1.12 A39 2018/04/21 (Raspberry Pi/32)
    * Origin: Chinwag | MysticBBS in Docker on a Pi! (21:2/116.1)
  • From g00r00@21:1/112 to deon on Sun Dec 23 17:14:10 2018
    I think this would work better. Create a temp dir in the message bases,
    do the stuff, and rename back and delete the temp dir. (If the tempdir fails to delete, something went wrong.)

    What I did is just create files named $temppack$ or something like that, and
    it will do all of the work right in the directory and then remove those temp files. Hopefully that will clear things up for you!

    --- Mystic BBS v1.12 A39 2018/04/21 (Windows/32)
    * Origin: Black Flag <ACiD Telnet HQ> blackflagbbs.com (21:1/112)
  • From Static@21:2/140 to g00r00 on Thu Dec 27 21:42:50 2018
    On 12/23/18, g00r00 said the following...

    There is no Move in FPC that I know of, just a rename and copy.
    I want to avoid using copy as much as I can since its obviously much slower.

    Does attempting to rename a file across block devices in FPC generate an error you can trap and then just fall back to copy/delete?

    --- Mystic BBS v1.12 A39 2018/04/21 (Linux/64)
    * Origin: Subcarrier BBS (21:2/140)