1. Forum
  2. Usenet
  3. COMP.OS.MSDOS.DJGPP

  • Issues using DJGPP with Rust

    From noname422@gmail.com@21:1/5 to All on Wed Apr 22 00:31:43 2020
    Hi everyone!

    I've been trying to use Rust to write software for MS-DOS. At first I was compiling it to real mode code, but I quickly realized that protected mode was more practical, and didn't increase the system requirements since LLVM (which Rust uses) targets the
    386 at a minimum anyway. So now I'm using DJGPP both for linking and to compile pieces of bootstrap C code. However, I've been running into issues with the linker generating incorrect addresses for function calls. I'm asking here because this seems to be
    related to the linking stage and to the combination of Rust with DJGPP.

    C code seems to have no issues calling either other C functions or Rust functions. However, Rust code seems to be having issues calling both other Rust functions and C functions. Both C code and Rust code is using IP-relative CALL instructions (opcode E8)
    . However, calls made from Rust code end up with incorrect offsets and often point to the middle of the target function instead of the beginning. If a function calls another function repeatedly the offset will stay the same, when it should be changing to
    reflect the changing distance to the target. The more functions a function calls, the more out of sync the offset seems to become, and eventually CALL instructions might even point to the middle of the wrong function altogether.

    I've tried changing all sorts of things with both Rust and DJGPP, including the relocation model used for Rust code, and DJGPP linker flags relating to position-independent code.

    Any help with this would be greatly appreciated.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to nona...@gmail.com on Wed Apr 22 13:00:18 2020
    On Wednesday, April 22, 2020 at 3:31:44 AM UTC-4, nona...@gmail.com wrote:
    Hi everyone!

    I've been trying to use Rust to write software for MS-DOS. At first I was compiling it to real mode code, but I quickly realized that protected mode was more practical, and didn't increase the system requirements since LLVM (which Rust uses) targets
    the 386 at a minimum anyway. So now I'm using DJGPP both for linking and to compile pieces of bootstrap C code. However, I've been running into issues with the linker generating incorrect addresses for function calls. I'm asking here because this seems
    to be related to the linking stage and to the combination of Rust with DJGPP.

    C code seems to have no issues calling either other C functions or Rust functions. However, Rust code seems to be having issues calling both other Rust functions and C functions. Both C code and Rust code is using IP-relative CALL instructions (opcode
    E8). However, calls made from Rust code end up with incorrect offsets and often point to the middle of the target function instead of the beginning. If a function calls another function repeatedly the offset will stay the same, when it should be changing
    to reflect the changing distance to the target. The more functions a function calls, the more out of sync the offset seems to become, and eventually CALL instructions might even point to the middle of the wrong function altogether.

    I've tried changing all sorts of things with both Rust and DJGPP, including the relocation model used for Rust code, and DJGPP linker flags relating to position-independent code.

    Any help with this would be greatly appreciated.

    Here's an update. I told LLVM to generate code using the “large” code model, which made it avoid relative CALL instructions, but things still break down when trying to pass pointers between functions.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to noname422@gmail.com on Fri Apr 24 20:29:53 2020
    Copy: djgpp@delorie.com

    On 2020-04-22 09:31, noname422@gmail.com [via djgpp@delorie.com] wrote:
    Hi everyone!

    I've been trying to use Rust to write software for MS-DOS. At first I was compiling it to real mode code, but I quickly realized that protected mode was more practical, and didn't increase the system requirements since LLVM (which Rust uses) targets
    the 386 at a minimum anyway. So now I'm using DJGPP both for linking and to compile pieces of bootstrap C code. However, I've been running into issues with the linker generating incorrect addresses for function calls. I'm asking here because this seems
    to be related to the linking stage and to the combination of Rust with DJGPP.

    C code seems to have no issues calling either other C functions or Rust functions. However, Rust code seems to be having issues calling both other Rust functions and C functions. Both C code and Rust code is using IP-relative CALL instructions (opcode
    E8). However, calls made from Rust code end up with incorrect offsets and often point to the middle of the target function instead of the beginning. If a function calls another function repeatedly the offset will stay the same, when it should be changing
    to reflect the changing distance to the target. The more functions a function calls, the more out of sync the offset seems to become, and eventually CALL instructions might even point to the middle of the wrong function altogether.

    I've tried changing all sorts of things with both Rust and DJGPP, including the relocation model used for Rust code, and DJGPP linker flags relating to position-independent code.

    Any help with this would be greatly appreciated.

    Hey, I like this idea, maybe I can help look into this.

    Could you provide some more information on how to reproduce the
    problem? What does your build environment look like, I suppose you're
    using ld from binutils? How do you invoke the compiler and linker?

    Also some example code or compiled object files would be useful for
    those not familiar with Rust (like myself).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to All on Fri Apr 24 12:59:43 2020
    I sent responses to this by email but it's not showing up on the Google group so I'll past it in here. If you get duplicate messages, sorry about that.

    The first message:

    Sure! I've had a somewhat long conversation on GitHub, where I've been trying to figure it out with the help of a few other people. Here's a link to that. The repository that this discussion is on doesn't currently have the DJGPP version of this, so the
    code there isn't that relevant.

    https://github.com/Serentty/rusty-dos/issues/3

    Maybe I can upload a ZIP file with the code I have at the moment. I'll see if I can get that up in a bit. The tools I'm using are a DJGPP cross compiler (including the linker), and the Rust compiler. I'm using Rust's Cargo build system, so it's that that
    ends up invoking both the Rust and C compilers. I have absolute paths to the DJGPP tools at the moment, so to build it locally you would need to modify both dos32.json (the target specification file) and build.rs (the build script which is responsible
    for compiling the C section).

    I'll ZIP it up and send it off in a few minutes.

    The second message:

    Before I go on, I should mention one small change I've made that I haven't mentioned so far: I told LLVM to generate code using the “large” code model in the dos32.json file, which describes the target platform. This makes it always loads the address
    of a function into a register before calling it, and it seems like it works as a workaround to get it to call the correct address for functions, but the address offset issue is still a big problem, as I'll explain below. This seems connected to it
    calling the wrong address for functions, so I have a hunch that if I can solve it that workaround will not be necessary.

    Here is a ZIP file with the source code, along with two prebuilt executables built by commenting and uncommenting different lines in the source.

    http://www.mediafire.com/file/nnwprp5bdv41gr9/Rust_DJGPP_Stuff.zip

    I uploaded it to MediaFire because Gmail doesn't let you send ZIP files with EXEs in them or encrypted ZIP files. The trick of just changing the extension to something else doesn't work anymore it seems. To build it yourself, you need a recent version of
    Rust (maybe the nightly version, get that one to be safe), DJGPP (which you need to point to in the two files I mentioned in the last message), and to run this Cargo command in the project directory:

    cargo build --target dos32.json --release -Z build-std=core,alloc

    It will build the executable in target/dos32/release if all goes well.

    If you can't get it building, I'll point out a specific issue that happens in the version that tries to use puts(). That's one of the executables that I've included. It seems like the pointer that puts() get is pointing to the wrong place, because the
    string that gets printed out is nonsense and doesn't even match the length of my string.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to All on Fri Apr 24 15:05:17 2020
    I should add, for the “loop” version, you often need to run CLS to clear the screen before running it, or else the text won't show up.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?S=C3=A9bastien_GUILLAUME?@21:1/5 to All on Mon Apr 27 10:33:58 2020
    Hi, nice idea.
    I have read your conversation in github.
    Can you post here the result of the file command applied to files you try to link?
    Remember that djgpp use coff format, which is different from PE for Windows and different to elf to.
    PE is descendent of coff, they have some common parts. Weird functions address can come from here.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to All on Mon Apr 27 12:02:21 2020
    On Monday, April 27, 2020 at 1:33:59 PM UTC-4, Sébastien GUILLAUME wrote:
    Hi, nice idea.
    I have read your conversation in github.
    Can you post here the result of the file command applied to files you try to link?
    Remember that djgpp use coff format, which is different from PE for Windows and different to elf to.
    PE is descendent of coff, they have some common parts. Weird functions address can come from here.

    I ran it on one of the object files for my program and this is what I got.

    dos32-dcb1f62a4632e187.dos32.frqh9yy0-cgu.0.rcgu.o: Intel 80386 COFF object file,
    not stripped, 4 sections, symbol offset=0x234, 15 symbols

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?S=C3=A9bastien_GUILLAUME?@21:1/5 to All on Mon Apr 27 15:18:10 2020
    Le lundi 27 avril 2020 21:02:22 UTC+2, nona...@gmail.com a écrit :
    On Monday, April 27, 2020 at 1:33:59 PM UTC-4, Sébastien GUILLAUME wrote:
    Hi, nice idea.
    I have read your conversation in github.
    Can you post here the result of the file command applied to files you try to link?
    Remember that djgpp use coff format, which is different from PE for Windows and different to elf to.
    PE is descendent of coff, they have some common parts. Weird functions address can come from here.

    I ran it on one of the object files for my program and this is what I got.

    dos32-dcb1f62a4632e187.dos32.frqh9yy0-cgu.0.rcgu.o: Intel 80386 COFF object file,
    not stripped, 4 sections, symbol offset=0x234, 15 symbols

    wich compiler made this files ?

    I don't know how to switch to nightly rust, gentoo doesn't give rustup package, so I can't build the rust part.

    This is what I gave when I try to compile startup.c with this command line :
    $ i586-pc-msdosdjgpp-gcc -c startup.c -o startup.o
    $ file startup.o
    startup.o: Intel 80386 COFF object file, no line number info, not stripped, 5 sections, symbol offset=0x19c, 16 symbols
    $ i586-pc-msdosdjgpp-strings -tx startup.o
    14 .text
    3c .data
    64 .bss
    8c .comment
    11c GCC: (GNU) 7.2.0
    19c .file
    1ae startup.c
    1c0 .text
    1e4 .data
    208 .bss
    22c .comment
    274 _main
    298 _exit
    2c0 .eh_frame
    2ca .eh_frame
    2d4 ___djgpp_nearptr_enable
    2ec _rust_main

    We can see that we have different number of section and symbols. It can be something interesting.

    What do you have with your tools?

    To realise my test, I used :
    - GNU Binutils 2.29
    - GCC 7.2.0
    - file-5.37
    - rustc 1.41.1
    - cargo 1.41.0

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to All on Mon Apr 27 21:39:57 2020
    On Monday, April 27, 2020 at 6:18:11 PM UTC-4, Sébastien GUILLAUME wrote:
    Le lundi 27 avril 2020 21:02:22 UTC+2, nona...@gmail.com a écrit :
    On Monday, April 27, 2020 at 1:33:59 PM UTC-4, Sébastien GUILLAUME wrote:
    Hi, nice idea.
    I have read your conversation in github.
    Can you post here the result of the file command applied to files you try to link?
    Remember that djgpp use coff format, which is different from PE for Windows and different to elf to.
    PE is descendent of coff, they have some common parts. Weird functions address can come from here.

    I ran it on one of the object files for my program and this is what I got.

    dos32-dcb1f62a4632e187.dos32.frqh9yy0-cgu.0.rcgu.o: Intel 80386 COFF object file,
    not stripped, 4 sections, symbol offset=0x234, 15 symbols

    wich compiler made this files ?

    I don't know how to switch to nightly rust, gentoo doesn't give rustup package, so I can't build the rust part.

    This is what I gave when I try to compile startup.c with this command line : $ i586-pc-msdosdjgpp-gcc -c startup.c -o startup.o
    $ file startup.o
    startup.o: Intel 80386 COFF object file, no line number info, not stripped, 5 sections, symbol offset=0x19c, 16 symbols
    $ i586-pc-msdosdjgpp-strings -tx startup.o
    14 .text
    3c .data
    64 .bss
    8c .comment
    11c GCC: (GNU) 7.2.0
    19c .file
    1ae startup.c
    1c0 .text
    1e4 .data
    208 .bss
    22c .comment
    274 _main
    298 _exit
    2c0 .eh_frame
    2ca .eh_frame
    2d4 ___djgpp_nearptr_enable
    2ec _rust_main

    We can see that we have different number of section and symbols. It can be something interesting.

    What do you have with your tools?

    To realise my test, I used :
    - GNU Binutils 2.29
    - GCC 7.2.0
    - file-5.37
    - rustc 1.41.1
    - cargo 1.41.0

    I'm using a nightly build of Rust and Cargo. GCC and binutils are from DJGPP built and gotten from here:

    https://github.com/andrewwutw/build-djgpp

    It's the GCC 7.2.0 version.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?S=C3=A9bastien_GUILLAUME?@21:1/5 to All on Tue Apr 28 09:35:01 2020
    OK,
    can you post an archive with object file used to link the exe?
    The build directory with all temporary files, from rustc and djgpp.
    I have read the new post github, the idea of converting files with objectcopy is interesting, I will build binutils to have the support of pecoff and coff-go32, it's easy with Gentoo.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to All on Tue Apr 28 13:14:19 2020
    On Tuesday, April 28, 2020 at 12:35:02 PM UTC-4, Sébastien GUILLAUME wrote:
    OK,
    can you post an archive with object file used to link the exe?
    The build directory with all temporary files, from rustc and djgpp.
    I have read the new post github, the idea of converting files with objectcopy is interesting, I will build binutils to have the support of pecoff and coff-go32, it's easy with Gentoo.

    Unfortunately stopping a Cargo build in the middle is difficult. It deletes the object files when it's done linking, so I think it caches them in some other format for when you want to rebuild. It might be best to just install nightly Rust unfortunately.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to noname422@gmail.com on Wed Apr 29 00:56:59 2020
    Copy: djgpp@delorie.com

    On 2020-04-28 22:14, noname422@gmail.com [via djgpp@delorie.com] wrote:
    On Tuesday, April 28, 2020 at 12:35:02 PM UTC-4, Sébastien GUILLAUME wrote:
    OK,
    can you post an archive with object file used to link the exe?
    The build directory with all temporary files, from rustc and djgpp.
    I have read the new post github, the idea of converting files with objectcopy is interesting, I will build binutils to have the support of pecoff and coff-go32, it's easy with Gentoo.

    Unfortunately stopping a Cargo build in the middle is difficult. It deletes the object files when it's done linking, so I think it caches them in some other format for when you want to rebuild. It might be best to just install nightly Rust
    unfortunately.


    I found that if you build with:

    $ cargo rustc --target dos32.json --release -Z build-std=core,alloc -- --emit=obj,asm

    it saves the intermediate assembly and object files.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?S=C3=A9bastien_GUILLAUME?@21:1/5 to All on Tue Apr 28 15:36:34 2020
    Le mardi 28 avril 2020 22:14:20 UTC+2, nona...@gmail.com a écrit :
    On Tuesday, April 28, 2020 at 12:35:02 PM UTC-4, Sébastien GUILLAUME wrote:
    OK,
    can you post an archive with object file used to link the exe?
    The build directory with all temporary files, from rustc and djgpp.
    I have read the new post github, the idea of converting files with objectcopy is interesting, I will build binutils to have the support of pecoff and coff-go32, it's easy with Gentoo.

    Unfortunately stopping a Cargo build in the middle is difficult. It deletes the object files when it's done linking, so I think it caches them in some other format for when you want to rebuild. It might be best to just install nightly Rust
    unfortunately.

    Before using this kind of high level tool, it's important to know if they are making what we are waiting. To be sure, begin with a simple Makefile is the best practice.

    I won't install nightly on my system, I don't want to break it and discover strange error message at the next firfox update.

    I have rebuild my binutils, all the needed format are suported.
    I will read this doc https://doc.rust-lang.org/rust-by-example/hello.html and this one https://doc.rust-lang.org/nomicon/ffi.html to know what is possible.

    Your idea is to call C from Rust in DOS and using DJGPP libc, that's right?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to All on Tue Apr 28 22:29:58 2020
    J.W., that's very useful. I knew you could get it to emit assembly, but not object files. Perhaps I could write a script to build the Rust code using Cargo, and then convert the object files to COFF and link them using DJGPP.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noname422@gmail.com@21:1/5 to All on Tue Apr 28 22:28:05 2020
    Sébastien, unfortunately Cargo (the Rust build system) is somewhat necessary with Rust, since it uses modules instead of header files. Invoking the Rust compiler manually with a makefile is possible, but somewhat of a tedious task.

    I won't install nightly on my system, I don't want to break it and discover strange error message at the next firfox update.

    Are you building Firefox yourself? That's the only reason I can see that happening.

    Your idea is to call C from Rust in DOS and using DJGPP libc, that's right?

    Yes, that's right. I want to write MS-DOS applications in Rust. So far I've been able to get it to sort of work, but these linking issues are preventing me from building larger programs.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)