1. Forum
  2. Usenet
  3. COMP.OS.MSDOS.DJGPP

  • Linking DXE3 with libraries

    From [via djgpp@delorie.com]" @21:1/5 to DJGPP List (E-mail) on Sat Jan 28 12:58:52 2023
    Thread GXE3 with std::vector has grown bit too long, so I'm beginning a new one

    I have not used DXE myself, so my thoughts could be inaccurate


    Some general thoughts:

    - DXE loader has no way to found symbols of running executable (which requests to load DXE) unless
    one tells where they are located

    - linking DXE directly with libraries (like libstdcxx.a or libc.a) will cause duplicate copy of
    symbols. That is going to cause problems when library have global state. I have seen similar
    problems, which have often caused crash of program) for example in Linux, when direct of indirect
    dependencies causes 2 different versions of shared library to be loaded:

    - there is a wat how to provide symbols of executable (including ones from libraries) when loading DXE:

         - http://www.delorie.com/djgpp/v2faq/faq22_15.html

         - https://www.delorie.com/djgpp/doc/utils/utils_20.html


    I tried to run DXE3RES under x86_64 Linux, but is seems that DXE3RES is broken when compiled for
    64-bit systems:

    [andris@ap ~]$ i686-pc-msdosdjgpp-dxe3res -o tmp.c /usr/i686-pc-msdosdjgpp/lib/libc.a
    /usr/i686-pc-msdosdjgpp/lib/libc.a: Exec format error


    Questions which should be answered to use DXE3 (I do not have answers myself, but can be easily
    tested, I'm not going to test myself):

    - are constructors and destructors of global objects executed, when these objects belong to DXE? It
    seems from source code, that they could. It is perhaps safer to verify that it actually works


    I think one should avoid linking DXE with library directly, but provide symbol list generated by
    DXE3RES (or manually) instead. Exception could be if DXE module is only user of library, but
    program itself does not use it


    Andris

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to djgpp@delorie.com] on Sat Jan 28 14:36:18 2023
    On 1/28/23, Andris Pavenis (andris.pavenis@iki.fi) [via
    djgpp@delorie.com] <djgpp@delorie.com> wrote:
    I have not used DXE myself, so my thoughts could be inaccurate

    Some general thoughts:

    - DXE loader has no way to found symbols of running executable (which requests to load DXE) unless
    one tells where they are located

    General issue

    - linking DXE directly with libraries (like libstdcxx.a or libc.a) will
    cause duplicate copy of symbols.

    Linking with libc.a can be especially problematic.

    However, if you have a dxe written in c++, and especially if your
    exe is C-only, linking dxe with libstdc++ is a good thing

    As for duplication of symbols: Well that's a known general issue
    because djgpp libc is a static library.

    I tried to run DXE3RES under x86_64 Linux, but is seems that DXE3RES is broken when compiled for
    64-bit systems:

    [andris@ap ~]$ i686-pc-msdosdjgpp-dxe3res -o tmp.c /usr/i686-pc-msdosdjgpp/lib/libc.a
    /usr/i686-pc-msdosdjgpp/lib/libc.a: Exec format error

    You should run dxe3res against a dxe module, not an archive,
    like: dxe3gen -o tmp.c my.dxe

    As for rest of your notes, I will try looking at them later.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to All on Sat Jan 28 06:28:57 2023
    Good Morning Andris,

    - DXE loader has no way to found symbols of running executable (which requests to load DXE) unless one tells where they are located

    Correct, you need a DXE_EXPORT_TABLE with DXE_EXPORT(func/var) to
    resolve at run time.  See https://bitbucket.org/neozeed/q2dos/src/master/dos/dxe.c for an example
    if interested.
    - linking DXE directly with libraries (like libstdcxx.a or libc.a)
    will cause duplicate copy of symbols. That is going to cause problems
    when library have global state. I have seen similar problems, which
    have often caused crash of program) for example in Linux, when direct
    of indirect dependencies causes 2 different versions of shared library
    to be loaded:
    I'm not quite sure how it works exactly, but with C we just resolve
    everything like above, which seems to be the correct way.  With C++
    things are trickier and we have to do -lstdcxx -lgcc or we get mangled
    names that can't even be exported.
    - are constructors and destructors of global objects executed, when
    these objects belong to DXE? It seems from source code, that they
    could. It is perhaps safer to verify that it actually works

    I believe so, yes.

    I think one should avoid linking DXE with library directly, but
    provide symbol list generated by DXE3RES (or manually) instead.
    Exception could be if DXE module is only user of library, but program
    itself does not use it


    OK, but how do we deal with with the above C++ problem?

    I will say that considering it is DOS the likliehood of someone swapping
    DXEs around is pretty slim, but I understand why you want to make it
    right.  I'd love to contribute code or even something insightful to
    these problems but I am not a compiler/compiler tools programmer.  I
    also apologize in advance for any dumb or obvious questions I may ask as
    a result of this conversation.  I just DXE3GEN like I have been for
    years and use the DXE_EXPORT... stuff to resolve at runtime.  This works
    quite well for C programs, but C++ requires additional work and it's not obvious what to do.

    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to All on Sat Jan 28 15:01:39 2023
    I tried to run DXE3RES under x86_64 Linux, but is seems that DXE3RES is
    broken when compiled for
    64-bit systems:

    [andris@ap ~]$ i686-pc-msdosdjgpp-dxe3res -o tmp.c
    /usr/i686-pc-msdosdjgpp/lib/libc.a
    /usr/i686-pc-msdosdjgpp/lib/libc.a: Exec format error

    You should run dxe3res against a dxe module, not an archive,
    like: dxe3gen -o tmp.c my.dxe

    Fixing my silly typo:
    dxe3res -o tmp.c my.dxe

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to djgpp@delorie.com] on Sat Jan 28 18:17:02 2023
    On 1/28/23 13:28, Frank Sapone (emoaddict15@gmail.com) [via djgpp@delorie.com] wrote:
    Good Morning Andris,

    - DXE loader has no way to found symbols of running executable (which requests to load DXE)
    unless one tells where they are located

    Correct, you need a DXE_EXPORT_TABLE with DXE_EXPORT(func/var) to resolve at run time.  See
    https://bitbucket.org/neozeed/q2dos/src/master/dos/dxe.c for an example if interested.
    - linking DXE directly with libraries (like libstdcxx.a or libc.a) will cause duplicate copy of
    symbols. That is going to cause problems when library have global state. I have seen similar
    problems, which have often caused crash of program) for example in Linux, when direct of
    indirect dependencies causes 2 different versions of shared library to be loaded:
    I'm not quite sure how it works exactly, but with C we just resolve everything like above, which
    seems to be the correct way. With C++ things are trickier and we have to do -lstdcxx -lgcc or we
    get mangled names that can't even be exported.

    The following could work (I only tried under Linux, so for native DJGPP and/or other libraries one
    should modify script):

    [andris@ap dxe]$ cat mk_lib_export_table.sh
    #! /bin/sh

    i686-pc-msdosdjgpp-nm --extern-only /usr/i686-pc-msdosdjgpp/lib/libstdc++.a |\ awk '/^[0-9A-H]/ && ($2 == "W" || $2 == "T") { print $3 }' | sort | uniq |\
    sed -e 's/^_//' >symbols.tmp

    (
       echo "#include <sys/dxe.h>"
       echo ""
       awk '{ print "extern void ", $1, "();"}' symbols.tmp
       echo ""
       echo "DXE_EXPORT_TABLE (exported_symbols)"
       awk '{print "  DXE_EXPORT(",$1,")"}' symbols.tmp
       echo "DXE_EXPORT_END"
       echo ""
       echo "int main() { return 0; }"
    ) | i686-pc-msdosdjgpp-gcc -v -O0 -fno-builtin -x c - -l stdc++


    Only added main() to avoid having linker error.

    Also linked executable to verify that symbols are extracted correctly.

    One can there put code to register generated export table instead.


    I only tried gcc-12.2.0.

    Output of script is given below

    Andris


    [andris@ap dxe]$ ./mk_lib_export_table.sh
    Using built-in specs.
    COLLECT_GCC=i686-pc-msdosdjgpp-gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/lto-wrapper
    Kohde: i686-pc-msdosdjgpp
    Configured with: ../gcc-12.2.0/configure --prefix=/usr --libexecdir=/usr/lib --target=i686-pc-msdosdjgpp --enable-languages=c,c++,fortran,objc,obj-c++ --enable-shared
    --enable-static --enable-th
    reads=no --with-system-zlib --with-isl --enable-lto --disable-libgomp --disable-multilib
    --enable-checking=release --disable-libstdcxx-pch --enable-libstdcxx-filesystem-ts
    --disable-install-libi
    berty
    Säiemalli: single
    Supported LTO compression algorithms: zlib zstd
    gcc-versio 12.2.0 (GCC)
    COLLECT_GCC_OPTIONS='-v' '-O0' '-fno-builtin' '-mtune=generic' '-march=pentiumpro' '-dumpdir' 'a-'
    /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/cc1 -quiet -v -remap - -quiet -dumpdir a- -dumpbase -
    -mtune=generic -march=pentiumpro -O0 -version -fno-builtin -o /tmp/cchC3Spt.s GNU C17 (GCC) versio 12.2.0 (i686-pc-msdosdjgpp)
           käännetty GNU C:n versiolla 12.1.1 20220730, GMP version 6.2.1, MPFR version 4.1.0-p13, MPC
    version 1.2.1, isl version isl-0.25-GMP

    warning: MPFR-otsakeversio 4.1.0-p13 eroaa kirjastoversiosta 4.1.1-p1.
    warning: MPC-otsakeversio 1.2.1 eroaa kirjastoversiosta 1.3.1.
    GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 jätetään puuttuva hakemisto ”/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/include-fixed” huomiotta
    #include "..." -haku alkaa täältä:
    #include <...> -haku alkaa täältä: /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/include /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/sys-include
    /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/include Hakulistan loppu.
    GNU C17 (GCC) versio 12.2.0 (i686-pc-msdosdjgpp)
           käännetty GNU C:n versiolla 12.1.1 20220730, GMP version 6.2.1, MPFR version 4.1.0-p13, MPC
    version 1.2.1, isl version isl-0.25-GMP

    warning: MPFR-otsakeversio 4.1.0-p13 eroaa kirjastoversiosta 4.1.1-p1.
    warning: MPC-otsakeversio 1.2.1 eroaa kirjastoversiosta 1.3.1.
    GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 Compiler executable checksum: 80ce3899c685b09c12b70d8ca8f6fbac COLLECT_GCC_OPTIONS='-v' '-O0' '-fno-builtin' '-mtune=generic' '-march=pentiumpro' '-dumpdir' 'a-'
    /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/bin/as -v -o /tmp/ccIgzJio.o
    /tmp/cchC3Spt.s
    GNU assembler version 2.34 (i686-pc-msdosdjgpp) using BFD version (GNU Binutils) 2.34
    COMPILER_PATH=/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/:/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/:/usr/lib/gcc/i686-pc-msdosdjgpp/:/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/:/usr/lib/gcc/i686-pc-msdosd
    jgpp/:/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/bin/
    LIBRARY_PATH=/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/:/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/lib/

    COLLECT_GCC_OPTIONS='-v' '-O0' '-fno-builtin' '-mtune=generic' '-march=pentiumpro' '-dumpdir' 'a.'
    /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/collect2 -plugin /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/liblto_plugin.so -plugin-opt=/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/lto-wrapper -plugin-opt=-freso
    lution=/tmp/cc8Rf9cH.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc
    -plugin-opt=-pass-through=-lgcc /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/lib/c rt0.o -L/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0 -L/usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/lib /tmp/ccIgzJio.o
    -lstdc++ -lgcc -lc -lgcc
    COLLECT_GCC_OPTIONS='-v' '-O0' '-fno-builtin' '-mtune=generic' '-march=pentiumpro' '-dumpdir' 'a.'
    /usr/lib/gcc/i686-pc-msdosdjgpp/12.2.0/../../../../i686-pc-msdosdjgpp/bin/stubify -v a.out
    stubify for djgpp V2.X executables, Copyright (C) 1995-2003 DJ Delorie
    stubify: a.out -> a.000 -> a.exe

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)