• Boot regression in Linux v6.4-rc3

    From Frank Scheiner@21:1/5 to All on Fri May 26 13:10:01 2023
    Dear all,

    there is a boot regression in effect in Linux v6.4-rc3 that affects at
    least:

    * rx2620 (w/2 x Montecito and zx1)
    * rx2800-i2 (w/1 x Tukwila)

    ...(see second part of [1] and following posts for more details, [2] and
    [3] for the respective logs), example here:

    ```
    ELILO v3.16 for EFI/IA-64
    ..
    Uncompressing Linux... done
    Loading file AC100221.initrd.img...done
    [ 0.000000] Linux version 6.4.0-rc3 (root@x4270) (ia64-linux-gcc
    (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39) #1 SMP Thu May 25 15:52:20
    CEST 2023
    [ 0.000000] efi: EFI v1.1 by HP
    [ 0.000000] efi: SALsystab=0x3ee7a000 ACPI 2.0=0x3fe2a000
    ESI=0x3ee7b000 SMBIOS=0x3ee7c000 HCDP=0x3fe28000
    [ 0.000000] PCDP: v3 at 0x3fe28000
    [ 0.000000] earlycon: uart8250 at MMIO 0x00000000f4050000 (options
    '9600n8')
    [ 0.000000] printk: bootconsole [uart8250] enabled
    [ 0.000000] ACPI: Early table checksum verification disabled
    [ 0.000000] ACPI: RSDP 0x000000003FE2A000 000028 (v02 HP )
    [ 0.000000] ACPI: XSDT 0x000000003FE2A02C 0000CC (v01 HP rx2620
    00000000 HP 00000000)
    [...]
    [ 3.793350] Run /init as init process
    Loading, please wait...
    Starting systemd-udevd version 252.6-1
    [ 3.951100] ------------[ cut here ]------------
    [ 3.951100] WARNING: CPU: 6 PID: 140 at kernel/module/main.c:1547 __layout_sections+0x370/0x3c0
    [ 3.949512] Unable to handle kernel paging request at virtual address 1000000000000000
    [ 3.951100] Modules linked in:
    [ 3.951100] CPU: 6 PID: 140 Comm: (udev-worker) Not tainted 6.4.0-rc3 #1
    [ 3.956161] (udev-worker)[142]: Oops 11003706212352 [1]
    [ 3.951774] Hardware name: hp server rx2620 , BIOS
    04.29
    11/30/2007
    [ 3.951774]
    [ 3.951774] Call Trace:
    [ 3.958339] Unable to handle kernel paging request at virtual address 1000000000000000
    [ 3.956161] Modules linked in:
    [ 3.951774] [<a0000001000156d0>] show_stack.part.0+0x30/0x60
    [ 3.951774] sp=e000000183a67b20 bsp=e000000183a61628
    [ 3.956161]
    [ 3.956161]
    ```

    [1]: https://lists.debian.org/debian-ia64/2023/05/msg00010.html

    [2]: https://pastebin.com/SAUKbG7Z

    [3]: https://pastebin.com/v1TTB2x3

    With the needed modules compiled into the kernel the rx2620 (only tested
    there yet) boots correctly, though for v6.4-rc2 with kernel oopses (with similar content), for v6.4-rc3 actually w/o kernel oopses.

    According to bisecting between:

    GOOD: `cec24b8b6bb841a19b5c5555b600a511a8988100` and

    BAD: `b6a7828502dc769e1a5329027bc5048222fa210a` (already in effect there)

    ...the problem was introduced with:

    ```
    root@x4270:/usr/src/linux-on-ramdisk# git bisect bad ac3b43283923440900b4f36ca5f9f0b1ca43b70e is the first bad commit
    commit ac3b43283923440900b4f36ca5f9f0b1ca43b70e
    Author: Song Liu <song@kernel.org>
    Date: Mon Feb 6 16:28:02 2023 -0800

    module: replace module_layout with module_memory

    module_layout manages different types of memory (text, data,
    rodata, etc.)
    in one allocation, which is problematic for some reasons:

    1. It is hard to enable CONFIG_STRICT_MODULE_RWX.
    2. It is hard to use huge pages in modules (and not break strict rwx).
    3. Many archs uses module_layout for arch-specific data, but it is not
    obvious how these data are used (are they RO, RX, or RW?)

    Improve the scenario by replacing 2 (or 3) module_layout per module
    with
    up to 7 module_memory per module:

    MOD_TEXT,
    MOD_DATA,
    MOD_RODATA,
    MOD_RO_AFTER_INIT,
    MOD_INIT_TEXT,
    MOD_INIT_DATA,
    MOD_INIT_RODATA,

    and allocating them separately. This adds slightly more entries to
    mod_tree (from up to 3 entries per module, to up to 7 entries per
    module). However, this at most adds a small constant overhead to
    __module_address(), which is expected to be fast.

    Various archs use module_layout for different data. These data are put
    into different module_memory based on their location in module_layout.
    IOW, data that used to go with text is allocated with
    MOD_MEM_TYPE_TEXT;
    data that used to go with data is allocated with MOD_MEM_TYPE_DATA,
    etc.

    module_memory simplifies quite some of the module code. For example,
    ARCH_WANTS_MODULES_DATA_IN_VMALLOC is a lot cleaner, as it just uses a
    different allocator for the data. kernel/module/strict_rwx.c is also
    much cleaner with module_memory.

    Signed-off-by: Song Liu <song@kernel.org>
    Cc: Luis Chamberlain <mcgrof@kernel.org>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Guenter Roeck <linux@roeck-us.net>
    Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
    Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
    Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu>
    Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
    Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>

    arch/arc/kernel/unwind.c | 12 +-
    arch/arm/kernel/module-plts.c | 9 +-
    arch/arm64/kernel/module-plts.c | 13 +-
    arch/ia64/kernel/module.c | 24 +--
    arch/mips/kernel/vpe.c | 11 +-
    arch/parisc/kernel/module.c | 51 ++----
    arch/powerpc/kernel/module_32.c | 7 +-
    arch/s390/kernel/module.c | 26 +--
    arch/x86/kernel/callthunks.c | 4 +-
    arch/x86/kernel/module.c | 4 +-
    include/linux/module.h | 89 +++++++---
    kernel/module/internal.h | 40 ++---
    kernel/module/kallsyms.c | 58 ++++---
    kernel/module/kdb.c | 17 +-
    kernel/module/main.c | 375 ++++++++++++++++++++--------------------
    kernel/module/procfs.c | 16 +-
    kernel/module/strict_rwx.c | 99 ++---------
    kernel/module/tree_lookup.c | 39 ++---
    18 files changed, 427 insertions(+), 467 deletions(-)

    root@x4270:/usr/src/linux-on-ramdisk# git bisect log
    git bisect start
    # status: waiting for both good and bad commits
    # good: [cec24b8b6bb841a19b5c5555b600a511a8988100] Merge tag 'char-misc-6.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc
    git bisect good cec24b8b6bb841a19b5c5555b600a511a8988100
    # status: waiting for bad commit, 1 good commit known
    # bad: [b6a7828502dc769e1a5329027bc5048222fa210a] Merge tag
    'modules-6.4-rc1' of
    git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux
    git bisect bad b6a7828502dc769e1a5329027bc5048222fa210a
    # bad: [3f0dedc39039a75670817a1afffa77b6cee077cb] dmaengine: remove MODULE_LICENSE in non-modules
    git bisect bad 3f0dedc39039a75670817a1afffa77b6cee077cb
    # bad: [b10addf37bbcaee66672eb54c15532266c8daea6] module: add
    symbol-name to pr_debug Absolute symbol
    git bisect bad b10addf37bbcaee66672eb54c15532266c8daea6
    # bad: [85e6f61c134f111232d27d3f63667c1bccbbc12d] module: move early
    sanity checks into a helper
    git bisect bad 85e6f61c134f111232d27d3f63667c1bccbbc12d
    # bad: [05777499a81298ef7e4a5e32a6f744f1f937a80c] ARM: dyndbg: allow
    including dyndbg.h in decompressor
    git bisect bad 05777499a81298ef7e4a5e32a6f744f1f937a80c
    # bad: [efaa2496bae66f0a78efa60d9b73ceef5ec63d79] module: fix MIPS module_layout -> module_memory
    git bisect bad efaa2496bae66f0a78efa60d9b73ceef5ec63d79
    # bad: [9e07f161717ab8e8ac1206bf82546511e24cbb7b] module: Remove the
    unused function within
    git bisect bad 9e07f161717ab8e8ac1206bf82546511e24cbb7b
    # bad: [ac3b43283923440900b4f36ca5f9f0b1ca43b70e] module: replace
    module_layout with module_memory
    git bisect bad ac3b43283923440900b4f36ca5f9f0b1ca43b70e
    # first bad commit: [ac3b43283923440900b4f36ca5f9f0b1ca43b70e] module:
    replace module_layout with module_memory
    ```

    ...and merged with commit `b6a7828502dc769e1a5329027bc5048222fa210a`:

    ```
    commit b6a7828502dc769e1a5329027bc5048222fa210a
    Merge: d06f5a3f7140 8660484ed1cf
    Author: Linus Torvalds <torvalds@linux-foundation.org>
    Date: Thu Apr 27 16:36:55 2023 -0700

    Merge tag 'modules-6.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux

    Pull module updates from Luis Chamberlain:
    "The summary of the changes for this pull requests is:

    - Song Liu's new struct module_memory replacement

    - Nick Alcock's MODULE_LICENSE() removal for non-modules

    - My cleanups and enhancements to reduce the areas where we vmalloc
    module memory for duplicates, and the respective debug code which
    proves the remaining vmalloc pressure comes from userspace.
    [...]
    ```

    Could someone have a look into this, please?

    Cheers,
    Frank

    P.S.
    There is also a bug for this specific commit:

    ```
    kmemleaks on ac3b43283923 ("module: replace module_layout with
    module_memory")
    ```

    ...on [4], reported on 2023-04-03, but I don't know if its content is
    related to the problems on ia64.

    [4]: https://bugzilla.kernel.org/show_bug.cgi?id=217296

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Song Liu@21:1/5 to frank.scheiner@web.de on Fri May 26 19:10:01 2023
    Hi Frank,

    Thanks for the report.

    On Fri, May 26, 2023 at 3:55 AM Frank Scheiner <frank.scheiner@web.de> wrote:

    Dear all,

    there is a boot regression in effect in Linux v6.4-rc3 that affects at
    least:

    * rx2620 (w/2 x Montecito and zx1)
    * rx2800-i2 (w/1 x Tukwila)

    ...(see second part of [1] and following posts for more details, [2] and
    [3] for the respective logs), example here:

    ```
    ELILO v3.16 for EFI/IA-64
    ..
    Uncompressing Linux... done
    Loading file AC100221.initrd.img...done
    [ 0.000000] Linux version 6.4.0-rc3 (root@x4270) (ia64-linux-gcc
    (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39) #1 SMP Thu May 25 15:52:20
    CEST 2023
    [ 0.000000] efi: EFI v1.1 by HP
    [ 0.000000] efi: SALsystab=0x3ee7a000 ACPI 2.0=0x3fe2a000
    ESI=0x3ee7b000 SMBIOS=0x3ee7c000 HCDP=0x3fe28000
    [ 0.000000] PCDP: v3 at 0x3fe28000
    [ 0.000000] earlycon: uart8250 at MMIO 0x00000000f4050000 (options '9600n8')
    [ 0.000000] printk: bootconsole [uart8250] enabled
    [ 0.000000] ACPI: Early table checksum verification disabled
    [ 0.000000] ACPI: RSDP 0x000000003FE2A000 000028 (v02 HP )
    [ 0.000000] ACPI: XSDT 0x000000003FE2A02C 0000CC (v01 HP rx2620 00000000 HP 00000000)
    [...]
    [ 3.793350] Run /init as init process
    Loading, please wait...
    Starting systemd-udevd version 252.6-1
    [ 3.951100] ------------[ cut here ]------------
    [ 3.951100] WARNING: CPU: 6 PID: 140 at kernel/module/main.c:1547 __layout_sections+0x370/0x3c0
    [ 3.949512] Unable to handle kernel paging request at virtual address 1000000000000000
    [ 3.951100] Modules linked in:
    [ 3.951100] CPU: 6 PID: 140 Comm: (udev-worker) Not tainted 6.4.0-rc3 #1
    [ 3.956161] (udev-worker)[142]: Oops 11003706212352 [1]
    [ 3.951774] Hardware name: hp server rx2620 , BIOS
    04.29
    11/30/2007
    [ 3.951774]
    [ 3.951774] Call Trace:
    [ 3.958339] Unable to handle kernel paging request at virtual address 1000000000000000
    [ 3.956161] Modules linked in:
    [ 3.951774] [<a0000001000156d0>] show_stack.part.0+0x30/0x60
    [ 3.951774] sp=e000000183a67b20 bsp=e000000183a61628
    [ 3.956161]
    [ 3.956161]
    ```

    [1]: https://lists.debian.org/debian-ia64/2023/05/msg00010.html

    [2]: https://pastebin.com/SAUKbG7Z

    [3]: https://pastebin.com/v1TTB2x3

    It seems the error happened during the WARN_ON_ONCE. Could you
    please try whether something like the following fixes it?

    diff --git i/kernel/module/main.c w/kernel/module/main.c
    index 0f9183f1ca9f..ae42dfc1a815 100644
    --- i/kernel/module/main.c
    +++ w/kernel/module/main.c
    @@ -1537,7 +1537,7 @@ static void __layout_sections(struct module
    *mod, struct load_info *info, bool i
    || is_init != module_init_layout_section(sname))
    continue;

    - if (WARN_ON_ONCE(type == MOD_INVALID))
    + if (type == MOD_INVALID)
    continue;

    s->sh_entsize =
    module_get_offset_and_type(mod, type, s, i);


    If that doesn't work, maybe we need something like this:

    diff --git i/arch/ia64/kernel/module.c w/arch/ia64/kernel/module.c
    index 3661135da9d9..4e9a7f0498e2 100644
    --- i/arch/ia64/kernel/module.c
    +++ w/arch/ia64/kernel/module.c
    @@ -815,7 +815,7 @@ apply_relocate_add (Elf64_Shdr *sechdrs, const
    char *strtab, unsigned int symind
    uint64_t gp;
    struct module_memory *mod_mem;

    - mod_mem = &mod->mem[MOD_DATA];
    + mod_mem
  • From Frank Scheiner@21:1/5 to Song Liu on Fri May 26 20:40:01 2023
    Hi Song,

    On 26.05.23 18:49, Song Liu wrote:
    Hi Frank,

    Thanks for the report.

    Sure, thanks for your help in this.

    It seems the error happened during the WARN_ON_ONCE. Could you
    please try whether something like the following fixes it?

    diff --git i/kernel/module/main.c w/kernel/module/main.c
    index 0f9183f1ca9f..ae42dfc1a815 100644
    --- i/kernel/module/main.c
    +++ w/kernel/module/main.c
    @@ -1537,7 +1537,7 @@ static void __layout_sections(struct module
    *mod, struct load_info *info, bool i
    || is_init != module_init_layout_section(sname))
    continue;

    - if (WARN_ON_ONCE(type == MOD_INVALID))
    + if (type == MOD_INVALID)
    continue;

    s->sh_entsize =
    module_get_offset_and_type(mod, type, s, i);

    Ok, tried that as -patch1 on top of v6.4-rc3, but didn't help, see [1].

    [1]: https://pastebin.com/UK9v30Ae

    If that doesn't work, maybe we need something like this:

    diff --git i/arch/ia64/kernel/module.c w/arch/ia64/kernel/module.c
    index 3661135da9d9..4e9a7f0498e2 100644
    --- i/arch/ia64/kernel/module.c
    +++ w/arch/ia64/kernel/module.c
    @@ -815,7 +815,7 @@ apply_relocate_add (Elf64_Shdr *sechdrs, const
    char *strtab, unsigned int symind
    uint64_t gp;
    struct module_memory *mod_mem;

    - mod_mem = &mod->mem[MOD_DATA];
    + mod_mem = &mod->mem[MOD_TEXT];
    if (mod_mem->size > MAX_LTOFF)
    /*
    * This takes advantage of fact that
    SHF_ARCH_SMALL gets allocated

    Tried that one as -patch2 on top of v6.4-rc3, but didn't help, see [2].

    [2]: https://pastebin.com/gLupBndU

    I also tried both patches as -patch1plus2 on top of v6.4-rc3 with a
    similar result, see [3].

    [3]: https://pastebin.com/7pXBG5vx

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Song Liu@21:1/5 to frank.scheiner@web.de on Fri May 26 23:20:01 2023
    On Fri, May 26, 2023 at 11:36 AM Frank Scheiner <frank.scheiner@web.de> wrote:

    Hi Song,

    On 26.05.23 18:49, Song Liu wrote:
    Hi Frank,

    Thanks for the report.

    Sure, thanks for your help in this.

    It seems the error happened during the WARN_ON_ONCE. Could you
    please try whether something like the following fixes it?

    diff --git i/kernel/module/main.c w/kernel/module/main.c
    index 0f9183f1ca9f..ae42dfc1a815 100644
    --- i/kernel/module/main.c
    +++ w/kernel/module/main.c
    @@ -1537,7 +1537,7 @@ static void __layout_sections(struct module
    *mod, struct load_info *info, bool i
    || is_init != module_init_layout_section(sname))
    continue;

    - if (WARN_ON_ONCE(type == MOD_INVALID))
    + if (type == MOD_INVALID)
    continue;

    s->sh_entsize =
    module_get_offset_and_type(mod, type, s, i);

    Ok, tried that as -patch1 on top of v6.4-rc3, but didn't help, see [1].

    [1]: https://pastebin.com/UK9v30Ae

    If that doesn't work, maybe we need something like this:

    diff --git i/arch/ia64/kernel/module.c w/arch/ia64/kernel/module.c
    index 3661135da9d9..4e9a7f0498e2 100644
    --- i/arch/ia64/kernel/module.c
    +++ w/arch/ia64/kernel/module.c
    @@ -815,7 +815,7 @@ apply_relocate_add (Elf64_Shdr *sechdrs, const
    char *strtab, unsigned int symind
    uint64_t gp;
    struct module_memory *mod_mem;

    - mod_mem = &mod->mem[MOD_DATA];
    + mod_mem = &mod->mem[MOD_TEXT];
    if (mod_mem->size > MAX_LTOFF)
    /*
    * This takes advantage of fact that SHF_ARCH_SMALL gets allocated

    Tried that one as -patch2 on top of v6.4-rc3, but didn't help, see [2].

    [2]: https://pastebin.com/gLupBndU

    I also tried both patches as -patch1plus2 on top of v6.4-rc3 with a
    similar result, see [3].

    [3]: https://pastebin.com/7pXBG5vx

    Thanks for running the test.

    I am not very familiar with the code, but I think we shouldn't hit that WARN_ON_ONCE. Could you please try with the follow patch to see
    which section caused this issue?

    Thanks,
    Song

    diff --git i/kernel/module/main.c w/kernel/module/main.c
    index 0f9183f1ca9f..caf3d30cd133 100644
    --- i/kernel/module/main.c
    +++ w/kernel/module/main.c
    @@ -1537,8 +1537,11 @@ static void __layout_sections(struct module
    *mod, struct load_info *info, bool i
    || is_init != module_init_layout_section(sname))
    continue;

    - if (WARN_ON_ONCE(type == MOD_INVALID))
    + if (WARN_ON_ONCE(type == MOD_INVALID)) {
    + pr_warn("%s: section %s (sh_flags
    %llx) matched to MOD_INVALID\n", __func__,
    + sname, s->sh_flags);
    continue;
    + }

    s->sh_entsize =
    module_get_offset_and_type(mod, type, s, i);
    pr_debug("\t%s\n", sname);

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Linus Torvalds@21:1/5 to mcgrof@kernel.org on Sat May 27 00:40:01 2023
    On Fri, May 26, 2023 at 2:59 PM Luis Chamberlain <mcgrof@kernel.org> wrote:

    Not saying that debugging commit ac3b4328392344 ("module: replace module_layout with module_memory") is going to be impossible, quite
    the contrary I think it would be good to root cause it, if possible,
    as perhaps it may also be similar to some other future oddball arch
    bug later that may come up.

    I don't have any context - the mailing lists in question that
    apparently this came in on aren't in lore.

    That said, that commit looks odd for the ia64 part.

    In particular, this part:

    - if (mod->core_layout.size > MAX_LTOFF)
    + struct module_memory *mod_mem;
    +
    + mod_mem = &mod->mem[MOD_DATA];

    in apply_relocate_add() (file: arch/ia64/kernel/module.c) seems suspect.

    The previous place that used to look at "mod->core_layout.base"
    converted that to "mod->mem[MOD_TEXT].base". As do other changes in
    other architectures.

    So that "MOD_DATA" looks *very* wrong. Shouldn't core_layout. be
    translated to use "MOD_TEXT" instead?

    Nothing else in the ia64 parts strike me as odd, but that one looks wrong to me.

    But this is my "monkey see, monkey do" pattern matching reaction, not
    from any deeper understanding of the problem (I can't even see the
    report) or really even the code.

    Linus

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Song Liu@21:1/5 to torvalds@linux-foundation.org on Sat May 27 01:00:01 2023
    On Fri, May 26, 2023 at 3:22 PM Linus Torvalds <torvalds@linux-foundation.org> wrote:

    On Fri, May 26, 2023 at 2:59 PM Luis Chamberlain <mcgrof@kernel.org> wrote:

    Not saying that debugging commit ac3b4328392344 ("module: replace module_layout with module_memory") is going to be impossible, quite
    the contrary I think it would be good to root cause it, if possible,
    as perhaps it may also be similar to some other future oddball arch
    bug later that may come up.

    I don't have any context - the mailing lists in question that
    apparently this came in on aren't in lore.

    That said, that commit looks odd for the ia64 part.

    In particular, this part:

    - if (mod->core_layout.size > MAX_LTOFF)
    + struct module_memory *mod_mem;
    +
    + mod_mem = &mod->mem[MOD_DATA];

    in apply_relocate_add() (file: arch/ia64/kernel/module.c) seems suspect.

    The previous place that used to look at "mod->core_layout.base"
    converted that to "mod->mem[MOD_TEXT].base". As do other changes in
    other architectures.

    So that "MOD_DATA" looks *very* wrong. Shouldn't core_layout. be
    translated to use "MOD_TEXT" instead?

    MOD_DATA is likely wrong here. But as Frank tested, changing it to MOD_TEXT didn't fix the issue. I suspect we missed some special cases when we updated layout_sections().

    Thanks,
    Song

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luis Chamberlain@21:1/5 to Frank Scheiner on Sat May 27 01:10:01 2023
    On Fri, May 26, 2023 at 12:55:14PM +0200, Frank Scheiner wrote:
    Dear all,

    there is a boot regression in effect in Linux v6.4-rc3 that affects at
    least:

    * rx2620 (w/2 x Montecito and zx1)
    * rx2800-i2 (w/1 x Tukwila)

    Jesus, ia64 is even dropped from qemu as of 2.11. We're now around qemu
    7.11 to give some perspective. I'm just wondering how to reproduce
    testing easily instead of this ping pong back and forth for this
    architecture for some oddball architectures.

    Through commit 96ec72a3425d1 ("ia64: Mark architecture as orphaned")
    it was noted even the old maintainer no longer had access to working
    machines and so it was orphan'd.

    Not saying that debugging commit ac3b4328392344 ("module: replace
    module_layout with module_memory") is going to be impossible, quite
    the contrary I think it would be good to root cause it, if possible,
    as perhaps it may also be similar to some other future oddball arch
    bug later that may come up.

    But certainly just trying to see what options we have to test this architecture.

    And what's the status of removal for ia64 anyway?

    Luis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to Song Liu on Sat May 27 08:30:01 2023
    Hi,

    On 26.05.23 23:01, Song Liu wrote:
    Thanks for running the test.

    Thanks for staying with me.

    I am not very familiar with the code, but I think we shouldn't hit that WARN_ON_ONCE. Could you please try with the follow patch to see
    which section caused this issue?

    Thanks,
    Song

    diff --git i/kernel/module/main.c w/kernel/module/main.c
    index 0f9183f1ca9f..caf3d30cd133 100644
    --- i/kernel/module/main.c
    +++ w/kernel/module/main.c
    @@ -1537,8 +1537,11 @@ static void __layout_sections(struct module
    *mod, struct load_info *info, bool i
    || is_init != module_init_layout_section(sname))
    continue;

    - if (WARN_ON_ONCE(type == MOD_INVALID))
    + if (WARN_ON_ONCE(type == MOD_INVALID)) {
    + pr_warn("%s: section %s (sh_flags
    %llx) matched to MOD_INVALID\n", __func__,
    + sname, s->sh_flags);
    continue;
    + }

    s->sh_entsize =
    module_get_offset_and_type(mod, type, s, i);
    pr_debug("\t%s\n", sname);

    I put that as -patch3 on top of 6.4-rc3, the result is on [1].

    [1]: https://pastebin.com/KqnWL2pM

    I this time put the whole console messages there, just in case some of
    the earlier messages could be of any help. To jump to the actual oopses,
    search for "Loading, please wait...".

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to Linus Torvalds on Sat May 27 09:10:01 2023
    On 27.05.23 00:22, Linus Torvalds wrote:
    [...]
    But this is my "monkey see, monkey do" pattern matching reaction, not
    from any deeper understanding of the problem (I can't even see the
    report) or really even the code.

    If it is of any help, my initial report is available for example via:

    https://marc.info/?l=linux-ia64&m=168509859125505&w=2

    ...the whole thread is currently at:

    https://marc.info/?t=168509868200003&r=1&w=2

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Linus Torvalds@21:1/5 to frank.scheiner@web.de on Sat May 27 19:30:01 2023
    On Sat, May 27, 2023 at 12:01 AM Frank Scheiner <frank.scheiner@web.de> wrote:

    If it is of any help, my initial report is available for example via:

    https://marc.info/?l=linux-ia64&m=168509859125505&w=2

    ...the whole thread is currently at:

    https://marc.info/?t=168509868200003&r=1&w=2

    This does make it clear just how great a mailing list archive lore is. Konstantin, is there any particular reason why
    linux-ia64@vger.kernel.org isn't in lore? Is it just a rational hatred
    of all things itanium?

    Anyway, the WARN_ON() is likely related, but the bug is clearly an
    unexpected page fault in __copy_user() when called by load_module().

    The ia64 oops output is nasty, presumably because ia64 aggressively
    inlines things. It would help a lot if you enabled debug info (maybe
    you already do?) and then run the oops through
    ./scripts/decode_stacktrace.sh which will figure out line numbers,
    inlining etc.

    Because I don't even see why it would call __copy_user() in the first
    place. This is 'finit_module()' that loads the module data from a
    file, not user space.

    So I guess it must be the strndup_user() in

    mod->args = strndup_user(uargs, ~0UL >> 1);

    but that doesn't look like it should even care about any module
    layout. Plus I would have expected to see strndup_user() in the call
    trace, but whatever.

    End result: that ia64 trace is very hard to read, and _maybe_ running
    it through the decode script might give more information about what it
    is that triggers...

    Linus

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to Linus Torvalds on Sat May 27 20:40:01 2023
    Hi,

    On 27.05.23 19:08, Linus Torvalds wrote:
    Anyway, the WARN_ON() is likely related, but the bug is clearly an
    unexpected page fault in __copy_user() when called by load_module().

    The ia64 oops output is nasty, presumably because ia64 aggressively
    inlines things. It would help a lot if you enabled debug info (maybe
    you already do?)

    I believe it is enabled - I have at least CONFIG_DEBUG_KERNEL=y and CONFIG_DEBUG_INFO=y - my kernel config is on [1] for reference.

    [1]: https://pastebin.com/HRQtZ9vb

    and then run the oops through
    ./scripts/decode_stacktrace.sh which will figure out line numbers,
    inlining etc.

    Because I don't even see why it would call __copy_user() in the first
    place. This is 'finit_module()' that loads the module data from a
    file, not user space.

    So I guess it must be the strndup_user() in

    mod->args = strndup_user(uargs, ~0UL >> 1);

    but that doesn't look like it should even care about any module
    layout. Plus I would have expected to see strndup_user() in the call
    trace, but whatever.

    End result: that ia64 trace is very hard to read, and _maybe_ running
    it through the decode script might give more information about what it
    is that triggers...

    Ok, I put the decoded console messages on [2].

    [2]: https://pastebin.com/dLYMijfS

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Linus Torvalds@21:1/5 to frank.scheiner@web.de on Sat May 27 22:00:01 2023
    On Sat, May 27, 2023 at 11:41 AM Frank Scheiner <frank.scheiner@web.de> wrote:

    Ok, I put the decoded console messages on [2].

    [2]: https://pastebin.com/dLYMijfS

    Ugh. Apparently ia64 decoding isn't great. But at least it gives
    multiple line numbers:

    load_module (kernel/module/main.c:2291 kernel/module/main.c:2412 kernel/module/main.c:2868)

    except your kernel obviously has those test-patches, so I still don't
    know exactly where they are.

    But it looks like it is in move_module(). Strange. I don't know how it
    gets to "__copy_user" from there...

    [ Looks at the ia64 code ]

    Oh.

    It turns out that it *says* __copy_user(), but the code is actually
    shared with the regular memcpy() function, which does

    GLOBAL_ENTRY(memcpy)
    and r28=0x7,in0
    and r29=0x7,in1
    mov f6=f0
    mov retval=in0
    br.cond.sptk .common_code
    ;;

    where that ".common_code" label is - surprise surprise - the common
    copy code, and so when the oops reports that the problem happened in __copy_user(), it actually is in this case just a normal memcpy.

    Ok, so it's probably the

    memcpy(dest, (void *)shdr->sh_addr, shdr->sh_size);

    in move_module() that takes a fault. And looking at the registers,
    the destination is in r17/r18, and your dump has

    unable to handle kernel paging request at virtual address 1000000000000000
    ...
    r17 : 0fffffffffffffff r18 : 1000000000000000

    so it's almost certainly that 'dest' that is bad.

    Which I guess shouldn't surprise anybody.

    But that's where my knowledge of ia64 and the new module loader layout ends.

    Linus

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to All on Sat May 27 23:20:02 2023
    SGksDQoNCk9uIDI3LjA1LjIzIDIxOjM0LCBMaW51cyBUb3J2YWxkcyB3cm90ZToNCj4gT24g U2F0LCBNYXkgMjcsIDIwMjMgYXQgMTE6NDHigK9BTSBGcmFuayBTY2hlaW5lciA8ZnJhbmsu c2NoZWluZXJAd2ViLmRlPiB3cm90ZToNCj4+DQo+PiBPaywgSSBwdXQgdGhlIGRlY29kZWQg Y29uc29sZSBtZXNzYWdlcyBvbiBbMl0uDQo+Pg0KPj4gWzJdOiBodHRwczovL3Bhc3RlYmlu LmNvbS9kTFlNaWpmUw0KPiANCj4gVWdoLiBBcHBhcmVudGx5IGlhNjQgZGVjb2RpbmcgaXNu J3QgZ3JlYXQuIEJ1dCBhdCBsZWFzdCBpdCBnaXZlcw0KPiBtdWx0aXBsZSBsaW5lIG51bWJl cnM6DQo+IA0KPiAgICAgbG9hZF9tb2R1bGUgKGtlcm5lbC9tb2R1bGUvbWFpbi5jOjIyOTEg a2VybmVsL21vZHVsZS9tYWluLmM6MjQxMg0KPiBrZXJuZWwvbW9kdWxlL21haW4uYzoyODY4 KQ0KPiANCj4gZXhjZXB0IHlvdXIga2VybmVsIG9idmlvdXNseSBoYXMgdGhvc2UgdGVzdC1w YXRjaGVzLCBzbyBJIHN0aWxsIGRvbid0DQo+IGtub3cgZXhhY3RseSB3aGVyZSB0aGV5IGFy ZS4NCg0KRXJtLCBJIHNlZS4gSSBkaWQgcmVjcmVhdGUgYSB2YW5pbGxhIHY2LjQtcmMzIGFu ZCByYW4gdGhhdCwgZGVjb2RlZCANCnJlc3VsdCBpcyBvbiBbMV0gLSBub3Qgc3VyZSBpZiBp dCBtYWtlcyBpdCBhIGxpdHRsZSBiZXR0ZXIuDQoNClsxXTogaHR0cHM6Ly9wYXN0ZWJpbi5j b20vejVYekVuaHENCg0KSSBkaWQgYWxzbyB0cnkgdG8gYnVpbGQgYW5kIHJ1biBhIFNQIGtl cm5lbCB0byBtYXliZSBnZXQgYSBiZXR0ZXIgDQpwaWN0dXJlIGluIHRoZSB0cmFjZXMsIGJ1 dCB0aGF0IHNlZW1zIHRvIHJlcXVpcmUgRkxBVE1FTSwgd2hpY2ggc2VlbXMgdG8gDQpub3Qg d29yayBvbiB0aGF0IG1hY2hpbmUgb3IgZHVlIHRvIHRoZSB3YXkgaXQgaXMgY29uZmlndXJl ZCAoYW5kIHllYWgsIA0KaXQgd2FzIGFsc28gdGhlIHdyb25nIGNvbW1pdCBJIHVzZWQgZm9y IGl0IGFuZCBpdCB3YXMgcGF0Y2hlZC4uLik6DQoNCmBgYA0KWyAgICAwLjAwMDAwMF0gTGlu dXggdmVyc2lvbiANCjYuNC4wLXJjMy05MzMxNzRhZTI4YmE3MmFiOGRlNWIzNWNiN2M5OGZj MjExMjM1MDk2LXBhdGNoM19zcCANCihyb290QHg0MjcwKSAoaWE2NC1saW51eC1nY2MgKEdD QykgMTIuMi4wLCBHTlUgbGQgKEdOVSBCaW51dGlscykgMi4zOSkgDQojMSBTYXQgTWF5IDI3 IDIxOjI4OjQ0IENFU1QgMjAyMw0KWy4uLl0NClsgICAgMC4wMDAwMDBdIEFDUEk6IFNTRFQg MHgwMDAwMDAwMDNGRTM1QkE4IDAwMDEzQyAodjAxIEhQICAgICByeDI2MjAgDQowMDAwMDAw NiBJTlRMIDIwMDUwMzA5KQ0KWyAgICAwLjAwMDAwMF0gQUNQSTogTG9jYWwgQVBJQyBhZGRy ZXNzIChfX19fcHRydmFsX19fXykNClsgICAgMC4wMDAwMDBdIDEgQ1BVcyBhdmFpbGFibGUs IDEgQ1BVcyB0b3RhbA0KWy4uLl0NClsgICAgMC4wMDAwMDBdIEtlcm5lbCBwYW5pYyAtIG5v dCBzeW5jaW5nOiBDYW5ub3QgdXNlIEZMQVRNRU0gd2l0aCANCjI0Njc4NE1CIGhvbGUNClsg ICAgMC4wMDAwMDBdIFBsZWFzZSBzd2l0Y2ggb3ZlciB0byBTUEFSU0VNRU0NClsgICAgMC4w MDAwMDBdIC0tLVsgZW5kIEtlcm5lbCBwYW5pYyAtIG5vdCBzeW5jaW5nOiBDYW5ub3QgdXNl IEZMQVRNRU0gDQp3aXRoIDI0Njc4NE1CIGhvbGUNClsgICAgMC4wMDAwMDBdIFBsZWFzZSBz d2l0Y2ggb3ZlciB0byBTUEFSU0VNRU0gXS0tLQ0KYGBgDQoNCj4gQnV0IGl0IGxvb2tzIGxp a2UgaXQgaXMgaW4gbW92ZV9tb2R1bGUoKS4gU3RyYW5nZS4gSSBkb24ndCBrbm93IGhvdyBp dA0KPiBnZXRzIHRvICJfX2NvcHlfdXNlciIgZnJvbSB0aGVyZS4uLg0KPiANCj4gWyBMb29r cyBhdCB0aGUgaWE2NCBjb2RlIF0NCj4gDQo+IE9oLg0KPiANCj4gSXQgdHVybnMgb3V0IHRo YXQgaXQgKnNheXMqIF9fY29weV91c2VyKCksIGJ1dCB0aGUgY29kZSBpcyBhY3R1YWxseQ0K PiBzaGFyZWQgd2l0aCB0aGUgcmVndWxhciBtZW1jcHkoKSBmdW5jdGlvbiwgd2hpY2ggZG9l cw0KPiANCj4gICAgR0xPQkFMX0VOVFJZKG1lbWNweSkNCj4gICAgICAgICAgYW5kICAgICBy Mjg9MHg3LGluMA0KPiAgICAgICAgICBhbmQgICAgIHIyOT0weDcsaW4xDQo+ICAgICAgICAg IG1vdiAgICAgZjY9ZjANCj4gICAgICAgICAgbW92ICAgICByZXR2YWw9aW4wDQo+ICAgICAg ICAgIGJyLmNvbmQuc3B0ayAuY29tbW9uX2NvZGUNCj4gICAgICAgICAgOzsNCj4gDQo+IHdo ZXJlIHRoYXQgIi5jb21tb25fY29kZSIgbGFiZWwgaXMgLSBzdXJwcmlzZSBzdXJwcmlzZSAt IHRoZSBjb21tb24NCj4gY29weSBjb2RlLCBhbmQgc28gd2hlbiB0aGUgb29wcyByZXBvcnRz IHRoYXQgdGhlIHByb2JsZW0gaGFwcGVuZWQgaW4NCj4gX19jb3B5X3VzZXIoKSwgaXQgYWN0 dWFsbHkgaXMgaW4gdGhpcyBjYXNlIGp1c3QgYSBub3JtYWwgbWVtY3B5Lg0KPiANCj4gT2ss IHNvIGl0J3MgcHJvYmFibHkgdGhlDQo+IA0KPiAgICAgICAgICBtZW1jcHkoZGVzdCwgKHZv aWQgKilzaGRyLT5zaF9hZGRyLCBzaGRyLT5zaF9zaXplKTsNCj4gDQo+IGluIG1vdmVfbW9k dWxlKCkgdGhhdCB0YWtlcyBhIGZhdWx0LiAgQW5kIGxvb2tpbmcgYXQgdGhlIHJlZ2lzdGVy cywNCj4gdGhlIGRlc3RpbmF0aW9uIGlzIGluIHIxNy9yMTgsIGFuZCB5b3VyIGR1bXAgaGFz DQo+IA0KPiAgICAgIHVuYWJsZSB0byBoYW5kbGUga2VybmVsIHBhZ2luZyByZXF1ZXN0IGF0 IHZpcnR1YWwgYWRkcmVzcyAxMDAwMDAwMDAwMDAwMDAwDQo+ICAgICAgLi4uDQo+ICAgICAg cjE3IDogMGZmZmZmZmZmZmZmZmZmZiByMTggOiAxMDAwMDAwMDAwMDAwMDAwDQo+IA0KPiBz byBpdCdzIGFsbW9zdCBjZXJ0YWlubHkgdGhhdCAnZGVzdCcgdGhhdCBpcyBiYWQuDQo+IA0K PiBXaGljaCBJIGd1ZXNzIHNob3VsZG4ndCBzdXJwcmlzZSBhbnlib2R5Lg0KPiANCj4gQnV0 IHRoYXQncyB3aGVyZSBteSBrbm93bGVkZ2Ugb2YgaWE2NCBhbmQgdGhlIG5ldyBtb2R1bGUg bG9hZGVyIGxheW91dCBlbmRzLg0KDQpUaGFua3MgZm9yIHlvdXIgaGVscCBhbmQgZ29pbmcg YXMgZmFyIGFzIHlvdSBjb3VsZCwgdGhhdCdzIGdyZWF0bHkgDQphcHByZWNpYXRlZC4gUnVu bmluZyB0aGF0IHN0dWZmIGlzIHN1cmVseSBlYXNpZXIgdGhhbiBkZWJ1Z2dpbmcgaXQuIDot KQ0KDQpDaGVlcnMsDQpGcmFuaw0K

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Song Liu@21:1/5 to torvalds@linux-foundation.org on Sun May 28 07:50:01 2023
    On Sat, May 27, 2023 at 12:34 PM Linus Torvalds <torvalds@linux-foundation.org> wrote:

    On Sat, May 27, 2023 at 11:41 AM Frank Scheiner <frank.scheiner@web.de> wrote:

    Ok, I put the decoded console messages on [2].

    [2]: https://pastebin.com/dLYMijfS

    Ugh. Apparently ia64 decoding isn't great. But at least it gives
    multiple line numbers:

    load_module (kernel/module/main.c:2291 kernel/module/main.c:2412 kernel/module/main.c:2868)

    except your kernel obviously has those test-patches, so I still don't
    know exactly where they are.

    But it looks like it is in move_module(). Strange. I don't know how it
    gets to "__copy_user" from there...

    [ Looks at the ia64 code ]

    Oh.

    It turns out that it *says* __copy_user(), but the code is actually
    shared with the regular memcpy() function, which does

    GLOBAL_ENTRY(memcpy)
    and r28=0x7,in0
    and r29=0x7,in1
    mov f6=f0
    mov retval=in0
    br.cond.sptk .common_code
    ;;

    where that ".common_code" label is - surprise surprise - the common
    copy code, and so when the oops reports that the problem happened in __copy_user(), it actually is in this case just a normal memcpy.

    Ok, so it's probably the

    memcpy(dest, (void *)shdr->sh_addr, shdr->sh_size);

    in move_module() that takes a fault. And looking at the registers,
    the destination is in r17/r18, and your dump has

    unable to handle kernel paging request at virtual address 1000000000000000
    ...
    r17 : 0fffffffffffffff r18 : 1000000000000000

    so it's almost certainly that 'dest' that is bad.

    Yeah, it appears we are writing to mod_mem[MOD_INVALID].

    From the log, the following sections are assigned to MOD_INVALID:

    [ 4.009109] __layout_sections: section .got (sh_flags 10000002)
    matched to MOD_INVALID
    [ 4.009109] __layout_sections: section .sdata (sh_flags 10000003)
    matched to MOD_INVALID
    [ 4.009109] __layout_sections: section .sbss (sh_flags 10000003)
    matched to MOD_INVALID

    AFAICT, .got should go to rodata, while .sdata and .sbss should go
    to (rw)data. However, reading the code before the module_memory
    change, I think they were all copied to (rw)data, which is not ideal but
    most likely OK.

    To match the behavior before the module_memory change, I think
    we need something like the following.

    Frank, could you please give it a try?

    Thanks,
    Song

    diff --git i/kernel/module/main.c w/kernel/module/main.c
    index 0f9183f1ca9f..e4e723e1eb21 100644
    --- i/kernel/module/main.c
    +++ w/kernel/module/main.c
    @@ -1514,14 +1514,14 @@ static void __layout_sections(struct module
    *mod, struct load_info *info, bool i
    MOD_RODATA,
    MOD_RO_AFTER_INIT,
    MOD_DATA,
    - MOD_INVALID, /* This is needed to match the masks array */
    + MOD_DATA,
    };
    static const int init_m_to_mem_type[] = {
    MOD_INIT_TEXT,
    MOD_INIT_RODATA,
    MOD_INVALID,
    MOD_INIT_DATA,
    - MOD_INVALID, /* This is needed to match the masks array */
    + MOD_INIT_DATA,
    };

    for (m = 0; m < ARRAY_SIZE(masks); ++m) {

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to All on Sun May 28 09:40:02 2023
    SGkgU29uZywgTGludXMsDQoNCk9uIDI4LjA1LjIzIDA3OjI0LCBTb25nIExpdSB3cm90ZToN Cj4gQUZBSUNULCAgLmdvdCBzaG91bGQgZ28gdG8gcm9kYXRhLCB3aGlsZSAuc2RhdGEgYW5k IC5zYnNzIHNob3VsZCBnbw0KPiB0byAocncpZGF0YS4gSG93ZXZlciwgcmVhZGluZyB0aGUg Y29kZSBiZWZvcmUgdGhlIG1vZHVsZV9tZW1vcnkNCj4gY2hhbmdlLCBJIHRoaW5rIHRoZXkg d2VyZSBhbGwgY29waWVkIHRvIChydylkYXRhLCB3aGljaCBpcyBub3QgaWRlYWwgYnV0DQo+ IG1vc3QgbGlrZWx5IE9LLg0KPiANCj4gVG8gbWF0Y2ggdGhlIGJlaGF2aW9yIGJlZm9yZSB0 aGUgbW9kdWxlX21lbW9yeSBjaGFuZ2UsIEkgdGhpbmsNCj4gd2UgbmVlZCBzb21ldGhpbmcg bGlrZSB0aGUgZm9sbG93aW5nLg0KPiANCj4gRnJhbmssIGNvdWxkIHlvdSBwbGVhc2UgZ2l2 ZSBpdCBhIHRyeT8NCj4gDQo+IFRoYW5rcywNCj4gU29uZw0KPiANCj4gZGlmZiAtLWdpdCBp L2tlcm5lbC9tb2R1bGUvbWFpbi5jIHcva2VybmVsL21vZHVsZS9tYWluLmMNCj4gaW5kZXgg MGY5MTgzZjFjYTlmLi5lNGU3MjNlMWViMjEgMTAwNjQ0DQo+IC0tLSBpL2tlcm5lbC9tb2R1 bGUvbWFpbi5jDQo+ICsrKyB3L2tlcm5lbC9tb2R1bGUvbWFpbi5jDQo+IEBAIC0xNTE0LDE0 ICsxNTE0LDE0IEBAIHN0YXRpYyB2b2lkIF9fbGF5b3V0X3NlY3Rpb25zKHN0cnVjdCBtb2R1 bGUNCj4gKm1vZCwgc3RydWN0IGxvYWRfaW5mbyAqaW5mbywgYm9vbCBpDQo+ICAgICAgICAg ICAgICAgICAgTU9EX1JPREFUQSwNCj4gICAgICAgICAgICAgICAgICBNT0RfUk9fQUZURVJf SU5JVCwNCj4gICAgICAgICAgICAgICAgICBNT0RfREFUQSwNCj4gLSAgICAgICAgICAgICAg IE1PRF9JTlZBTElELCAgICAvKiBUaGlzIGlzIG5lZWRlZCB0byBtYXRjaCB0aGUgbWFza3Mg YXJyYXkgKi8NCj4gKyAgICAgICAgICAgICAgIE1PRF9EQVRBLA0KPiAgICAgICAgICB9Ow0K PiAgICAgICAgICBzdGF0aWMgY29uc3QgaW50IGluaXRfbV90b19tZW1fdHlwZVtdID0gew0K PiAgICAgICAgICAgICAgICAgIE1PRF9JTklUX1RFWFQsDQo+ICAgICAgICAgICAgICAgICAg TU9EX0lOSVRfUk9EQVRBLA0KPiAgICAgICAgICAgICAgICAgIE1PRF9JTlZBTElELA0KPiAg ICAgICAgICAgICAgICAgIE1PRF9JTklUX0RBVEEsDQo+IC0gICAgICAgICAgICAgICBNT0Rf SU5WQUxJRCwgICAgLyogVGhpcyBpcyBuZWVkZWQgdG8gbWF0Y2ggdGhlIG1hc2tzIGFycmF5 ICovDQo+ICsgICAgICAgICAgICAgICBNT0RfSU5JVF9EQVRBLA0KPiAgICAgICAgICB9Ow0K PiANCj4gICAgICAgICAgZm9yIChtID0gMDsgbSA8IEFSUkFZX1NJWkUobWFza3MpOyArK20p IHsNCg0KVGhhbmtzLCB0aGF0IHBhdGNoIChhcyAtcGF0Y2g0IG9uIHRvcCBvZiB2Ni40LXJj MykgZml4ZXMgdGhlIGJvb3QgDQpyZWdyZXNzaW9uIGZvciBtZSBvbiB0aGUgcngyNjIwOg0K DQpgYGANCkVMSUxPIHYzLjE2IGZvciBFRkkvSUEtNjQNCi4uDQpVbmNvbXByZXNzaW5nIExp bnV4Li4uIGRvbmUNCkxvYWRpbmcgZmlsZSBBQzEwMDIyMS5pbml0cmQuaW1nLi4uZG9uZQ0K WyAgICAwLjAwMDAwMF0gTGludXggdmVyc2lvbiANCjYuNC4wLXJjMy00NGMwMjZhNzNiZTgw MzhmMDNkYmRlZWYwMjhiNjQyODgwY2YxNTExLXBhdGNoNCAocm9vdEB4NDI3MCkgDQooaWE2 NC1saW51eC1nY2MgKEdDQykgMTIuMi4wLCBHTlUgbGQgKEdOVSBCaW51dGlscykgMi4zOSkg IzEgU01QIFN1biBNYXkgDQoyOCAwOTowODo0NCBDRVNUIDIwMjMNClsgICAgMC4wMDAwMDBd IGVmaTogRUZJIHYxLjEgYnkgSFANClsgICAgMC4wMDAwMDBdIGVmaTogU0FMc3lzdGFiPTB4 M2VlN2EwMDAgQUNQSSAyLjA9MHgzZmUyYTAwMCANCkVTST0weDNlZTdiMDAwIFNNQklPUz0w eDNlZTdjMDAwIEhDRFA9MHgzZmUyODAwMA0KWyAgICAwLjAwMDAwMF0gUENEUDogdjMgYXQg MHgzZmUyODAwMA0KWyAgICAwLjAwMDAwMF0gZWFybHljb246IHVhcnQ4MjUwIGF0IE1NSU8g MHgwMDAwMDAwMGY0MDUwMDAwIChvcHRpb25zIA0KJzk2MDBuOCcpDQpbICAgIDAuMDAwMDAw XSBwcmludGs6IGJvb3Rjb25zb2xlIFt1YXJ0ODI1MF0gZW5hYmxlZA0KWyAgICAwLjAwMDAw MF0gQUNQSTogRWFybHkgdGFibGUgY2hlY2tzdW0gdmVyaWZpY2F0aW9uIGRpc2FibGVkDQpb ICAgIDAuMDAwMDAwXSBBQ1BJOiBSU0RQIDB4MDAwMDAwMDAzRkUyQTAwMCAwMDAwMjggKHYw MiBIUCAgICApDQpbICAgIDAuMDAwMDAwXSBBQ1BJOiBYU0RUIDB4MDAwMDAwMDAzRkUyQTAy QyAwMDAwQ0MgKHYwMSBIUCAgICAgcngyNjIwIA0KMDAwMDAwMDAgSFAgICAwMDAwMDAwMCkN ClsuLi5dDQpbICAgIDMuODEwMzQ2XSBSdW4gL2luaXQgYXMgaW5pdCBwcm9jZXNzDQpMb2Fk aW5nLCBwbGVhc2Ugd2FpdC4uLg0KU3RhcnRpbmcgc3lzdGVtZC11ZGV2ZCB2ZXJzaW9uIDI1 Mi42LTENClsgICAgMy45ODUzNzhdIGUxMDAwOiBJbnRlbChSKSBQUk8vMTAwMCBOZXR3b3Jr IERyaXZlcg0KWyAgICAzLjk4OTM3OF0gZTEwMDA6IENvcHlyaWdodCAoYykgMTk5OS0yMDA2 IEludGVsIENvcnBvcmF0aW9uLg0KWyAgICAzLjk5MzM3NV0gR1NJIDI5IChsZXZlbCwgbG93 KSAtPiBDUFUgNyAoMHgwNzAwKSB2ZWN0b3IgNTMNClsgICAgNC4wMzAzODJdIEFDUEk6IGJ1 cyB0eXBlIFVTQiByZWdpc3RlcmVkDQpbICAgIDQuMDMwMzgyXSB1c2Jjb3JlOiByZWdpc3Rl cmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVyIHVzYmZzDQpbICAgIDQuMDM0MzgyXSB1c2Jjb3Jl OiByZWdpc3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVyIGh1Yg0KWyAgICA0LjAzNDM4Ml0g dXNiY29yZTogcmVnaXN0ZXJlZCBuZXcgZGV2aWNlIGRyaXZlciB1c2INClsgICAgNC4wNDA2 MDldIEdTSSAxOCAobGV2ZWwsIGxvdykgLT4gQ1BVIDAgKDB4MDAwMCkgdmVjdG9yIDU0DQpb ICAgIDQuMDQwNjIxXSBlaGNpLXBjaSAwMDAwOjAwOjAxLjI6IEVIQ0kgSG9zdCBDb250cm9s bGVyDQpbLi4uXQ0KWyAgT0sgIF0gRmluaXNoZWQgc3lzdGVtZC11cGRhdGUtdXTigKYgLSBS ZWNvcmQgUnVubGV2ZWwgQ2hhbmdlIGluIFVUTVAuDQpbICAgMTQuNTY4NjA2XSBpb2MxOiBM U0k1M0MxMDMwIEMwOiBDYXBhYmlsaXRpZXM9e0luaXRpYXRvcixUYXJnZXR9DQoNCkRlYmlh biBHTlUvTGludXggMTIgcngyNjIwIC0NCg0KcngyNjIwIGxvZ2luOg0KYGBgDQoNCkdyZWF0 ISBJJ2xsIGdpdmUgaXQgYSB0cnkgb24gbXkgcngyODAwLWkyLCB0b28sIGJ1dCBhc3N1bWUg aXQgd2lsIHdvcmsgDQp0aGVyZSwgdG9vLg0KDQpDaGVlcnMsDQpGcmFuaw0K

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to All on Sun May 28 10:20:01 2023
    SGkgYWdhaW4sDQoNCk9uIDI4LjA1LjIzIDA5OjMwLCBGcmFuayBTY2hlaW5lciB3cm90ZToN Cj4gWy4uLl0NCj4gVGhhbmtzLCB0aGF0IHBhdGNoIChhcyAtcGF0Y2g0IG9uIHRvcCBvZiB2 Ni40LXJjMykgZml4ZXMgdGhlIGJvb3QgDQo+IHJlZ3Jlc3Npb24gZm9yIG1lIG9uIHRoZSBy eDI2MjA6DQo+IA0KPiBbLi4uXQ0KPiANCj4gR3JlYXQhIEknbGwgZ2l2ZSBpdCBhIHRyeSBv biBteSByeDI4MDAtaTIsIHRvbywgYnV0IGFzc3VtZSBpdCB3aWwgd29yayANCj4gdGhlcmUs IHRvby4NCg0KSW5kZWVkLCAtcGF0Y2g0IGFsc28gbWFrZXMgaXQgd29yayBvbiB0aGUgcngy ODAwLWkyOg0KDQpgYGANCkVMSUxPIHYzLjE2IGZvciBFRkkvSUEtNjQNCi4uDQpVbmNvbXBy ZXNzaW5nIExpbnV4Li4uIGRvbmUNCkxvYWRpbmcgZmlsZSBBQzEwMDI3Qi5pbml0cmQuaW1n Li4uZG9uZQ0KWyAgICAwLjAwMDAwMF0gTGludXggdmVyc2lvbiANCjYuNC4wLXJjMy00NGMw MjZhNzNiZTgwMzhmMDNkYmRlZWYwMjhiNjQyODgwY2YxNTExLXBhdGNoNCAocm9vdEB4NDI3 MCkgDQooaWE2NC1saW51eC1nY2MgKEdDQykgMTIuMi4wLCBHTlUgbGQgKEdOVSBCaW51dGls cykgMi4zOSkgIzEgU01QIFN1biBNYXkgDQoyOCAwOTowODo0NCBDRVNUIDIwMjMNClsgICAg MC4wMDAwMDBdIGVmaTogRUZJIHYyLjEgYnkgSFANClsgICAgMC4wMDAwMDBdIGVmaTogU0FM c3lzdGFiPTB4ZGZkZDYzYTE4IEVTST0weGRmZGQ2M2YxOCBBQ1BJIA0KMi4wPTB4M2QzYzQw MTQgSENEUD0weGRmZmZmODc5OCBTTUJJT1M9MHgzZDM2ODAwMA0KWyAgICAwLjAwMDAwMF0g UENEUDogdjMgYXQgMHhkZmZmZjg3OTgNClsgICAgMC4wMDAwMDBdIGVhcmx5Y29uOiB1YXJ0 ODI1MCBhdCBJL08gcG9ydCAweDQwMDAgKG9wdGlvbnMgJzExNTIwMG44JykNClsgICAgMC4w MDAwMDBdIHByaW50azogYm9vdGNvbnNvbGUgW3VhcnQ4MjUwXSBlbmFibGVkDQpbICAgIDAu MDAwMDAwXSBBQ1BJOiBFYXJseSB0YWJsZSBjaGVja3N1bSB2ZXJpZmljYXRpb24gZGlzYWJs ZWQNClsgICAgMC4wMDAwMDBdIEFDUEk6IFJTRFAgMHgwMDAwMDAwMDNEM0M0MDE0IDAwMDAy NCAodjAyIEhQICAgICkNClsgICAgMC4wMDAwMDBdIEFDUEk6IFhTRFQgMHgwMDAwMDAwMDNE M0M0NTgwIDAwMDEyNCAodjAxIEhQICAgICBSWDI4MDAtMiANCjAwMDAwMDAxICAgICAgMDEw MDAwMTMpDQpbLi4uXQ0KWyAgIDM2LjY0OTUzMV0gUnVuIC9pbml0IGFzIGluaXQgcHJvY2Vz cw0KTG9hZGluZywgcGxlYXNlIHdhaXQuLi4NClN0YXJ0aW5nIHN5c3RlbWQtdWRldmQgdmVy c2lvbiAyNTIuNi0xDQpbICAgMzYuODY1NjM1XSBwcHNfY29yZTogTGludXhQUFMgQVBJIHZl ci4gMSByZWdpc3RlcmVkDQpbICAgMzYuODY5MzIxXSBwcHNfY29yZTogU29mdHdhcmUgdmVy LiA1LjMuNiAtIENvcHlyaWdodCAyMDA1LTIwMDcgDQpSb2RvbGZvIEdpb21ldHRpIDxnaW9t ZXR0aUBsaW51eC5pdD4NClsgICAzNi44ODUwMjVdIFBUUCBjbG9jayBzdXBwb3J0IHJlZ2lz dGVyZWQNClsgICAzNi45MTA4NTJdIEFDUEk6IGJ1cyB0eXBlIFVTQiByZWdpc3RlcmVkDQpb ICAgMzYuOTE4MTk4XSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBpbnRlcmZhY2UgZHJpdmVy IHVzYmZzDQpbICAgMzYuOTI0NzYyXSB1c2Jjb3JlOiByZWdpc3RlcmVkIG5ldyBpbnRlcmZh Y2UgZHJpdmVyIGh1Yg0KWyAgIDM2LjkyMjEzM10gaWdiOiBJbnRlbChSKSBHaWdhYml0IEV0 aGVybmV0IE5ldHdvcmsgRHJpdmVyDQpbICAgMzYuOTMxMzg2XSB1c2Jjb3JlOiByZWdpc3Rl cmVkIG5ldyBkZXZpY2UgZHJpdmVyIHVzYg0KWyAgIDM2LjkzODE0OV0gaWdiOiBDb3B5cmln aHQgKGMpIDIwMDctMjAxNCBJbnRlbCBDb3Jwb3JhdGlvbi4NClsuLi5dDQpbICBPSyAgXSBG aW5pc2hlZCBhcHQtZGFpbHktdXBncmFkZeKApiBhcHQgdXBncmFkZSBhbmQgY2xlYW4gYWN0 aXZpdGllcy4NCg0KRGViaWFuIEdOVS9MaW51eCAxMiByeDI4MDAtaTIgLQ0KDQpyeDI4MDAt aTIgbG9naW46DQpgYGANCg0KSSdsbCB0cnkgdG8gdGVzdCB0aGlzIG9uIG90aGVyIG1hY2hp bmVzIChyeDQ2NDAgdy9NYWRpc29uLCByeDI2NjAgDQp3L01vbnRlY2l0by9Nb250dmFsZSwg cng2NjAwIHcvTW9udHZhbGUpIGFzIHdlbGwsIHN0YXJ0aW5nIG9uIFR1ZXNkYXkgaWYgDQpw b3NzaWJsZS4NCg0KKioqKg0KDQpUaGVyZSBpcyBhbiBpc3N1ZSAtIG9ubHkgZm9yIHRoZSBy eDI4MDAtaTIgLSBzZWVtaW5nbHkgcmVsYXRlZCB0byBpdCdzIA0KUENJZSBOSUMocykgYW5k IE1TSXMsIGJ1dCB0aGlzIGlzIGFscmVhZHkgdGhlcmUgaW4gNi4zLnggKHNlZSBmaXJzdCBw YXJ0IA0Kb2YgWzFdKSBhbmQgKipub3QgcmVsYXRlZCB0byB0aGUgcHJvYmxlbSBvZiB0aGlz IHRocmVhZCAoQUZBSUNUKSoqIGFuZCAtIA0KbW9yZSBpbXBvcnRhbnQgLSBkb2Vzbid0IGFm ZmVjdCBvcGVyYXRpb246IFRoZSBtYWNoaW5lIGlzIHdvcmtpbmcgDQpkaXNrbGVzcyB1c2lu ZyBvbmUgb2YgdGhvc2UgaW50ZXJmYWNlcyBzbyBpdCBjYW4ndCBiZSB0aGF0IGJhZC4gSSds bCANCmxvb2sgaW50byBiaXNlY3RpbmcgdGhhdCBpc3N1ZSBhcyB3ZWxsLg0KDQpbMV06IGh0 dHBzOi8vbGlzdHMuZGViaWFuLm9yZy9kZWJpYW4taWE2NC8yMDIzLzA1L21zZzAwMDEwLmh0 bWwNCg0KQ2hlZXJzLA0KRnJhbmsNCg==

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From John Paul Adrian Glaubitz@21:1/5 to Frank Scheiner on Sun May 28 12:20:01 2023
    Hi!

    On Sun, 2023-05-28 at 09:30 +0200, Frank Scheiner wrote:
    Frank, could you please give it a try?

    Thanks,
    Song

    diff --git i/kernel/module/main.c w/kernel/module/main.c
    index 0f9183f1ca9f..e4e723e1eb21 100644
    --- i/kernel/module/main.c
    +++ w/kernel/module/main.c
    @@ -1514,14 +1514,14 @@ static void __layout_sections(struct module
    *mod, struct load_info *info, bool i
    MOD_RODATA,
    MOD_RO_AFTER_INIT,
    MOD_DATA,
    - MOD_INVALID, /* This is needed to match the masks array */
    + MOD_DATA,
    };
    static const int init_m_to_mem_type[] = {
    MOD_INIT_TEXT,
    MOD_INIT_RODATA,
    MOD_INVALID,
    MOD_INIT_DATA,
    - MOD_INVALID, /* This is needed to match the masks array */
    + MOD_INIT_DATA,
    };

    for (m = 0; m < ARRAY_SIZE(masks); ++m) {

    Thanks, that patch (as -patch4 on top of v6.4-rc3) fixes the boot
    regression for me on the rx2620:

    Sounds good. Would be great to get this into 6.4 before release.

    Adrian

    --
    .''`. John Paul Adrian Glaubitz
    : :' : Debian Developer
    `. `' Physicist
    `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Song Liu@21:1/5 to frank.scheiner@web.de on Mon May 29 01:10:01 2023
    On Sun, May 28, 2023 at 1:10 AM Frank Scheiner <frank.scheiner@web.de> wrote:

    Hi again,

    On 28.05.23 09:30, Frank Scheiner wrote:
    [...]
    Thanks, that patch (as -patch4 on top of v6.4-rc3) fixes the boot regression for me on the rx2620:

    [...]

    Great! I'll give it a try on my rx2800-i2, too, but assume it wil work there, too.

    Indeed, -patch4 also makes it work on the rx2800-i2:

    Thanks for running the test!

    I will send the official patch.

    Thanks,
    Song


    ```
    ELILO v3.16 for EFI/IA-64
    ..
    Uncompressing Linux... done
    Loading file AC10027B.initrd.img...done
    [ 0.000000] Linux version 6.4.0-rc3-44c026a73be8038f03dbdeef028b642880cf1511-patch4 (root@x4270) (ia64-linux-gcc (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39) #1 SMP Sun May
    28 09:08:44 CEST 2023
    [ 0.000000] efi: EFI v2.1 by HP
    [ 0.000000] efi: SALsystab=0xdfdd63a18 ESI=0xdfdd63f18 ACPI
    2.0=0x3d3c4014 HCDP=0xdffff8798 SMBIOS=0x3d368000
    [ 0.000000] PCDP: v3 at 0xdffff8798
    [ 0.000000] earlycon: uart8250 at I/O port 0x4000 (options '115200n8')
    [ 0.000000] printk: bootconsole [uart8250] enabled
    [ 0.000000] ACPI: Early table checksum verification disabled
    [ 0.000000] ACPI: RSDP 0x000000003D3C4014 000024 (v02 HP )
    [ 0.000000] ACPI: XSDT 0x000000003D3C4580 000124 (v01 HP RX2800-2 00000001 01000013)
    [...]
    [ 36.649531] Run /init as init process
    Loading, please wait...
    Starting systemd-udevd version 252.6-1
    [ 36.865635] pps_core: LinuxPPS API ver. 1 registered
    [ 36.869321] pps_core: Software ver. 5.3.6 - Copyright 2005-2007
    Rodolfo Giometti <giometti@linux.it>
    [ 36.885025] PTP clock support registered
    [ 36.910852] ACPI: bus type USB registered
    [ 36.918198] usbcore: registered new interface driver usbfs
    [ 36.924762] usbcore: registered new interface driver hub
    [ 36.922133] igb: Intel(R) Gigabit Ethernet Network Driver
    [ 36.931386] usbcore: registered new device driver usb
    [ 36.938149] igb: Copyright (c) 2007-2014 Intel Corporation.
    [...]
    [ OK ] Finished apt-daily-upgrade… apt upgrade and clean activities.

    Debian GNU/Linux 12 rx2800-i2 -

    rx2800-i2 login:
    ```

    I'll try to test this on other machines (rx4640 w/Madison, rx2660 w/Montecito/Montvale, rx6600 w/Montvale) as well, starting on Tuesday if possible.

    ****

    There is an issue - only for the rx2800-i2 - seemingly related to it's
    PCIe NIC(s) and MSIs, but this is already there in 6.3.x (see first part
    of [1]) and **not related to the problem of this thread (AFAICT)** and -
    more important - doesn't affect operation: The machine is working
    diskless using one of those interfaces so it can't be that bad. I'll
    look into bisecting that issue as well.

    [1]: https://lists.debian.org/debian-ia64/2023/05/msg00010.html

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Konstantin Ryabitsev@21:1/5 to Linus Torvalds on Tue May 30 22:40:01 2023
    On Sat, May 27, 2023 at 10:08:24AM -0700, Linus Torvalds wrote:
    This does make it clear just how great a mailing list archive lore is. Konstantin, is there any particular reason why
    linux-ia64@vger.kernel.org isn't in lore? Is it just a rational hatred
    of all things itanium?

    We only add things to lore when someone asks, and nobody's asked. :) I guess I'll consider this an ask and put it on the radar.

    Regards,
    -K

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Linus Torvalds@21:1/5 to konstantin@linuxfoundation.org on Tue May 30 23:30:02 2023
    On Tue, May 30, 2023 at 4:21 PM Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:

    We only add things to lore when someone asks, and nobody's asked. :) I guess I'll consider this an ask and put it on the radar.

    Thanks. It would probably be good to see if there are any other
    vger.kernel.org lists with any appreciable traffic that aren't on
    lore.

    Linus

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Konstantin Ryabitsev@21:1/5 to Linus Torvalds on Tue May 30 23:30:02 2023
    On Tue, May 30, 2023 at 05:04:56PM -0400, Linus Torvalds wrote:
    On Tue, May 30, 2023 at 4:21 PM Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:

    We only add things to lore when someone asks, and nobody's asked. :) I guess
    I'll consider this an ask and put it on the radar.

    Thanks. It would probably be good to see if there are any other vger.kernel.org lists with any appreciable traffic that aren't on
    lore.

    Yes, that will auto-fix itself as we continue to migrate things over to subspace ("new vger").

    -K

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to Song Liu on Wed May 31 20:20:01 2023
    Hi Linus, hi Song,

    On 29.05.23 00:46, Song Liu wrote:
    [...]
    Thanks for running the test!

    I will send the official patch.

    Thanks,
    Song

    With the fix merged and to conclude this, I'd like to add that it was a pleasure to work with you on this problem, although I didn't do much.

    Looking forward to the next occasion - for your sake maybe on another architecture, but can't promise... ;-)

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luis Chamberlain@21:1/5 to frank.scheiner@web.de on Wed May 31 21:40:01 2023
    On Wed, May 31, 2023 at 11:16 AM Frank Scheiner <frank.scheiner@web.de> wrote:
    Looking forward to the next occasion - for your sake maybe on another architecture, but can't promise... ;-)

    I think it would be prudent for Song to also ask you to test his
    future upcoming modules patches on ia64 given how hard it is to
    procure such hardware too.

    Luis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to All on Wed May 31 21:20:01 2023
    T24gMzEuMDUuMjMgMjE6MTQsIEx1aXMgQ2hhbWJlcmxhaW4gd3JvdGU6DQo+IE9uIFdlZCwg TWF5IDMxLCAyMDIzIGF0IDExOjE24oCvQU0gRnJhbmsgU2NoZWluZXIgPGZyYW5rLnNjaGVp bmVyQHdlYi5kZT4gd3JvdGU6DQo+PiBMb29raW5nIGZvcndhcmQgdG8gdGhlIG5leHQgb2Nj YXNpb24gLSBmb3IgeW91ciBzYWtlIG1heWJlIG9uIGFub3RoZXINCj4+IGFyY2hpdGVjdHVy ZSwgYnV0IGNhbid0IHByb21pc2UuLi4gOy0pDQo+IA0KPiBJIHRoaW5rIGl0IHdvdWxkIGJl IHBydWRlbnQgZm9yIFNvbmcgdG8gYWxzbyBhc2sgeW91IHRvIHRlc3QgaGlzDQo+IGZ1dHVy ZSB1cGNvbWluZyBtb2R1bGVzIHBhdGNoZXMgb24gaWE2NCBnaXZlbiBob3cgaGFyZCBpdCBp cyB0bw0KPiBwcm9jdXJlIHN1Y2ggaGFyZHdhcmUgdG9vLg0KDQpTdXJlIQ0KDQpDaGVlcnMs DQpGcmFuaw0K

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)