• ports/ia64: klibc-utils in Sid should use static binaries

    From Frank Scheiner@21:1/5 to All on Sat Jun 2 21:40:02 2018
    Dear all,

    recently a kernel package for ia64 appeared on ftp.p.d.o and hence I
    thought it might be a good time to give it a try to create a Debian
    GNU/Linux Sid for ia64 FS with `debootstrap` from Sid but executing from
    a Debian Wheezy installation. After some trial and error I was
    successful - at least to some degree.

    Many thanks to all people that were/are involved in getting Debian back
    on Itanium. :-)

    ## Long story short ##

    1. The kernel image provided by [1] does not work on my rx2620 but
    panics early:

    ```
    [ 0.000000] Linux version 4.17.0-rc7-mckinley (debian-kernel@lists.debian.org) (gcc version 7.3.0 (Debian 7.3.0-18))
    #1 SMP Debian 4.17~rc7-1~exp1 (2018-05-29)
    [...]
    [ 0.052000] Kernel panic - not syncing: corrupted stack end detected
    inside scheduler
    [ 0.052000]
    [ 0.052000] ---[ end Kernel panic - not syncing: corrupted stack end detected inside scheduler
    [ 0.052000] ]---
    [ 1.048000] random: fast init done
    [ 53.528000] random: crng init done
    ```

    [1]: http://ftp.ports.debian.org/debian-ports/pool-ia64/main/l/linux/linux-image-4.17.0-rc7-mckinley_4.17~rc7-1~exp1_ia64.deb

    ..but I remember I could successfully run a v4.15.11 Debian kernel
    (though built under Debian Wheezy with the .config of the Wheezy kernel
    as basis for `make oldconfig`) on this machine in mid of April. I
    currently run the Sid FS with the v3.14.x kernel from Wheezy backports
    and the `klibc-utils` from Wheezy.

    2. The currently available `klibc-utils` (for the initramfs) on Sid are
    broken for ia64 (most likely due to no longer being built as static
    binaries like in Wheezy, see [2] for details). When executed most tools segfault, others (without interpreter, meaning non-shared) do not:
    ```
    root@rx2620:/usr/lib/klibc/bin# file cat zcat

    cat: ELF 64-bit LSB executable, IA-64, version 1 (SYSV), statically
    linked, interpreter /lib/klibc-LUey2dkAvJnuUY2c2omL8wPG9DQ.so, stripped

    zcat: ELF 64-bit LSB executable, IA-64, version 1 (SYSV), statically
    linked, stripped

    root@rx2620:/usr/lib/klibc/bin# ./cat /proc/cpuinfo
    Segmentation fault

    root@rx2620:/usr/lib/klibc/bin# lscpu | gzip -c | ./zcat
    Architecture: ia64
    [...]
    Model name: Dual-Core Intel(R) Itanium(R) Processor 9020
    [...]
    ```
    ...whereas on Wheezy the tools (also without interpreter!) work without
    issue:
    ```
    root@rx2620:/tmp/wheezy/usr/lib/klibc/bin# file cat zcat

    cat: ELF 64-bit LSB executable, IA-64, version 1 (SYSV), statically
    linked, stripped

    zcat: ELF 64-bit LSB executable, IA-64, version 1 (SYSV), statically
    linked, stripped

    root@rx2620:/tmp/wheezy/usr/lib/klibc/bin# ./cat /proc/cpuinfo
    processor : 0
    vendor : GenuineIntel
    arch : IA-64
    family : 32
    model : 0
    model name : Dual-Core Intel(R) Itanium(R) Processor 9020
    [...]
    ```

    [2]: https://salsa.debian.org/kernel-team/klibc/commit/07809ffd86e1e4ca5a4e98e4708a1c1fae10babf

    I don't have a clue how to rebuild the `klibc-utils` package in this Sid
    FS with some of the build dependencies missing. So could someone please
    rebuild them with the ia64 parts of [2] reverted and upload a new
    package to ftp.p.d.o?

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From John Paul Adrian Glaubitz@21:1/5 to Frank Scheiner on Mon Jun 4 11:10:02 2018
    Hi Frank!

    On 06/02/2018 09:31 PM, Frank Scheiner wrote:
    [2]: https://salsa.debian.org/kernel-team/klibc/commit/07809ffd86e1e4ca5a4e98e4708a1c1fae10babf

    I don't have a clue how to rebuild the `klibc-utils` package in this Sid FS with some of the build dependencies missing. So could someone please rebuild them with the ia64 parts of [2] reverted and upload a new package to ftp.p.d.o?

    Which build dependencies are you missing? I just did a test build
    with sbuild and the build dependencies install just fine without
    any issues.

    Note: You can always get older versions of certain packages from snapshot.debian.org in case you run into a situation where build
    dependencies cannot be installed.

    Also, you should create an account on salsa.debian.org, fork the
    klibc repository into your own salsa home, create a branch for the
    ia64-fix, revert the patch and send a pull request to Ben.

    Please don't forget to add an entry to debian/changelog with
    "dch", but I think you're familiar with that already.

    Adrian

    --
    .''`. John Paul Adrian Glaubitz
    : :' : Debian Developer - glaubitz@debian.org
    `. `' Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
    `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to John Paul Adrian Glaubitz on Mon Jun 4 17:00:01 2018
    Hi Adrian,

    On 06/04/2018 11:02 AM, John Paul Adrian Glaubitz wrote:
    Which build dependencies are you missing? I just did a test build
    with sbuild and the build dependencies install just fine without
    any issues.

    I tried to follow the guide on the Debian wiki ([1]) and the `apt[-get] build-dep klibc` command fails:
    ```
    root@rx2620:~# apt build-dep klibc
    Reading package lists... Done
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.
    The following information may help to resolve the situation:

    The following packages have unmet dependencies:
    builddeps:klibc : Depends: debhelper (>= 9~) but it is not going to be installed
    E: Unable to correct problems, you have held broken packages.

    root@rx2620:~# apt install debhelper
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    [...]

    The following packages have unmet dependencies:
    debhelper : Depends: po-debconf but it is not going to be installed
    E: Unable to correct problems, you have held broken packages.

    root@rx2620:~# apt install po-debconf
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    [...]

    The following packages have unmet dependencies:
    po-debconf : Depends: gettext (>= 0.16) but it is not going to be
    installed
    Depends: intltool-debian (>= 0.34.2+20060512) but it is
    not going to be installed
    E: Unable to correct problems, you have held broken packages.

    root@rx2620:~# apt install gettext
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    [...]

    The following packages have unmet dependencies:
    gettext : Depends: libcroco3 (>= 0.6.2) but it is not going to be
    installed
    Depends: libxml2 (>= 2.9.1) but it is not going to be installed
    E: Unable to correct problems, you have held broken packages.
    ```

    [1]: https://wiki.debian.org/BuildingTutorial#The_packaging_workflow

    UPDATE: I think I found the problem. I had deb lines for unstable
    **and** unreleased **and** experimental (plus deb-src line for sid from
    f.d.o) in my `/etc/apt/sources.list`. Using only the deb line for
    "unstable" and the deb-src line allows me to install the build
    dependencies for klibc:
    ```
    root@rx2620:~# apt build-dep klibc
    Reading package lists... Done
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    The following NEW packages will be installed:
    autopoint bison debhelper dh-autoreconf dh-strip-nondeterminism dwz
    gettext gettext-base intltool-debian
    libarchive-zip-perl libbison-dev libcroco3
    libfile-stripnondeterminism-perl libfreetype6 libgraphite2-3 libharfbuzz0b
    libicu-le-hb0 libicu60 libpng16-16 libxml2 po-debconf
    0 upgraded, 21 newly installed, 0 to remove and 3 not upgraded.
    Need to get 16.1 MB of archives.
    After this operation, 60.3 MB of additional disk space will be used.
    Do you want to continue? [Y/n]
    [...]
    ```

    Maybe I should use sbuild in the future.

    Also, you should create an account on salsa.debian.org, fork the
    klibc repository into your own salsa home, create a branch for the
    ia64-fix, revert the patch and send a pull request to Ben.

    I wasn't sure if code parts for an unofficial Debian arch can be
    included in the official repos, but on the other hand as we did the same
    thing for the GRUB patches to support GPT on sparc64, it actually makes
    perfect sense to do so. What was I thinking... :-/


    Please don't forget to add an entry to debian/changelog with
    "dch", but I think you're familiar with that already.

    Right, but still thanks for the reminder. :-)

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From John Paul Adrian Glaubitz@21:1/5 to Frank Scheiner on Mon Jun 4 17:10:02 2018
    On 06/04/2018 04:59 PM, Frank Scheiner wrote:
    I tried to follow the guide on the Debian wiki ([1]) and the `apt[-get] build-dep klibc` command fails:


    [1]: https://wiki.debian.org/BuildingTutorial#The_packaging_workflow

    Why aren't you using sbuild? You should always build in a clean environment. Not on your normal system.


    Maybe I should use sbuild in the future.

    Indeed.

    Also, you should create an account on salsa.debian.org, fork the
    klibc repository into your own salsa home, create a branch for the
    ia64-fix, revert the patch and send a pull request to Ben.

    I wasn't sure if code parts for an unofficial Debian arch can be included in the official repos, but on the other hand as we did the same thing for the GRUB patches to support GPT on sparc64, it actually makes perfect sense to do so. What was I
    thinking... :-/

    The source code is shared among release and ports architectures.

    Adrian

    --
    .''`. John Paul Adrian Glaubitz
    : :' : Debian Developer - glaubitz@debian.org
    `. `' Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
    `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From James Clarke@21:1/5 to John Paul Adrian Glaubitz on Wed Jun 6 02:40:01 2018
    On 4 Jun 2018, at 16:02, John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> wrote:
    On 06/04/2018 04:59 PM, Frank Scheiner wrote:
    I tried to follow the guide on the Debian wiki ([1]) and the `apt[-get] build-dep klibc` command fails:


    [1]: https://wiki.debian.org/BuildingTutorial#The_packaging_workflow

    Why aren't you using sbuild? You should always build in a clean environment. Not on your normal system.


    Maybe I should use sbuild in the future.

    Indeed.

    Also, you should create an account on salsa.debian.org, fork the
    klibc repository into your own salsa home, create a branch for the
    ia64-fix, revert the patch and send a pull request to Ben.

    I wasn't sure if code parts for an unofficial Debian arch can be included in the official repos, but on the other hand as we did the same thing for the GRUB patches to support GPT on sparc64, it actually makes perfect sense to do so. What was I
    thinking... :-/

    The source code is shared among release and ports architectures.

    Is there a good reason why we're not fixing the crashing instead of
    working around it?

    James

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to James Clarke on Wed Jun 6 14:10:03 2018
    On 06/06/2018 02:33 AM, James Clarke wrote:
    Is there a good reason why we're not fixing the crashing instead of
    working around it?

    At least I am currently missing the knowledge to quickly fix the "real"
    issue - which is still to be determined. And on the other hand, if this workaround was good enough from 2008 to 2016 (see [1] and [2]), why
    invest the effort?

    [1]: https://salsa.debian.org/kernel-team/klibc/blame/e4240918f3816cb219c9d13cfe8d257b7441df8b/debian/patches/ia64-static

    [2]: https://salsa.debian.org/kernel-team/klibc/commit/07809ffd86e1e4ca5a4e98e4708a1c1fae10babf#c3d0e206bfb2cbee4f1c9fe6b8257d287bca279d

    But this does also not prevent a fix from someone that is already
    familiar with klibc - but this will still require some time I assume.

    In the meantime - with the workaround enabled - the "standard" initramfs
    as used in Debian becomes usable again on ia64 with the next release of
    klibc already.

    Cheers,
    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to John Paul Adrian Glaubitz on Wed Jun 6 14:40:02 2018
    On 06/06/2018 02:13 PM, John Paul Adrian Glaubitz wrote:
    On 06/06/2018 02:04 PM, Frank Scheiner wrote:
    In the meantime - with the workaround enabled - the "standard" initramfs as used in Debian becomes usable again on ia64 with the next release of klibc already.

    Have you submitted a pull request to the klibc package on Salsa?

    Yes, about 45 mins ago, check [1] for details.

    [1]: https://salsa.debian.org/kernel-team/klibc/merge_requests/1

    Frank

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From John Paul Adrian Glaubitz@21:1/5 to Frank Scheiner on Wed Jun 6 14:20:01 2018
    On 06/06/2018 02:04 PM, Frank Scheiner wrote:
    In the meantime - with the workaround enabled - the "standard" initramfs as used in Debian becomes usable again on ia64 with the next release of klibc already.

    Have you submitted a pull request to the klibc package on Salsa?

    Adrian

    --
    .''`. John Paul Adrian Glaubitz
    : :' : Debian Developer - glaubitz@debian.org
    `. `' Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
    `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Scheiner@21:1/5 to All on Sun Jul 15 10:50:01 2018
    Dear Adrian, Ben,

    as a follow-up to [1], I'm using klibc-utils with my patch since shortly
    before submitting the merge request ("Build static tools for ia64 again"
    ([2])) but rebuilt it yesterday with a checkout of the current state of
    the master branch of kernel-team/klibc ([3]) as source folder instead of
    the original one of the current source of the klibc-utils package.

    [1]: https://lists.debian.org/debian-ia64/2018/06/msg00000.html

    [2]: https://salsa.debian.org/kernel-team/klibc/merge_requests/1

    [3]: https://salsa.debian.org/kernel-team/klibc/tree/417eafe62fc13011bf024ded9eef1b3b16c372cd

    After installing the package (package changes further below), I can successfully use e.g. the `cat` tool of klibc-utils 2.0.4-12, which
    didn't work with 2.0.4-11 (see [1]). And of course the tools of
    klibc-utils 2.0.4-12 - specifically `ipconfig` - also work during OS
    startup.

    ```
    root@rx2800-i2:~# dpkg -i klibc-utils_2.0.4-12_ia64.deb
    (Reading database ... 82973 files and directories currently installed.) Preparing to unpack klibc-utils_2.0.4-12_ia64.deb ...
    Unpacking klibc-utils (2.0.4-12) over (2.0.4-12) ...
    Setting up klibc-utils (2.0.4-12) ...

    root@rx2800-i2:~# dpkg -l klibc-utils
    Desired=Unknown/Install/Remove/Purge/Hold
    |
    Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name Version
    Architecture Description +++-===================================-======================-======================-============================================================================
    ii klibc-utils 2.0.4-12 ia64
    small utilities built with klibc for early boot

    root@rx2800-i2:~# cd /usr/lib/klibc/bin/

    root@rx2800-i2:/usr/lib/klibc/bin# file cat
    cat: ELF 64-bit LSB executable, IA-64, version 1 (SYSV), statically
    linked, stripped

    root@rx2800-i2:/usr/lib/klibc/bin# ./cat /proc/cpuinfo
    processor : 0
    vendor : GenuineIntel
    arch : IA-64
    family : 32
    model : 2
    model name : Intel(R) Itanium(R) Processor 9320
    revision : 4
    archrev : 0
    features : branchlong, 16-byte atomic ops, 0x8
    cpu number : 0
    cpu regs : 4
    cpu MHz : 1329.945
    itc MHz : 797.967420
    BogoMIPS : 2920.44
    siblings : 8
    physical id: 0
    core id : 0
    thread id : 0
    [...]

    root@rx2800-i2:~# update-initramfs -k 4.14.52-gentoo -c
    update-initramfs: Generating /boot/initrd.img-4.14.52-gentoo

    root@rx2800-i2:~# echo $?

    0

    root@rx2800-i2:~# reboot

    Broadcast message from root@rx2800-i2 (pts/0) (Sat Jul 14 13:50:36 2018):

    The system is going down for reboot NOW!

    [...]

    Booting Core LAN Port 1
    Performing a non-directed lanboot...
    ..

    Client MAC Address: 00 23 7D 11 22 33


    LAN Configuration:
    Client MAC Address : 00 23 7D 11 22 33
    Client IP Address : 172.16.2.123
    Subnet Mask : 255.255.0.0
    Gateway IP Address : 172.16.0.1
    Boot Server IP Address : 172.16.0.2
    DHCP Server IP Address : 172.16.0.1

    Boot File Name: /AC10027B
    Boot File Size: 378664

    Optional Data: NONE

    ELILO v3.14 for EFI/IA-64
    ..
    Uncompressing Linux... done
    Loading file AC10027B.initrd.img...done
    Linux version 4.14.52-gentoo (root@rx2620) (gcc version 7.3.0 (Gentoo
    7.3.0-r3 p1.4)) #1 SMP Tue Jul 10 21:49:25 CEST 2018
    [...]
    Loading, please wait...
    starting version 238
    [...]
    Begin: Loading essential drivers ... done.
    Begin: Running /scripts/init-premount ... done.
    Begin: Mounting root file system ... Begin: Running /scripts/nfs-top ...
    done.
    Begin: RRPC: Registered named UNIX socket transport module.
    unning /scripts/RPC: Registered udp transport module.
    nfs-premount ...RPC: Registered tcp transport module.
    done.
    RPC: Registered tcp NFSv4.1 backchannel transport module.
    IP-Config: enp8s0f0 hardware address 00:23:7d:11:22:33 mtu 1500 DHCP
    IP-Config: no response after 2 secs - giving up
    IP-Config: enp8s0f0 hardware address 00:23:7d:11:22:33 mtu 1500 DHCP
    igb 0000:08:00.0 enp8s0f0: igb: enp8s0f0 NIC Link is Up 1000 Mbps Full
    Duplex, Flow Control: RX/TX
    ipconfig(1484): unaligned access to 0x60000fffffe6746f,
    ip=0x4000000000006141
    ipconfig(1484): unaligned access to 0x60000fffffe6746f,
    ip=0x4000000000006141
    IP-Config: enp8s0f0 guessed broadcast address 172.16.255.255
    IP-Config: enp8s0f0 complete (dhcp from 172.16.0.1):
    address: 172.16.2.123 broadcast: 172.16.255.255 netmask:
    255.255.0.0
    gateway: 172.16.0.1 dns0 : 172.16.0.1 dns1 : 0.0.0.0

    host : rx2800-i2
    domain : domain.tld
    rootserver: 172.16.0.2 rootpath: 172.16.0.2:/srv/nfs/rx2800-i2/root
    filename : /AC10027B
    done.
    Begin: Running /scripts/nfs-bottom ... done.
    Begin: Running /scripts/init-bottom ... done.
    INIT: version 2.88 booting
    [...]
    Debian GNU/Linux buster/sid rx2800-i2 ttyS1

    rx2800-i2 login:
    ```

    Also works with systemd:
    ```
    [...]
    systemd[1]: systemd 239 running in system mode. (+PAM +AUDIT +SELINUX
    +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS
    +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid)
    systemd[1]: Detected architecture ia64.

    Welcome to Debian GNU/Linux buster/sid!

    systemd[1]: Set hostname to <rx2800-i2>.
    [...]
    Debian GNU/Linux buster/sid rx2800-i2 ttyS1

    rx2800-i2 login:
    ```


    ****

    klibc_2.0.4-12_ia64.changes:
    ```
    Format: 1.8
    Date: Sun, 11 Mar 2018 18:10:13 +0000
    Source: klibc
    Binary: libklibc-dev libklibc klibc-utils
    Architecture: ia64
    Version: 2.0.4-12
    Distribution: UNRELEASED
    Urgency: medium
    Maintainer: maximilian attems <maks@debian.org>
    Changed-By: Ben Hutchings <ben@decadent.org.uk>
    Description:
    klibc-utils - small utilities built with klibc for early boot
    libklibc - minimal libc subset for use with initramfs
    libklibc-dev - kernel headers used during the build of klibc
    Changes:
    klibc (2.0.4-12) UNRELEASED; urgency=medium
    .
    [ Ben Hutchings ]
    * debian/control: Point Vcs URLs to Salsa
    .
    [ Frank Scheiner ]
    * [klibc] ia64: Build static tools (again)
    Checksums-Sha1:
    0f3ac7a4f580d081f903e586181c2abfe9dcbefe 178148
    klibc-utils_2.0.4-12_ia64.deb
    5a82463652ab79575ac55453e4adf37be5fdc51a 4853
    klibc_2.0.4-12_ia64.buildinfo
    c52a80d7f2eee9db254350d9b09d1e1e9c9cf7ac 154396 libklibc-dev_2.0.4-12_ia64.deb
    68b3b96b7d5919f4f36ebb0e6f6f5f03d28371b7 74680 libklibc_2.0.4-12_ia64.deb Checksums-Sha256:
    d509e14ae0cbd3adecbcb50973423f03c35d21e2f3150768e67502c2ddf96c9d
    178148 klibc-utils_2.0.4-12_ia64.deb
    7c876d5bfa09a28d6e0f85380211637c34e91a4c1b77095e8f2e28e27b1bc8d7 4853 klibc_2.0.4-12_ia64.buildinfo
    0c1f58ef38cd3d2cc269c51a958d7011fc8d97868c9daad8f6e4791323caa0b1
    154396 libklibc-dev_2.0.4-12_ia64.deb
    74209b8fca0e999824b9b4f77f6aab134537305e8f65dddb21c97a6157523054 74680 libklibc_2.0.4-12_ia64.deb
    Files:
    733c3efb105d717b75bd2dd49f63b725 178148 libs optional klibc-utils_2.0.4-12_ia64.deb
    df01a0035b1d948e569e6aec1caaacc9 4853 libs optional klibc_2.0.4-12_ia64.buildinfo
    bdc798d3b96ca0e931042c1e376f9f48 154396 libdevel optional libklibc-dev_2.0.4-12_ia64.deb
    a122d02d87aeec467f2bcf3d084ff7b6 74680 libs optional libklibc_2.0.4-12_ia64.deb
    ```

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)