• Profile 23.0 testing with stages and binhost (part 1 of 2)

    From Andreas K. Huettel@21:1/5 to All on Wed Feb 21 00:56:30 2024
    Hi all,

    for the following arches (and only these)

    ** GLIBC:
    ** alpha, arm, ia64, loong, m68k, ppc, ppc64, riscv, s390, sparc, x86

    ** MUSL:
    ** riscv

    the 23.0 profiles are ready for testing, including stage downloads,
    binary packages, and update instructions for existing installations.

    IMPORTANT Draft update instructions IMPORTANT https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_update_instructions

    Stage downloads (temporarily, for all listed arches):
    [preferably] https://distfiles.gentoo.org/experimental/x86/23.0_stages/ [direct/osu] https://gentoo.osuosl.org/experimental/x86/23.0_stages/

    The changes can be seen here https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_transition

    and the timeline so far here https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_timeline

    Please try things out, and file bugs if you encounter problems.

    In particular, also double-check the documentation - the update
    path table was a lot of manual work, and it's not impossible that
    C-C / C-V went wrong somwhere.

    The update instructions also double as the news item that should be
    published 2 weeks from now on these arches (unless there are
    unexpected problems). I'll reply to this e-mail with a separate
    mail containing the full text, for easier discussion.

    Note 1: The next steps are, in 2 weeks:
    * make 23.0 profiles the same stability level as 17.x profiles,
    * degrade 17.x profiles all to exp (so the CI doesn't explode)
    * publish news item
    * replace stage downloads with 23.0 version (in situ)

    Note 2: Musl is still missing out on *stable* arches, since this
    requires musl-1.2.4 stable.

    Note 3: The remaining arches follow when they are ready.
    Why are they not ready?
    * amd64: comes last :)
    * arm64: bug #916381 still needs to be implemented
    * hppa: slow hardware (one gcc build takes ~1 week)
    * mips: slow qemu emulation (one gcc build takes ~1 day),
    many variants and complex profile structure
    * musl on stable arches: needs musl-1.2.4 stable

    Cheers & have fun,
    Andreas

    --
    Andreas K. Hüttel
    dilfridge@gentoo.org
    Gentoo Linux developer
    (council, toolchain, base-system, perl, libreoffice)
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2

    iQKTBAABCgB9FiEE/Rnm0xsZLuTcY+rT3CsWIV7VQSoFAmXVPC5fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEZE MTlFNkQzMUIxOTJFRTREQzYzRUFEM0RDMkIxNjIxNUVENTQxMkEACgkQ3CsWIV7V QSq//RAAiXgqj3d9j2Cl2fnpB9/n44Q0Y0N9cKQAtzPl4FNVrg2CYmqrGYc8X2Jm pSZ/hIIOQ+NfgC2Tp/hTyZpQDetE07bVVmpVg5UplShubA46DlsnWbbUsUQXf1jh 4i2bL6Pi0snzNiidOn8t3uaJNKe8IO2W+HauGbo3ZSrMtmib+0KehLQ92OVZyRlt WNMoW/vVXgdLXYCbOMOc9H7KZ+Sm3o7u8Ixod3G3V4gPcwesEPyEDT0FdnuCQg1P +nnEG0fpajxN/EkhgXL5npMeNQ09cw1tH2qrmFCnWNs/ssKdMN2TsC42rS/WkW33 pDhkHhvOa0zOgg1nDmDdCnl85Mwq9GnCzo7co+jwxoKl1WRxgWh9ATd2wMHux1hk z/k1dk9eoQWW4TDmjlGuNbMcibvbolT+QvsssdI7vZa3/kUZUxHHCK+kbRC8M5HR EXIJcUkc49eqzFTFNS22ZVyB4rBjExzVAP+GXpKFd6LIkPkZMEvIwpBGpWCc/87S I6mSmRhw9ZgMw/DXGGMCvldx4JoLC3Y3K6rU7zYDxEHm/RVeXSCWL8HR1+fD21ae bR47Foum83wdRY9s/HSzvSi9LbKmfgShsDRgeDB/pGhb8ZiEdYrK7Xyq
  • From Andreas K. Huettel@21:1/5 to All on Wed Feb 21 00:58:27 2024
    News item / update instructions draft:


    Title: Profile upgrade to version 23.0 available
    Author: Andreas K. Huettel <dilfridge@gentoo.org>
    Posted: yyyy-mm-dd
    Revision: 1
    News-Item-Format: 2.0
    Display-If-Keyword: alpha
    Display-If-Keyword: arm
    Display-If-Keyword: ia64
    Display-If-Keyword: loong
    Display-If-Keyword: m68k
    Display-If-Keyword: ppc
    Display-If-Keyword: ppc64
    Display-If-Keyword: riscv
    Display-If-Keyword: s390
    Display-If-Keyword: sparc
    Display-If-Keyword: x86

    [*** Ignore this message for now if you are using musl.
    musl profiles and stages are not ready yet. ***]

    A profile upgrade to version 23.0 is available for your architecture.
    The new 23.0 profiles enable some toolchain hardening features and
    performance enhancements by default, and standardize settings.
    You can find the list of changes on the wiki tracking page [1].

    We strongly advise to precisely follow the upgrade instructions found
    below. The 17.0, 17.1, 20.0, and 22.0 profiles will be marked deprecated
    in 2 months and removed a year later. The exact dates depend on the architecture, see [2].

    Upgrade instructions

    Note 1: The use of binary packages is completely optional, and also not
    as much tested as the source-based upgrade path yet. If you prefer to
    only use the traditional source-based installation, omit the "--getbinpkg" parameter in all emerge invocations.

    Note 2: If you have manually changed your CHOST to a value different from
    what the stages and profiles set, you may have to do that in the future too. In that case you should know what you are doing, hopefully; please read the instructions with a critical eye then.

    1. Ensure your system backups are up to date. Please also update
    your system fully and depclean before proceeding.
    glibc older than 2.36 and musl older than 1.2.4 is not supported anymore.

    2. If you are still using one of the long-deprecated amd64 17.0 profiles
    (other than x32 or musl), then first complete the migration to the
    corresponding 17.1 profile. Instructions can be found at [3].

    3. If you are currently using systemd in a split-usr configuration, then first
    complete the migration to the corresponding merged-usr profile of the
    same profile version. Details on how to do this can be found in the news
    item [4].

    4. Run "emerge --info" and note down the value of the CHOST variable.

    5. Edit /etc/portage/make.conf; if there is a line defining the CHOST variable,
    remove it. Also delete all lines defining CHOST_... variables.

    6. Select the 23.0 profile corresponding to your current profile, either using
    "eselect profile" or by manually setting the profile symlink.
    Note that old profiles are by default split-usr and the 23.0 profiles by
    default merged-usr, i.e. example upgrades are
    default/linux/amd64/17.1 ==> default/linux/amd64/23.0/split-usr
    default/linux/amd64/17.1/systemd/merged-usr ==> default/linux/amd64/23.0/systemd
    A detailed table can be found at [5].
    In rare cases (hppa, x86) the table will tell you to pick between two choices.
    What you need should be obvious from your *OLD* CHOST value (from step 4).

    7. Delete the contents of your binary package cache at ${PKGDIR}
    rm -r /var/cache/binpkgs/*

    8. In the file or directory /etc/portage/binrepos.conf (if existing), update
    the URI in all configuration such that they point to 23.0 profile binhost
    directories. The exact paths can be found in the table at [5], too.

    9. Rebuild or reinstall from binary (if available) the following packages in
    this order, with the same version as already active:
    emerge --ask --oneshot --getbinpkg sys-devel/binutils
    (you may have to run binutils-config and re-select your binutils now)
    emerge --ask --oneshot --getbinpkg sys-devel/gcc
    (If this command fails because of mismatched "openmp" useflag requirements,
    make sure you have FEATURES=preserved-libs enabled, ignore the advice given
    by emerge, and try again with only --nodeps added to the command line.)
    (you may have to run gcc-config and re-select your gcc now)
    and the C library, i.e. for glibc-based systems
    emerge --ask --oneshot --getbinpkg sys-libs/glibc
    or for musl-based systems
    emerge --ask --oneshot --getbinpkg sys-libs/musl

    10. Re-run "emerge --info" and check if CHOST has changed compared to step 3.

    If the CHOST has NOT changed, skip to step 13 (env-update). Otherwise,

    11. Recheck with binutils-config and gcc-config that valid installed versions
    of binutils and gcc are selected.

    12. Check /etc/env.d, /etc/env.d/binutils, and /etc/env.d/gcc for files that
    refer to the *OLD* CHOST value, and remove them.
    Examples how to do this can be found in the similar procedure at [6].

    13. Run env-update && source /etc/profile

    14. Re-emerge libtool:
    emerge --ask --oneshot --getbinpkg libtool

    15. Just for safety, delete the contents of your binary package cache at
    ${PKGDIR} again:
    rm -r /var/cache/binpkgs/*

    16. Rebuild world:
    emerge --ask --emptytree --getbinpkg @world


    [1] https://wiki.g.o/wiki/Project:Toolchain/23.0_profile_transition
    [2] https://wiki.g.o/wiki/Project:Toolchain/23.0_profile_timeline
    [3] https://www.gentoo.org/support/news-items/2019-06-05-amd64-17-1-profiles-are-now-stable.html
    [4] https://www.gentoo.org/support/news-items/2022-12-01-systemd-usrmerge.html [5] https://wiki.g.o/wiki/Project:Toolchain/23.0_update_table
    [6] https://wiki.g.o/wiki/Changing_the_CHOST_variable#Verifying_things_work

    --
    Andreas K. Hüttel
    dilfridge@gentoo.org
    Gentoo Linux developer
    (council, toolchain, base-system, perl, libreoffice)
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2

    iQKTBAABCgB9FiEE/Rnm0xsZLuTcY+rT3CsWIV7VQSoFAmXVPKNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEZE MTlFNkQzMUIxOTJFRTREQzYzRUFEM0RDMkIxNjIxNUVENTQxMkEACgkQ3CsWIV7V QSpS9Q//ejrR4cD3zyRFWfR0irJYFa/GRgbdzuSv34jeSW1nc696zsgxHLFhjRtq DltZkPblQk/BQHO4jy+Tv/syDc44wMNC6Fr374GG524QFE3pm35raSrL55RsGqKs dgijPNjOj7E6BRQMb/5/R5/16+0ZQSg6ZvBwdCp009wBPTNnYF3AFPsvY3E5UA1d 3gon/IP0WcK6SJASdKoFNOBACFFAYOkz0JacP9s5FLyzrSqCSaE2eG0YnzOeU+Gm IpYvAWs+P9WvKtsAadMIcOQU6zkrTJ40vo2s9G3TLXzzjdMjvNExnx57OBw1fpYU lJInWK4Q2zq0joiVtDEG2CP7ZndtHsDPGTLg2fxnyITj3ea6GbS0/Ike97SwIHag sbixTH2CHxpyDeLLGBplITqMOO9JGoU2CspGkKQayfQferRlCV+SgCDXuTREEe1Y waKQM6lH8kd+waQq48d7N/hXWFkb2zydHZKK2Vijp+UZgzq/iMptCUzEGVjmifNL JXC9QsQL28c8JOcXkYmLoAZpVDgKmQAx9km/2EUyMCIYUWCd+klw+Pg9Rgk3Qm0i jbDphZzCgS5Vjjt6QN17wHOFVcZTD1aXD4kTQJSIxqUXpISwm408qysN