• Re: another attempt at Y2038

    From Steve McIntyre@21:1/5 to John Paul Adrian Glaubitz on Tue Oct 18 15:50:01 2022
    XPost: linux.debian.ports.arm

    On Tue, Oct 18, 2022 at 03:33:17PM +0200, John Paul Adrian Glaubitz wrote:
    Hi Helmut!

    On 10/18/22 12:48, Helmut Grohne wrote:
    I was also wondering about this Y2038 thingy and did some experiments.
    I'm reporting what I found to document it, but don't see much actionable
    stuff here. Many thanks to Arnd Bergmann for his input.

    Attempt #1: rebootstrap

    Given that I develop rebootstrap, attempting to use it for a time64
    bootstrap seemed quite natural. I've been talking to this with Steve
    multiple times including DC22. The question was how to plug it in. In
    the end I went for Arnd's suggestion to set DPKG_*_APPEND variables to
    modify dpkg-buildflags. Not every package uses these flags, but a
    majority do. For a survey, this is probably good enough.

    I would love to do that for m68k as well. We could use this opportunity to >rebuild the m68k port with 32-bit alignment which would solve quite a number >of problems since many projects like LLVM and Qt assume a minimum alignment >of 32 bits while m68k still defaults to 16 bits.

    Since the time64 rebootstrap would break the ABI anyway, we could use to fix >alignment issue on m68k once and for all :-).

    So the reason that we're talking about doing a replacement armhf port
    is for two reasons:

    * it's probably one of the few 32-bit arches that is likely to still
    be in routine use beyond 2038 (*many* embedded users depending on
    Debian stuff)

    * it's feasible to rebootstrap and break ABI as there isn't a large
    corpus of older-ABI binaries that people will care about supporting
    in the future. (This rules out i386 - the main reason to keep it
    around is just for the older binaries AFAICS.)

    I don't want to be rude, but I really don't see how m68k fits here. Of
    course, feel free to do a rebootstrap if you like, but I genuinely
    don't see any great need for it.

    --
    Steve McIntyre, Cambridge, UK. steve@einval.com “Changing random stuff until your program works is bad coding
    practice, but if you do it fast enough it’s Machine Learning.”
    -- https://twitter.com/manisha72617183

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From John Paul Adrian Glaubitz@21:1/5 to Helmut Grohne on Tue Oct 18 15:40:01 2022
    XPost: linux.debian.ports.arm

    Hi Helmut!

    On 10/18/22 12:48, Helmut Grohne wrote:
    I was also wondering about this Y2038 thingy and did some experiments.
    I'm reporting what I found to document it, but don't see much actionable stuff here. Many thanks to Arnd Bergmann for his input.

    Attempt #1: rebootstrap

    Given that I develop rebootstrap, attempting to use it for a time64
    bootstrap seemed quite natural. I've been talking to this with Steve
    multiple times including DC22. The question was how to plug it in. In
    the end I went for Arnd's suggestion to set DPKG_*_APPEND variables to
    modify dpkg-buildflags. Not every package uses these flags, but a
    majority do. For a survey, this is probably good enough.

    I would love to do that for m68k as well. We could use this opportunity to rebuild the m68k port with 32-bit alignment which would solve quite a number
    of problems since many projects like LLVM and Qt assume a minimum alignment
    of 32 bits while m68k still defaults to 16 bits.

    Since the time64 rebootstrap would break the ABI anyway, we could use to fix alignment issue on m68k once and for all :-).

    Adrian

    --
    .''`. John Paul Adrian Glaubitz
    : :' : Debian Developer
    `. `' Physicist
    `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From John Paul Adrian Glaubitz@21:1/5 to Steve McIntyre on Tue Oct 18 16:00:01 2022
    XPost: linux.debian.ports.arm

    On 10/18/22 15:47, Steve McIntyre wrote:
    I don't want to be rude, but I really don't see how m68k fits here. Of course, feel free to do a rebootstrap if you like, but I genuinely
    don't see any great need for it.

    I don't see the need for starting another "Why does Debian Ports exist?" discussion again. If you want to discuss about what's needed, probably,
    a lot of open source projects would just have to go away. ¯\_(ツ)_/¯

    People just enjoy hacking code. There doesn't always have to have a commercial interest and a bigger reason behind it and that's perfectly fine.

    Adrian

    --
    .''`. John Paul Adrian Glaubitz
    : :' : Debian Developer
    `. `' Physicist
    `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Steve McIntyre on Tue Oct 18 16:50:01 2022
    XPost: linux.debian.ports.arm

    Hi,

    Steve McIntyre wrote:
    likely to still be in routine use beyond 2038

    Sidenote towards ISO 9660 image producers:

    Don't forget to check from time to time whether Linux removed the
    int bottleneck in fs/isofs/.
    See:
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38

    The bug report was originally about a Y2028 rollover caused by 8-bit signedness. It was closed after
    https://github.com/torvalds/linux/commit/34be4dbf87fc
    which left the int problem for future adventures.

    Currently it's still "int iso_date()" in:
    https://github.com/torvalds/linux/blob/master/fs/isofs/isofs.h line 109
    https://github.com/torvalds/linux/blob/master/fs/isofs/util.c line 19


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Karoly Balogh@21:1/5 to John Paul Adrian Glaubitz on Tue Oct 18 18:30:01 2022
    On Tue, 18 Oct 2022, John Paul Adrian Glaubitz wrote:

    Given that I develop rebootstrap, attempting to use it for a time64 bootstrap seemed quite natural. I've been talking to this with Steve multiple times including DC22. The question was how to plug it in. In
    the end I went for Arnd's suggestion to set DPKG_*_APPEND variables to modify dpkg-buildflags. Not every package uses these flags, but a
    majority do. For a survey, this is probably good enough.

    I would love to do that for m68k as well. We could use this opportunity to rebuild the m68k port with 32-bit alignment which would solve quite a number of problems since many projects like LLVM and Qt assume a minimum alignment of 32 bits while m68k still defaults to 16 bits.

    Are you talking about time64 related alignment here, or in general?

    Isn't the alignment is dictated by the 68k SysV ABI? Does this mean that
    Debian and another X 68k distro (say, embedded builds), that decides not
    to go ahead with this change not to be ABI compatible?

    Doesn't this need compiler changes as well? (Example: natural alignment of
    the stack on m68k (non-CF) is still two, enforced by the CPU behavior, and needs changes to stackframe management if you want to change this to 4).

    Charlie

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard@21:1/5 to John Paul Adrian Glaubitz on Wed Oct 19 12:10:01 2022
    On October 18, 2022 1:33:17 PM UTC, John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> wrote:

    I would love to do that for m68k as well. We could use this opportunity to >rebuild the m68k port with 32-bit alignment which would solve quite a number >of problems since many projects like LLVM and Qt assume a minimum alignment >of 32 bits while m68k still defaults to 16 bits.

    Should have probably done that long ago when the TLS thing broke the ABI.

    But.. did anyone do something similar before? I am wondering how many programs may store data on disk in binary form without any consideration for CPU alignment these days? Also, would we change the kernel and interfaces at the same time? How about boot
    programs and similar?

    Richard

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arnd Bergmann@21:1/5 to Thomas Schmitt on Thu Oct 20 11:00:01 2022
    XPost: linux.debian.ports.arm

    On Tue, Oct 18, 2022, at 16:41, Thomas Schmitt wrote:
    Hi,

    Steve McIntyre wrote:
    likely to still be in routine use beyond 2038

    Sidenote towards ISO 9660 image producers:

    Don't forget to check from time to time whether Linux removed the
    int bottleneck in fs/isofs/.
    See:
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38

    The bug report was originally about a Y2028 rollover caused by 8-bit signedness. It was closed after
    https://github.com/torvalds/linux/commit/34be4dbf87fc
    which left the int problem for future adventures.

    Currently it's still "int iso_date()" in:
    https://github.com/torvalds/linux/blob/master/fs/isofs/isofs.h line 109
    https://github.com/torvalds/linux/blob/master/fs/isofs/util.c line 19


    Have a nice day :)

    That's just a trivially fixed bug, right? I don't recall
    ever seeing a bug report or a patch for it in the past. Would
    you like to send the patch, or should I?

    Arnd

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arnd Bergmann@21:1/5 to Arnd Bergmann on Thu Oct 20 11:10:01 2022
    XPost: linux.debian.ports.arm

    On Thu, Oct 20, 2022, at 10:57, Arnd Bergmann wrote:
    On Tue, Oct 18, 2022, at 16:41, Thomas Schmitt wrote:

    Currently it's still "int iso_date()" in:
    https://github.com/torvalds/linux/blob/master/fs/isofs/isofs.h line 109
    https://github.com/torvalds/linux/blob/master/fs/isofs/util.c line 19


    Have a nice day :)

    That's just a trivially fixed bug, right? I don't recall
    ever seeing a bug report or a patch for it in the past. Would
    you like to send the patch, or should I?

    I see you actually put the patch into the debian bug
    tracker, though I can't find it on lore.kernel.org. Where
    did you submit it?

    Arnd

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Arnd Bergmann on Thu Oct 20 12:10:01 2022
    XPost: linux.debian.ports.arm

    Hi,

    Arnd Bergmann wrote:
    That's just a trivially fixed bug, right?

    Yes. Very simple. Just s/int/time64_t/ .


    I don't recall ever seeing a bug report or a patch for it in the past.

    I did not submit it to LKML or linux-scsi, because i got no reply to
    earlier attempts to bring in my userland knowledge about ISO 9660 and
    optical drives:

    https://lore.kernel.org/linux-scsi/20201120140633.1673-1-scdbackup@gmx.net/T/
    [PATCH] isofs: fix Oops with zisofs and large PAGE_SIZE

    https://lore.kernel.org/linux-scsi/20201006094026.1730-1-scdbackup@gmx.net/T/
    [PATCH v2 0/2] Fix automatic CD tray loading for data reading via sr
    [PATCH v2 1/2] cdrom: delegate automatic CD tray loading to callers of
    cdrom_open()
    [PATCH v2 2/2] sr: fix automatic tray loading for data reading

    I made more bug fixes and a wishlist patch two years ago.
    But keeping them up to date with the agile kernel development is quite a
    big task for me. (As said: userlander.)

    Especially fs/isofs has no maintainer, so i could only submit to linux-scsi because of the proximity to cdrom and sr. I had hoped that above two
    patches would be considered as modest self-introduction, but obviously my social skills are not sufficient.

    kernelnewbies could not help me either. I am still subscribed there in
    the hope to find allies. At one occasion i saw a chance when Adverg
    Ebashinskii asked for low hanging fruit in fs:
    https://www.mail-archive.com/search?q=Kernel+bug+tracker&l=kernelnewbies%40kernelnewbies.org
    I proposed the Y2038 patch:
    https://www.mail-archive.com/kernelnewbies@kernelnewbies.org/msg21629.html with the offer to reduce my "Signed-off-by:" to "Suggested-by:".

    This is what i think needed mending in kernel 5.10 two years ago:

    [PATCH 0/3] Fix the old CD read-ahead bug for media with a single TAO track
    (Most drives report the 2 unreadable TAO run-out blocks as
    part of the medium capacity.)

    [PATCH 0/1] sr: verify that last_written block is readable before deriving
    size from it
    (Size assessment of optical media in Linux is quite a mess
    and can overshoot beyond the TAO run-out problem.)

    [PATCH 0/4] Attribute size 0 to sr device if no readable medium is loaded
    (Linux reports the size of the last loaded medium, or 2048
    bytes if a blank medium is inserted.)

    [PATCH 0/4] Make mount -t iso9660 -o session=N work on DVD and BD media up
    to N=168
    (While -o sbsector= works for all media types, session= works
    only with CD media.)

    [PATCH 0/1] isofs: truncate oversized Rock Ridge names to 255 bytes
    (Truncation currently happens if name length is >= 254 and
    then cuts off much more of the name than needed.)

    When those bugs would be fixed (or mitigated in case of TAO), i hoped to
    get a favor for my own hobby:

    [PATCH 0/3] Introduce a new ioctl CDROM_SIMUL_CHANGE for burn programs
    (Currently Linux knows about new content created via ioctl
    SG_IO only after eject and reload of the Medium.)

    The housekeeping aspects of kernel development are really hard for me to master. I don't strive to become a regular contributor. But seeing those
    bugs since years causes me to mention them when there is hope to meet
    kernel developers.

    So:
    Thanks a lot for replying. Is there a chance to get you interested in the
    other bugs above and maybe even my wish for ioctl CDROM_SIMUL_CHANGE ?


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arnd Bergmann@21:1/5 to Thomas Schmitt on Thu Oct 20 16:00:01 2022
    XPost: linux.debian.ports.arm

    On Thu, Oct 20, 2022, at 12:06, Thomas Schmitt wrote:
    Arnd Bergmann wrote:

    https://lore.kernel.org/linux-scsi/20201120140633.1673-1-scdbackup@gmx.net/T/
    [PATCH] isofs: fix Oops with zisofs and large PAGE_SIZE

    https://lore.kernel.org/linux-scsi/20201006094026.1730-1-scdbackup@gmx.net/T/
    [PATCH v2 0/2] Fix automatic CD tray loading for data reading via sr
    [PATCH v2 1/2] cdrom: delegate automatic CD tray loading to callers of
    cdrom_open()
    [PATCH v2 2/2] sr: fix automatic tray loading for data reading

    I made more bug fixes and a wishlist patch two years ago.
    But keeping them up to date with the agile kernel development is quite a
    big task for me. (As said: userlander.)

    These look like you did everything right, and they should have been
    picked up by the scsi maintainers. If you ever re-send them, I would
    suggest putting the maintainers in 'To:' for the email and the mailing
    list in Cc, and asking them specifically to pick up the patches.

    Especially fs/isofs has no maintainer, so i could only submit to linux-scsi because of the proximity to cdrom and sr. I had hoped that above two
    patches would be considered as modest self-introduction, but obviously my social skills are not sufficient.

    I think the hard part here is knowing who to send the patches to.
    Unmaintained file systems are particularly tricky, in this case I
    would have used

    To: Alexander Viro <viro@zeniv.linux.org.uk>
    To: Jan Kara <jack@suse.cz>
    To: Andrew Morton <akpm@linux-foundation.org>
    Cc: Arnd Bergmann <arnd@arndb.de>
    Cc: Deepa Dinamani <deepa.kernel@gmail.com>
    Cc: linux-fsdevel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Cc: y2038@lists.linaro.org

    Can you rebase the patch on top of v6.1-rc1 and send it to
    this list of people? I'll reply with a 'Reviewed-by' tag,
    and one of the above should be able to pick it up.

    This is what i think needed mending in kernel 5.10 two years ago:

    [PATCH 0/3] Fix the old CD read-ahead bug for media with a single TAO track
    (Most drives report the 2 unreadable TAO run-out blocks as
    part of the medium capacity.)

    [PATCH 0/1] sr: verify that last_written block is readable before deriving
    size from it
    (Size assessment of optical media in Linux is quite a mess
    and can overshoot beyond the TAO run-out problem.)

    [PATCH 0/4] Attribute size 0 to sr device if no readable medium is loaded
    (Linux reports the size of the last loaded medium, or 2048
    bytes if a blank medium is inserted.)

    [PATCH 0/4] Make mount -t iso9660 -o session=N work on DVD and BD media up
    to N=168
    (While -o sbsector= works for all media types, session= works
    only with CD media.)

    [PATCH 0/1] isofs: truncate oversized Rock Ridge names to 255 bytes
    (Truncation currently happens if name length is >= 254 and
    then cuts off much more of the name than needed.)

    When those bugs would be fixed (or mitigated in case of TAO), i hoped to
    get a favor for my own hobby:

    [PATCH 0/3] Introduce a new ioctl CDROM_SIMUL_CHANGE for burn programs
    (Currently Linux knows about new content created via ioctl
    SG_IO only after eject and reload of the Medium.)

    The housekeeping aspects of kernel development are really hard for me to master. I don't strive to become a regular contributor. But seeing those
    bugs since years causes me to mention them when there is hope to meet
    kernel developers.

    So:
    Thanks a lot for replying. Is there a chance to get you interested in the other bugs above and maybe even my wish for ioctl CDROM_SIMUL_CHANGE ?

    I mainly care about the y2038 issue here, haven't done anything interesting
    on cdrom drivers in 20 years ;-)

    Arnd

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Arnd Bergmann on Thu Oct 20 16:50:01 2022
    XPost: linux.debian.ports.arm

    Hi,

    i wrote:
    https://lore.kernel.org/linux-scsi/20201120140633.1673-1-scdbackup@gmx.net/T/
    https://lore.kernel.org/linux-scsi/20201006094026.1730-1-scdbackup@gmx.net/T/

    Arnd Bergmann wrote:
    These look like you did everything right, and they should have been
    picked up by the scsi maintainers.

    I thought the same, but also that insisting, screaming, or trampling
    won't help.


    If you ever re-send them, I would
    suggest putting the maintainers in 'To:' for the email and the mailing
    list in Cc,

    I remember to have looked into the MAINTAINERS file and the git log for
    names.
    (In the case of the powerpc64 Oops, it looks like i added you in Cc:.
    If i only could remember why ...)


    I think the hard part here is knowing who to send the patches to. Unmaintained file systems are particularly tricky, in this case I
    would have used

    To: Alexander Viro <viro@zeniv.linux.org.uk>
    To: Jan Kara <jack@suse.cz>
    To: Andrew Morton <akpm@linux-foundation.org>
    Cc: Arnd Bergmann <arnd@arndb.de>
    Cc: Deepa Dinamani <deepa.kernel@gmail.com>
    Cc: linux-fsdevel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Cc: y2038@lists.linaro.org

    Well, that's quite disjoint from the list which i figured out:
    linux-kernel@vger.kernel.org
    linux-scsi@vger.kernel.org
    axboe@kernel.dk
    jejb@linux.ibm.com
    martin.petersen@oracle.com


    Can you rebase the patch on top of v6.1-rc1 and send it to
    this list of people?

    I know the word "rebase" but cannot promise that i can fill it with
    substance soon ...

    What kernel branches should i choose for sr and for isofs ?


    I mainly care about the y2038 issue here,

    If you want to do us both a favor then bring the changes from
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38
    into the kernel.
    Feel free to take my reasoning and demonstration text. The code change is trivial. So if at all, i would be fully recognized by "Suggested-by:".

    It might be worth to verify my claims:

    The only callers of iso_date() are in isofs/inode.c and isofs/rock.c
    and put the result into struct inode.i_{a,c,m}time.tv_sec which is
    of type time64_t.
    The time value of iso_date() essentially stems from mktime64().

    and to exercise the demonstration by a xorriso-made ISO.


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Arnd Bergmann@21:1/5 to Thomas Schmitt on Thu Oct 20 18:10:01 2022
    XPost: linux.debian.ports.arm

    On Thu, Oct 20, 2022, at 16:45, Thomas Schmitt wrote:
    I think the hard part here is knowing who to send the patches to.
    Unmaintained file systems are particularly tricky, in this case I
    would have used

    To: Alexander Viro <viro@zeniv.linux.org.uk>
    To: Jan Kara <jack@suse.cz>
    To: Andrew Morton <akpm@linux-foundation.org>
    Cc: Arnd Bergmann <arnd@arndb.de>
    Cc: Deepa Dinamani <deepa.kernel@gmail.com>
    Cc: linux-fsdevel@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Cc: y2038@lists.linaro.org

    Well, that's quite disjoint from the list which i figured out:
    linux-kernel@vger.kernel.org
    linux-scsi@vger.kernel.org
    axboe@kernel.dk
    jejb@linux.ibm.com
    martin.petersen@oracle.com


    This would be the correct list for the cdrom driver patches,
    my list above would be for the isofs time64 patch.

    Can you rebase the patch on top of v6.1-rc1 and send it to
    this list of people?

    I know the word "rebase" but cannot promise that i can fill it with
    substance soon ...

    What kernel branches should i choose for sr and for isofs ?

    It's generally ok to just use the latest -rc1 (right now 6.1-rc1)
    as the base, unless a maintainer asks you to use their tree
    as a base.

    I mainly care about the y2038 issue here,

    If you want to do us both a favor then bring the changes from
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38
    into the kernel.
    Feel free to take my reasoning and demonstration text. The code change is trivial.

    Done now. I've left you as author and first Signed-off-by
    though. While the change itself is trivial, the important bit
    is identifying the problem, and you did that.

    It might be worth to verify my claims:

    The only callers of iso_date() are in isofs/inode.c and isofs/rock.c
    and put the result into struct inode.i_{a,c,m}time.tv_sec which is
    of type time64_t.
    The time value of iso_date() essentially stems from mktime64().

    and to exercise the demonstration by a xorriso-made ISO.

    I could immediately tell that your patch is correct when I saw it.

    Arnd

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Arnd Bergmann on Thu Oct 20 19:10:01 2022
    XPost: linux.debian.ports.arm

    Hi,

    Arnd Bergmann wrote:
    This would be the correct list for the cdrom driver patches,
    my list above would be for the isofs time64 patch.

    Good to know. I'll keep both lists in my cheat sheet.
    (I guess that y2038@lists.linaro.org is not appropriate for non-Y2038
    patches.)


    It's generally ok to just use the latest -rc1 (right now 6.1-rc1)

    Understood. (Two minutes after i sent my latest mail i got yours.)


    I wrote:
    If you want to do us both a favor then bring the changes from
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38
    into the kernel.

    Done now.

    Thank you.

    Also thanks to the bystanders for their tolerance.
    At least my hijacking of this thread was a successful attempt at Y2038.


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Arnd Bergmann on Thu Oct 20 18:30:02 2022
    XPost: linux.debian.ports.arm

    Hi,

    Arnd Bergmann wrote:
    Can you rebase the patch on top of v6.1-rc1

    I wrote:
    What kernel branches should i choose for sr and for isofs ?

    Ok, that question was probably stupid.

    May i understand your advise as
    "on top of the newest vX.Y-rcZ in github.com/torvalds/linux"
    ?
    (Given that my rebase attempts will not happen quite now.)


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)