1. Forum
  2. Usenet
  3. NEWS.SOFTWARE.NNTP

  • Timing of inn 2.7?

    From Kevin Bowling@21:1/5 to All on Sun Jun 6 11:11:13 2021
    Hi,

    I'd like to put the new sqlite overview into service. Is there any
    timeline for the 2.7 release at this point?

    Regards,
    Kevin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Mon Jun 7 15:39:45 2021
    Hi Kevin,

    I'd like to put the new sqlite overview into service.  Is there any
    timeline for the 2.7 release at this point?

    We still have not discussed a timeline for the 2.7 release in the
    inn-workers mailing-list. Nonetheless, if you want to put the new
    ovsqlite storage method into service, feel free to just install a
    CURRENT snapshot from
    https://downloads.isc.org/isc/inn/snapshots/

    ovsqlite works pretty fine (I run it on my news server without any problem). The 2.7 release is no more than a CURRENT snapshot labelled 2.7.0.
    CURRENT is currently stable and can be used in production. No changes
    are scheduled as for ovsqlite code.

    --
    Julien ÉLIE

    « Et rose elle a vécu ce que vivent les roses :
    L'espace d'un matin. »
    (François de Malherbe)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Bo Lindbergh@21:1/5 to Kevin Bowling on Sat Jun 12 00:21:52 2021
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kevin Bowling@21:1/5 to Bo Lindbergh on Fri Jun 11 22:22:00 2021
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :)

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From The Doctor@21:1/5 to kevin.bowling@kev009.com on Sat Jun 12 11:43:26 2021
    In article <sa1g9p$1jum$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :)


    Can you update for daily snapshots?

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b God did not fall silent; frauds multiplied. -unknown

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From The Doctor@21:1/5 to The Doctor on Sat Jun 12 11:46:31 2021
    In article <sa26ku$inq$55@gallifrey.nk.ca>,
    The Doctor <doctor@doctor.nl2k.ab.ca> wrote:
    In article <sa1g9p$1jum$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >>> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :)


    Can you update for daily snapshots?

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT


    Please use https://archives.eyrie.org/software/inn/snapshots/

    are source of daily updates.

    Forgot about that.


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b >God did not fall silent; frauds multiplied. -unknown


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b God did not fall silent; frauds multiplied. -unknown

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kevin Bowling@21:1/5 to The Doctor on Mon Jun 21 10:05:55 2021
    On 6/12/21 4:46 AM, The Doctor wrote:
    In article <sa26ku$inq$55@gallifrey.nk.ca>,
    The Doctor <doctor@doctor.nl2k.ab.ca> wrote:
    In article <sa1g9p$1jum$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >>>> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :)


    Can you update for daily snapshots?

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT


    Please use https://archives.eyrie.org/software/inn/snapshots/

    are source of daily updates.

    Forgot about that.


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b
    God did not fall silent; frauds multiplied. -unknown



    I was asked to rename it to inn-current https://archives.eyrie.org/software/inn/snapshots/

    One outstanding issue seems to be the the snapshots are generated daily.
    That seems to cause them to flush quickly and unnecessarily. Will
    need Russ or Julien to fix that to only generate them if something changes.

    Regards,
    Kevin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From The Doctor@21:1/5 to kevin.bowling@kev009.com on Mon Jun 21 20:07:33 2021
    In article <saqgtj$1lio$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/12/21 4:46 AM, The Doctor wrote:
    In article <sa26ku$inq$55@gallifrey.nk.ca>,
    The Doctor <doctor@doctor.nl2k.ab.ca> wrote:
    In article <sa1g9p$1jum$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >>> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :)


    Can you update for daily snapshots?

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT


    Please use https://archives.eyrie.org/software/inn/snapshots/

    are source of daily updates.

    Forgot about that.


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici >doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware
    AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b
    God did not fall silent; frauds multiplied. -unknown



    I was asked to rename it to inn-current >https://archives.eyrie.org/software/inn/snapshots/

    One outstanding issue seems to be the the snapshots are generated daily.
    That seems to cause them to flush quickly and unnecessarily. Will
    need Russ or Julien to fix that to only generate them if something changes.


    Depends. I just compile daily and check for flaws myself.

    Regards,
    Kevin


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b
    A pure heart does not cause unlawful division.-unknown Beware https://mindspring.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kevin Bowling@21:1/5 to The Doctor on Mon Jun 21 23:22:43 2021
    On 6/21/21 1:07 PM, The Doctor wrote:
    In article <saqgtj$1lio$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/12/21 4:46 AM, The Doctor wrote:
    In article <sa26ku$inq$55@gallifrey.nk.ca>,
    The Doctor <doctor@doctor.nl2k.ab.ca> wrote:
    In article <sa1g9p$1jum$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >>>> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :) >>>>>

    Can you update for daily snapshots?

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT


    Please use https://archives.eyrie.org/software/inn/snapshots/

    are source of daily updates.

    Forgot about that.


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici
    doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware
    AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b
    God did not fall silent; frauds multiplied. -unknown



    I was asked to rename it to inn-current
    https://archives.eyrie.org/software/inn/snapshots/

    One outstanding issue seems to be the the snapshots are generated daily.
    That seems to cause them to flush quickly and unnecessarily. Will
    need Russ or Julien to fix that to only generate them if something changes. >>

    Depends. I just compile daily and check for flaws myself.

    That's some unnecessary compiles, check the commit logs for inn it's not
    like Linux or something that changes on a daily basis.

    If you are on FreeBSD you only need to recompile when a dep or a commit
    to inn-CURRENT itself happens to perform very thorough QA of
    inn-CURRENT. poudriere makes it easy to curate your own package set
    with all these deps handled in a holistic way, check this article out https://danschmid.de/en/blog/poudriere-guide

    I'd just like the snapshots to be generated if something has changed so
    I don't need to bump the port so often (or, if I do, it's because things
    are actually changing :)).

    Regards,
    Kevin



    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From The Doctor@21:1/5 to kevin.bowling@kev009.com on Tue Jun 22 13:37:13 2021
    In article <sarvjk$2snl$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/21/21 1:07 PM, The Doctor wrote:
    In article <saqgtj$1lio$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/12/21 4:46 AM, The Doctor wrote:
    In article <sa26ku$inq$55@gallifrey.nk.ca>,
    The Doctor <doctor@doctor.nl2k.ab.ca> wrote:
    In article <sa1g9p$1jum$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article ><s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :) >>>>>>

    Can you update for daily snapshots?

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT


    Please use https://archives.eyrie.org/software/inn/snapshots/

    are source of daily updates.

    Forgot about that.


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici
    doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware
    AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism >https://www.empire.kred/ROOTNK?t=94a1f39b
    God did not fall silent; frauds multiplied. -unknown



    I was asked to rename it to inn-current
    https://archives.eyrie.org/software/inn/snapshots/

    One outstanding issue seems to be the the snapshots are generated daily. >>> That seems to cause them to flush quickly and unnecessarily. Will
    need Russ or Julien to fix that to only generate them if something changes. >>>

    Depends. I just compile daily and check for flaws myself.

    That's some unnecessary compiles, check the commit logs for inn it's not
    like Linux or something that changes on a daily basis.

    If you are on FreeBSD you only need to recompile when a dep or a commit
    to inn-CURRENT itself happens to perform very thorough QA of
    inn-CURRENT. poudriere makes it easy to curate your own package set
    with all these deps handled in a holistic way, check this article out >https://danschmid.de/en/blog/poudriere-guide


    Hopefully there is a starter's section.

    I'd just like the snapshots to be generated if something has changed so
    I don't need to bump the port so often (or, if I do, it's because things
    are actually changing :)).


    Any diff published about inn?

    Regards,
    Kevin





    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b Changing the dictionary does not move us forward. -unknown Beware https://mindspring.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kevin Bowling@21:1/5 to The Doctor on Tue Jun 22 10:23:00 2021
    On 6/22/21 6:37 AM, The Doctor wrote:
    In article <sarvjk$2snl$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/21/21 1:07 PM, The Doctor wrote:
    In article <saqgtj$1lio$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>, >>> Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/12/21 4:46 AM, The Doctor wrote:
    In article <sa26ku$inq$55@gallifrey.nk.ca>,
    The Doctor <doctor@doctor.nl2k.ab.ca> wrote:
    In article <sa1g9p$1jum$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article
    <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    I've revived the inn-CURRENT port on FreeBSD so it is ready to use :) >>>>>>>

    Can you update for daily snapshots?

    https://cgit.freebsd.org/ports/tree/news/inn-CURRENT


    Please use https://archives.eyrie.org/software/inn/snapshots/

    are source of daily updates.

    Forgot about that.


    --
    Member - Liberal International This is doctor@@nl2k.ab.ca Ici
    doctor@@nl2k.ab.ca
    Yahweh, Queen & country!Never Satan President Republic!Beware
    AntiChrist rising!
    Look at Psalms 14 and 53 on Atheism
    https://www.empire.kred/ROOTNK?t=94a1f39b
    God did not fall silent; frauds multiplied. -unknown



    I was asked to rename it to inn-current
    https://archives.eyrie.org/software/inn/snapshots/

    One outstanding issue seems to be the the snapshots are generated daily. >>>> That seems to cause them to flush quickly and unnecessarily. Will
    need Russ or Julien to fix that to only generate them if something changes.


    Depends. I just compile daily and check for flaws myself.

    That's some unnecessary compiles, check the commit logs for inn it's not
    like Linux or something that changes on a daily basis.

    If you are on FreeBSD you only need to recompile when a dep or a commit
    to inn-CURRENT itself happens to perform very thorough QA of
    inn-CURRENT. poudriere makes it easy to curate your own package set
    with all these deps handled in a holistic way, check this article out
    https://danschmid.de/en/blog/poudriere-guide


    Hopefully there is a starter's section.

    Just take it slow, one step at a time. It's easier than it seems.

    I'd just like the snapshots to be generated if something has changed so
    I don't need to bump the port so often (or, if I do, it's because things
    are actually changing :)).


    Any diff published about inn?

    https://inn.eyrie.org/trac/log/trunk is the log for inn CURRENT branch
    (trunk).

    Regards,
    Kevin






    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kevin Bowling@21:1/5 to Bo Lindbergh on Thu Jul 1 21:41:12 2021
    On 6/11/21 3:21 PM, Bo Lindbergh wrote:
    In article <s9j341$2rvp$1@842ffb22-07e1-11e5-a459-00266cf00584.csiph.com>,
    Kevin Bowling <kevin.bowling@kev009.com> wrote:
    I'd like to put the new sqlite overview into service.

    Ah, egoboo! :-P


    /Bo Lindbergh


    Just a followup, it took several days to rebuild the overview DB (I
    think some of this is ZFS' "fault", it throws things to a condition
    variable (which enters a sleep queue) when data is not immediately
    available. makehistory could help that with a bit more file read
    parallelism in the future.

    It went from about 6GB (ZFS lz4 compression) tradoverview to 17GB with
    ZFS compression disabled and application level zlib. nnrpd performance
    is great. expireover is still taking ages, but it no longer seems to eventually block innd so that's a big improvement. I think expireover
    is another negative interaction with ZFS.

    Based on https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html#sqlite
    ...

    Some sqlite ZFS tuning:
    * Set 'recordsize' to '64k'
    * Set 'compression' to 'none'

    In ovsqlite.conf:
    * Set 'compress' to 'true'
    * Set 'pagesize' to '65536'

    I cranked the transaction time and row limit way up as well as the
    cachesize, but the ovsqlite process didn't really seem like a bottleneck
    in makehistory since it forks the writes and continues on. I'm just
    running with the default small cachesize now (so the ZFS ARC can be the
    primary freespace block cache) and it works great.

    Regards,
    Kevin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Fri Jul 2 09:11:36 2021
    Hi Kevin,

    Just a followup, it took several days to rebuild the overview DB (I
    think some of this is ZFS' "fault", it throws things to a condition
    variable (which enters a sleep queue) when data is not immediately
    available. makehistory could help that with a bit more file read
    parallelism in the future.

    It went from about 6GB (ZFS lz4 compression) tradoverview to 17GB with
    ZFS compression disabled and application level zlib.

    With ovsqlite+zlib compression, for a 1,12 GB resulting database with
    3,278,095 overview records, overview rebuild took 36 mn on my slow Intel
    Atom CPU N2800 1.86GHz with 2GB RAM running Debian.

    Strange that it took several days for you for a resulting 17 GB
    database. Lots of sleeps from ZFS then...

    expireover runs in 5:20 mn.

    --
    Julien ÉLIE

    « Un sourire coûte moins cher que l'électricité mais donne autant de
    lumière. » (Abbé Pierre)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kevin Bowling@21:1/5 to All on Fri Jul 2 12:08:44 2021
    On 7/2/21 12:11 AM, Julien ÉLIE wrote:
    Hi Kevin,

    Just a followup, it took several days to rebuild the overview DB (I
    think some of this is ZFS' "fault", it throws things to a condition
    variable (which enters a sleep queue) when data is not immediately
    available. makehistory could help that with a bit more file read
    parallelism in the future.

    It went from about 6GB (ZFS lz4 compression) tradoverview to 17GB with
    ZFS compression disabled and application level zlib.

    With ovsqlite+zlib compression, for a 1,12 GB resulting database with 3,278,095 overview records, overview rebuild took 36 mn on my slow Intel
    Atom CPU N2800 1.86GHz with 2GB RAM running Debian.

    Thanks for the numbers, it's definitely intriguing enough I will go
    looking into why ZFS struggles with this workload, it might be a regression.

    sqlite> select count(*) from artinfo;
    52397838

    Strange that it took several days for you for a resulting 17 GB
    database.  Lots of sleeps from ZFS then...

    expireover runs in 5:20 mn.

    To learn from your experience, are you running your server on flash?
    This is on a hybrid array of two mirrored HDD pairs with a SLOG (flash
    fsync fastpath) and L2ARC (flash read cache). The disks never get
    saturated but it still takes 10ms or whatever to grab a block that isn't
    in cache, and ZFS causes the process to sleep during that grab (so
    getting CPU time from the scheduler can add another millisecond or more
    delay). I did this because flash was expensive and fragile back when I assembled this version of my news server but I could probably move the
    entire spool to flash now.

    Regards,
    Kevin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Mon Jul 5 12:40:05 2021
    Hi Kevin,

    With ovsqlite+zlib compression, for a 1,12 GB resulting database with
    3,278,095 overview records, overview rebuild took 36 mn on my slow
    Intel Atom CPU N2800 1.86GHz with 2GB RAM running Debian.

    Thanks for the numbers, it's definitely intriguing enough I will go
    looking into why ZFS struggles with this workload, it might be a
    regression.

    sqlite> select count(*) from artinfo;
    52397838

    sqlite> select count(*) from artinfo;
    3426154


    Strange that it took several days for you for a resulting 17 GB
    database.  Lots of sleeps from ZFS then...

    expireover runs in 5:20 mn.

    To learn from your experience, are you running your server on flash?
    This is on a hybrid array of two mirrored HDD pairs with a SLOG (flash
    fsync fastpath) and L2ARC (flash read cache).

    I only have 1 hard drive (/dev/sda) on a dedicated server. A legacy
    rotating disk :-) not on flash.
    (I'm planning to migrate to a VPS during this summer.)

    Model Family: Hitachi/HGST Travelstar Z7K500
    Device Model: HGST HTE725050A7E630
    Serial Number: TV650BXFG0ZG6R
    User Capacity: 500,107,862,016 bytes [500 GB]
    Sector Sizes: 512 bytes logical, 4096 bytes physical
    Rotation Rate: 7200 rpm
    Form Factor: 2.5 inches

    --
    Julien ÉLIE

    « The most effective way to remember your wife's birthday is to forget
    it once… » (Nash)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)