Hi,
On Tue, 07 Mar 2023 09:01:34 +0000
Michael <
confabulate@kintzios.com> wrote:
On Tuesday, 7 March 2023 07:52:01 GMT Mickaël Bucas wrote:
Le mar. 7 mars 2023 à 05:36, Bryan Gardiner <bog@khumba.net> a
écrit :
Hi folks,
How can I follow Portage's compressed build logs in real time as
they are generated?
...
tail -c +1 -f build.log.gz | od -t x1
but the following hangs with no output:
tail -c +1 -f build.log.gz | gunzip
Reading the man page, "zless" is just a wrapper around "less".
Neat, zless is a lot simpler than I thought.
So it should support the same options including typing "F" at the
end of a file to keep trying to read when the end of file is
reached.
I made a small test, but it didn't work:
# Create a growing file
$ yes | nl | gzip > zless-test.gz &
# Try to follow at the end
$ zless zless-test.gz
With ">" to go to the end and "F" to continue, I didn't get the
expected behavior, it stood still at the point I was viewing.
I don't know if it's really a bug or if I made a mistake...
(Don't forget to stop the growing file :) )
I suppose it makes sense that a simple "gunzip -c file.gz | less"
doesn't work, assuming zless is equivalent to that. gunzip sees the
end of the file and exits rather than waiting for additional content.
Then less sees its pipe close.
So I think the "tail -c +1 -f build.log.gz" bit is needed, and appears
to work since od shows data appended every few seconds.
You could try:
tail -c +1 -f build.log.gz | gunzip | less
I think it should work, but I haven't tried it.
Thanks, but I suspect that less is still at the whim of gunzip here,
and gunzip isn't producing anything.
Maybe gunzip just has a massive output buffer. If I do something a bit
more complicated:
(N=0; while true; do
(( N++ ))
echo "$N" | gzip
if (( N % 4000 == 0 )); then sleep 1; fi
done) | gunzip | od -A d -t x1
then nothing is shown for 15 seconds or so, then od quickly dumps
51952 bytes of hex output, then another long pause before dumping
another 60480 bytes of hex. If I try this with "pigz -cd" instead of
"gunzip" then output starts flowing after just ~400 bytes, but sadly
that still doesn't work for build.log.gz.
Anyway, I've had another thought, that since portage is flushing lines
to the gzipped log fairly frequently, it's probably not getting the
best compression ratio. So I went and tested recompressing some logs,
and got a decent improvement:
- nodejs-18.9.1: build.log.gz is 112KB, recompressed it's 68.9KB
(38.5% savings).
- qtwebengine-5.15.8_p20230112: build.log.gz is 1.44MB, recompressed
it's 1.01MB (29.9% savings).
- gtk+-3.24.37: build.log.gz is 157KB, recompressed it's 120KB
(23.6% savings).
So maybe I'm better off abandoning this feature, and setting something
up to compress build logs after the fact. Then good ol' tail -f will
work just fine.
Thanks, cheers,
Bryan
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEEKEeHjyeKMp3e+zLyU++8oGPmGDwFAmQJjg0ACgkQU++8oGPm GDyEDxAAmPI+MZz4+kJKy0jdwworGErhnxpui5DIeJFL0u/BhInkGH4Lntm5LEiq ORjRwQ1VeCxcAVSs2ft1tLpob5LF7NlYaCklUUQzrmHwfZGyttIDGV2CgYBXH4Sd RnXXXQJI5YgC3OWAgotp+YOlohopLJWh4sx/LI4KuTd+YpvFyf2QVvH7cHoqW+0R pkUZ12x3yNsDQnARYw83mm8sksMkbTerv3IBY0Ez9yvZsQMsefQc6g4xanaUouOW VkKehhKeg9z2lC7BDwCbAJg9iZ0L+v0YXC9FY+SHsyc8hbBt4pgomY2PTNj8fOO3 TN3ercRmdVcWCZEjqi0u6MEqs8pKPsrvALamV4Fk4bW/qjXvBSHpR7V9Kkv4gP84 OFTj3IgY9Qe211SG8wxs7+6duBqnWoNXjOLD/uQNQXwOlSu2Bk031sjovePwJrda q8zq1b1yz9fGtfZrubkWeUk8Q1cRyX+1EclEz7GH9nEKL5CUYqB36Bw+a85If41N sCEcvbBwSh8M7lVDLDTRNMYDJLP2tTjQGESWji7MSH1SiPc76exe6y5HK9sXZSRq TkSMH1STDfxrtxuZptg6/NX1mDml3RFFr4xQoj3QqtHIQcRCOUUtgP/QyrXYZcwy pcy7GQ9ih8AqupueaTIPSUMc1hUsuedBR4b9SsvfTrOEwvwyHzA=
=hffS
-----END PGP SIGNATURE-----
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)