Is anyone running TVHeadend TV-recording software on a Raspberry Pi? If so, have you found times when you get loads of "Continuity counter errors"
during a recording?

My setup has a satellite DVB-S/USB tuner (PCTV 491e) and a dual-tuner
Hauppauge WinTV Dual. I save recordings to a USB hard drive (spinning,
rather than SSD). All these USB devices connect to separate USB ports on the
Pi - ie they don't go via a USB hub.

Normally I can record from all these tuners simultaneously, but sometimes I
get a whole run of continuity errors on all the tuners, and once it starts,
it affects all future recordings until I reboot the Pi, then I get no
further problems for several days until it starts again.

Up to now, I've blamed reception problems, but I've recently enabled TVH debugging which shows the *times* when these errors are occurring, and it
looks as if they start when more than one tuner is in use. For example. last night:

- recording on satellite (ITV HD) started at 20:00: no errors whatsoever, until...

- second and third recordings started on terrestrial tuners at 20:55
(recording at 21:00, with 5 mins pre-padding): now I see errors on all three tuners on the various video and sound streams, at a rate of about 150
(across all three tuners) per hour

I'm using Raspberry Pi OS, in a download dated 2021-01-11 (11 Jan 2021),
with "uname -a" giving "Linux [hostname] 5.4.83-v7l+ #1379 SMP Mon Dec 14 13:11:54 GMT 2020 armv7l GNU/Linux". TVH is V 4.2.8-36.

I remember having some sort of problem (I forget the details) when I
upgraded to a later kernel, so I've got a note on my RaspPiOS installation
zip "Do not upgrade to kernel 5.10" - which is why I'm still running an old version of RaspPiOS and TVH.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-02-27, NY <me@privacy.invalid> wrote:

Is anyone running TVHeadend TV-recording software on a Raspberry Pi? If so, have you found times when you get loads of "Continuity counter errors"
during a recording?

My setup has a satellite DVB-S/USB tuner (PCTV 491e) and a dual-tuner Hauppauge WinTV Dual. I save recordings to a USB hard drive (spinning,
rather than SSD). All these USB devices connect to separate USB ports on the Pi - ie they don't go via a USB hub.

Normally I can record from all these tuners simultaneously, but sometimes I get a whole run of continuity errors on all the tuners, and once it starts, it affects all future recordings until I reboot the Pi, then I get no
further problems for several days until it starts again.

Up to now, I've blamed reception problems, but I've recently enabled TVH debugging which shows the *times* when these errors are occurring, and it looks as if they start when more than one tuner is in use. For example. last night:

- recording on satellite (ITV HD) started at 20:00: no errors whatsoever, until...

- second and third recordings started on terrestrial tuners at 20:55 (recording at 21:00, with 5 mins pre-padding): now I see errors on all three tuners on the various video and sound streams, at a rate of about 150
(across all three tuners) per hour

I'm using Raspberry Pi OS, in a download dated 2021-01-11 (11 Jan 2021),
with "uname -a" giving "Linux [hostname] 5.4.83-v7l+ #1379 SMP Mon Dec 14 13:11:54 GMT 2020 armv7l GNU/Linux". TVH is V 4.2.8-36.

I remember having some sort of problem (I forget the details) when I
upgraded to a later kernel, so I've got a note on my RaspPiOS installation zip "Do not upgrade to kernel 5.10" - which is why I'm still running an old version of RaspPiOS and TVH.

For those times when the problem has started and will affect
future recordings, what is the memory consumption? Are there
stale processes left behind?

A couple of suggestions:

"free -m" can keep track of memory consumption. Storing one
result in a file shortly after reboot and then each day on the
journey toward appearance of the problem would help identify any
trends.

"ps augx" can show processes. Some processes will start shortly
after boot and then stay forever. Others will come and go. If
you put one output per day or so into a file, you can compare and
see what is going on.

HTH

--
Robert Riches
spamtrap42@jacob21819.net
(Yes, that is one of my email addresses.)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On a sunny day (Mon, 27 Feb 2023 13:44:42 -0000) it happened "NY" <me@privacy.invalid> wrote in <ttic4c$39e80$1@dont-email.me>:

Is anyone running TVHeadend TV-recording software on a Raspberry Pi? If so, >have you found times when you get loads of "Continuity counter errors"
during a recording?

My setup has a satellite DVB-S/USB tuner (PCTV 491e) and a dual-tuner >Hauppauge WinTV Dual. I save recordings to a USB hard drive (spinning,
rather than SSD). All these USB devices connect to separate USB ports on the >Pi - ie they don't go via a USB hub.

Normally I can record from all these tuners simultaneously, but sometimes I >get a whole run of continuity errors on all the tuners, and once it starts, >it affects all future recordings until I reboot the Pi, then I get no
further problems for several days until it starts again.

Up to now, I've blamed reception problems, but I've recently enabled TVH >debugging which shows the *times* when these errors are occurring, and it >looks as if they start when more than one tuner is in use. For example. last >night:

- recording on satellite (ITV HD) started at 20:00: no errors whatsoever, >until...

- second and third recordings started on terrestrial tuners at 20:55 >(recording at 21:00, with 5 mins pre-padding): now I see errors on all three >tuners on the various video and sound streams, at a rate of about 150
(across all three tuners) per hour

I'm using Raspberry Pi OS, in a download dated 2021-01-11 (11 Jan 2021),
with "uname -a" giving "Linux [hostname] 5.4.83-v7l+ #1379 SMP Mon Dec 14 >13:11:54 GMT 2020 armv7l GNU/Linux". TVH is V 4.2.8-36.

I remember having some sort of problem (I forget the details) when I
upgraded to a later kernel, so I've got a note on my RaspPiOS installation >zip "Do not upgrade to kernel 5.10" - which is why I'm still running an old >version of RaspPiOS and TVH.

Not sure if it helps in your case,
but I had a similar probem recording many security cameras (6) at the same time on my RPi 4 4 GB.

What I did and fixed it, was two things:
In /etc/rc.local
added this line:
sysctl vm.swappiness=5

and in /root/crtab added this line:
0,15,30,45 * * * * /bin/echo 3> /proc/sys/vm/drop_caches

After adding that do:
crontab crtab
to activate the new crtab.

crontab -l
should show the new crontab settings.

What it all does is clear the cache every 15 minutes to prevent swap space usage
and swap partition overflow due to too little memory left.

Since then I have loaded the system by doing many things, recording and playing at the same time
withouth it even giving a hitch in the audio playback.
Even my core i5 laptop cannot do that :-)

Maye be worth a try to play with those cache settings.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am 27.02.23 um 14:44 schrieb NY:

Is anyone running TVHeadend TV-recording software on a Raspberry Pi?
If so, have you found times when you get loads of "Continuity counter errors" during a recording?

I used to do that.
The more traffic there was on the USB bus the more errors I got.

Getting a PI4 and connecting the HDD via USB3 improved the situation
quite a bit, but the problem did never go away completely.

For earlier devices overclocking seemed to help a little bit.

In the end I gave up solved the problem by getting a used ZBOX for
recording.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 28/02/2023 03:50, Robert Riches wrote:

On 2023-02-27, NY <me@privacy.invalid> wrote:

Is anyone running TVHeadend TV-recording software on a Raspberry Pi? If so, >> have you found times when you get loads of "Continuity counter errors"
during a recording?

My setup has a satellite DVB-S/USB tuner (PCTV 491e) and a dual-tuner
Hauppauge WinTV Dual. I save recordings to a USB hard drive (spinning,
rather than SSD). All these USB devices connect to separate USB ports on the >> Pi - ie they don't go via a USB hub.

Normally I can record from all these tuners simultaneously, but sometimes I >> get a whole run of continuity errors on all the tuners, and once it starts, >> it affects all future recordings until I reboot the Pi, then I get no
further problems for several days until it starts again.

Up to now, I've blamed reception problems, but I've recently enabled TVH
debugging which shows the *times* when these errors are occurring, and it
looks as if they start when more than one tuner is in use. For example. last >> night:

- recording on satellite (ITV HD) started at 20:00: no errors whatsoever,
until...

- second and third recordings started on terrestrial tuners at 20:55
(recording at 21:00, with 5 mins pre-padding): now I see errors on all three >> tuners on the various video and sound streams, at a rate of about 150
(across all three tuners) per hour

I'm using Raspberry Pi OS, in a download dated 2021-01-11 (11 Jan 2021),
with "uname -a" giving "Linux [hostname] 5.4.83-v7l+ #1379 SMP Mon Dec 14
13:11:54 GMT 2020 armv7l GNU/Linux". TVH is V 4.2.8-36.

I remember having some sort of problem (I forget the details) when I
upgraded to a later kernel, so I've got a note on my RaspPiOS installation >> zip "Do not upgrade to kernel 5.10" - which is why I'm still running an old >> version of RaspPiOS and TVH.

For those times when the problem has started and will affect
future recordings, what is the memory consumption? Are there
stale processes left behind?

A couple of suggestions:

"free -m" can keep track of memory consumption. Storing one
result in a file shortly after reboot and then each day on the
journey toward appearance of the problem would help identify any
trends.

"ps augx" can show processes. Some processes will start shortly
after boot and then stay forever. Others will come and go. If
you put one output per day or so into a file, you can compare and
see what is going on.

Thanks. I'll keep checking "free -m". So far I've seen Free Mem reduce
from about 7000 (GB?) to 890 and then spontaneously increase when I next
looked a few hours later, but the Swap statistics remain at 99 total/free.

I'll see if I can spot any trends or memory-hogging processes in "ps".

When I next see the problem happening, the free and ps readings should
be interesting... So far, the problem has not happened since I started monitoring.

And then I can try Jan's crontab lines to clear caches.


I am considering a crontab entry to do a "shutdown -r -y" in the small
hours of each morning, if all else fails...

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 1 Mar 2023 14:47:17 +0000, NY <me@privacy.net> declaimed the
following:

Thanks. I'll keep checking "free -m". So far I've seen Free Mem reduce
from about 7000 (GB?) to 890 and then spontaneously increase when I next >looked a few hours later, but the Swap statistics remain at 99 total/free.

Probably MB -- the biggest R-Pi 4 only has 8GB. Here is a fresh boot of my 4GB R-Pi 4 (apologies about line wrap in the news client)

pi@rpi4b4gb:~$ free -m
total used free shared buff/cache
available
Mem: 3838 218 3158 34 461 3455
Swap: 99 0 99
pi@rpi4b4gb:~$

You might also be interested in the output of "top", though that is interactive and not a one-shot command.


top - 11:52:22 up 7 min, 3 users, load average: 0.01, 0.27, 0.19
Tasks: 178 total, 1 running, 177 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.2 us, 0.2 sy, 0.0 ni, 99.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0
st
MiB Mem : 3838.9 total, 3155.9 free, 218.9 used, 464.1 buff/cache
MiB Swap: 100.0 total, 100.0 free, 0.0 used. 3455.2 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
1536 pi 20 0 11348 2972 2540 R 1.0 0.1 0:00.13 top
350 root 0 -20 0 0 0 I 0.3 0.0 0:03.44 kworker/u9:1-brcmf_wq/mmc+
380 avahi 20 0 6916 2736 2476 S 0.3 0.1 0:01.17 avahi-daemon
1 root 20 0 33908 8892 7020 S 0.0 0.2 0:03.06
systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.01
kthreadd
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00
rcu_par_gp
5 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 slub_flushwq
6 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 netns
10 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 mm_percpu_wq
11 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_tasks_rude_
12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_tasks_trace
13 root 20 0 0 0 0 S 0.0 0.0 0:00.10 ksoftirqd/0
14 root 20 0 0 0 0 I 0.0 0.0 0:00.09
rcu_sched
15 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
16 root 20 0 0 0 0 S 0.0 0.0 0:00.00
cpuhp/0
17 root 20 0 0 0 0 S 0.0 0.0 0:00.00
cpuhp/1
18 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
19 root 20 0 0 0 0 S 0.0 0.0 0:00.07 ksoftirqd/1
21 root 0 -20 0 0 0 I 0.0 0.0 0:00.01 kworker/1:0H-events_highp+
22 root 20 0 0 0 0 S 0.0 0.0 0:00.00
cpuhp/2
23 root rt 0 0 0 0 S 0.0 0.0 0:00.01 migration/2
24 root 20 0 0 0 0 S 0.0 0.0 0:00.05 ksoftirqd/2


--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 01/03/2023 16:53, Dennis Lee Bieber wrote:

On Wed, 1 Mar 2023 14:47:17 +0000, NY <me@privacy.net> declaimed the following:

Thanks. I'll keep checking "free -m". So far I've seen Free Mem reduce >>from about 7000 (GB?) to 890 and then spontaneously increase when I next
looked a few hours later, but the Swap statistics remain at 99 total/free.

I'm a plonker: what I meant was 7000 MB or 7 GB, not 7000 GB. *I* knew
what I meant ;-)

Probably MB -- the biggest R-Pi 4 only has 8GB. Here is a fresh boot of my 4GB R-Pi 4 (apologies about line wrap in the news client)

pi@rpi4b4gb:~$ free -m
total used free shared buff/cache available
Mem: 3838 218 3158 34 461 3455 Swap: 99 0 99

My Pi has just started misbehaving again.

Free -m gives

total used free shared buff/cache
available
Mem: 7875 247 372 70 7255 7309 Swap: 99 0 99


Whereas previously it was around

total used free shared buff/cache
available
Mem: 7875 234 6981 70 660 7334 Swap: 99 0 99

So "free" has dropped dramatically and buff/cache has increased
dramatically.


Jan's

/bin/echo 3 > /proc/sys/vm/drop_caches

when executed manually, does restore things: "free -m" is

total used free shared buff/cache
available
Mem: 7875 247 7413 72 214 7333 Swap: 99 0 99

Am I correct that the command should have a space between what Jan gave
as "3>", so as to say "echo the digit 3 redirected into 'file' /proc/sys/vm/drop_caches"?


I will see whether the problem still happens. I'll clear TVH's count of
errors (60 continuity errors since a recording a few hours ago, before
which there were 0) and see if the count increases again in a recording
in an hour or so.


I suppose the problem could be due either to a software error or a
hardware error (in Pi or the USB tuners), and rebooting (without power
off) may clear any/all of those, so identifying which of those
mechanisms is to blame could be a problem ;-)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 01/03/2023 19:41, NY wrote:
[free -m]

I'm a plonker: what I meant was 7000 MB or 7 GB, not 7000 GB. *I* knew
what I meant ;-)

Use free -h instead, as it will then print the units it's using. You can
also use -h with other commands such as ls, du or df which will
otherwise display sizes with lots of digits.

---druck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 01/03/2023 21:18, druck wrote:

On 01/03/2023 19:41, NY wrote:
[free -m]

I'm a plonker: what I meant was 7000 MB or 7 GB, not 7000 GB. *I* knew
what I meant ;-)

Use free -h instead, as it will then print the units it's using. You can
also use -h with other commands such as ls, du or df which will
otherwise display sizes with lots of digits.

---druck

Also Linux will use any spare memory for disk caches and buffers. That
doesn't mean its 'running out'

$ free -h
total used free shared buff/cache
available
Mem: 7.7Gi 2.5Gi 1.4Gi 754Mi 3.8Gi
4.2Gi
Swap: 2.0Gi 335Mi 1.7Gi

Note that 'available' = 'free'+ 'buff/cache'

--
Religion is regarded by the common people as true, by the wise as
foolish, and by the rulers as useful.

(Seneca the Younger, 65 AD)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 27/02/2023 13:44, NY wrote:

Is anyone running TVHeadend TV-recording software on a Raspberry Pi? If
so, have you found times when you get loads of "Continuity counter
errors" during a recording?

It's early days yet, and I've only been using Jan's "echo 3 > /proc/sys/vm/drop_caches" crontab fix for a day or so, but so far it's
looking good.

I'll post again after a few more days, by which time the problem would
probably have occurred - so the longer I go without seeing any
continuity errors, the more likely it is that Jan's fix was the right one.

I'll be frustrated if what I was ascribing to reception faults turns out
to be a memory-usage problem on the Pi. I could have asked and received
the fix ages ago if I'd known...

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 02/03/2023 15:30, NY wrote:

On 27/02/2023 13:44, NY wrote:

Is anyone running TVHeadend TV-recording software on a Raspberry Pi?
If so, have you found times when you get loads of "Continuity counter
errors" during a recording?

It's early days yet, and I've only been using Jan's "echo 3 > /proc/sys/vm/drop_caches" crontab fix for a day or so, but so far it's looking good.

I'll post again after a few more days, by which time the problem would probably have occurred - so the longer I go without seeing any
continuity errors, the more likely it is that Jan's fix was the right one.

The Pi has now been running non-stop for a week and TVHeadend seems to
be behaving a lot better.

I did a "deluge test" by scheduling simultaneous recordings of multiple
SD and HD channels on each of the DVB-S2 and DVB-T2 decoders. Over an
hour's recording, I had no errors whatsoever, despite presumably very
large amount of data being written to the USB-connected spinning HDD and
a smaller amount of data being read from each of the tuners over each
one's USB.

(Do DVB tuners tend to send all the data for a mux over USB to software,
or are they told which SIDs to pick out and only send the required ones
over USB? In other words, is the amount of data that a tuner sends
dependent on how many simultaneous recordings are made on the same
tuner/mix?)

Of all the recordings I've made, defaulting to DVB-S but using DVB-T for
second and third simultaneous recordings, I think one recording had one continuity error. Interestingly, the Status page reported more errors
(maybe 10 at worst per hour) but the actual recording did not contain
any errors and nothing appeared in the diagnostic log.

Jan, I think your drop_caches fix has done the job. Many thanks. When I
do eventually reboot, the swappiness value will be changed at boot-time
and I'll see what effect (if any) that has.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

NY wrote:

Do DVB tuners tend to send all the data for a mux over USB to software,
or are they told which SIDs to pick out and only send the required ones
over USB?

USB tuners tend to send the whole mux, and let the software filter out
the unwanted PIDs, I've always been a bit wary of them for that reason
and the high interrupt rates that generates.

PCI tuners tend to have hardware PID filters (I did once buy a satellite
tuner that was essentially a USB tuner on a PCI card with a USB
interface on-board).

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)