• The Registers take on C4s woes

    From Tweed@21:1/5 to All on Tue Oct 19 18:32:36 2021
    https://www.theregister.com/2021/10/19/c4_subtitles/


    Confusion continues to reign in the world of television, including UK
    national broadcaster Channel 4, weeks after a broadcast centre cockup
    wrought havoc upon servers.

    Things went horribly wrong at Red Bee Media's broadcast centre back on 25 September. Yes, that was the weekend before we ran an accidentally
    appropriate episode of Who, Me?

    A fire suppression was triggered and severely damaged a lot of critical hardware. The net result was that a number of UK television channels
    (including the BBC as well as Channel 4) suffered a wobble. While others
    have recovered, Channel 4 remains unable to provide accessibility services, such as subtitles or audio description.

    A bit of background: an industry insider (who asked to remain anonymous) explained to The Register that TV companies generally provide several items
    to playout providers, usually file-based. Video and audio tend to be
    batched together in one MPEG stream. Audio Description (AD) and subtitles arrive in another. And so on.

    "In the 'olden' days of 2006, we manually got all of the different
    components and placed them into the various different output servers," explained our insider.

    The playout server described in our Who, Me? story would grab the video,
    audio, AD and subtitles – which explains its sudden shutdown in our story, that caused similar problems.

    Automation took over in the last decade, meaning that skilled intervention
    was perhaps not so essential. Up until everything went wrong.

    Red Bee posted a Twitter thread last week, apologising for the situation,
    but insisting it was "getting back on track."

    As soon as we receive the media (the programmes), our teams proceed to
    create and add pre-recorded subtitles, audio descriptions and signing. This process has been disrupted since the incident in the Broadcast Centre in
    London on 25 Sep, but we are getting back on track. (2/3)

    — Red Bee Media (@RedBeeMedia) October 15, 2021
    We don't know how far away those tracks are. Presumably quite some distance
    if, after three weeks, Red Bee Media has yet to haul itself back onto them.

    Our source noted that it was not the first time the fire alarm had gone
    off, but was the first time that nobody had managed to get to the override
    in time. The thinking behind the system was apparently "kill the flames and
    sod the consequences." As for those consequences: "Now we know... it kills servers."

    As for the hardware, hot-swap spares are likely limited and there is every possibility that parts of the kit could be quite difficult to find
    nowadays. Red Bee Media got the pictures and audio up and running again quickly, but the ancillary systems (such as the ones dealing with
    subtitles) are clearly proving problematic.

    We have asked Red Bee Media to comment but it has yet to respond. A spokesperson told the BBC:

    "Things are improving every day and we are able to deliver more and more accessible programmes, but we are unfortunately still experiencing issues
    with receiving the media for which our access teams create pre-recorded subtitles, audio descriptions and signing."

    As for Channel 4, it fired up an emergency backup but reported that the
    system supposed to provide subtitles failed. The building of an entirely
    new system is now apparently underway, according to a release emitted by
    the broadcaster.

    "Not only will this enable our channels to move back out of disaster
    recovery," it said, "but it also means we will be able to provide
    subtitles, audio description and sign language services as well."

    It's not great, and there is every chance that well over a month will have passed before full recovery. According to Channel 4, AD and sign language services were "irretrievably lost during the incident," making one wonder exactly what backup strategy was in place. We asked the broadcaster to
    explain how its backups worked (or if it was something it had outsourced)
    but have yet to receive a response.

    The ongoing woes are a salutary, and highly public, reminder to all administrators of organisations large and small. Check your backups.
    Rehearse your disaster recovery. Customers have very, very long memories

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Brian Gaff (Sofa)@21:1/5 to Tweed on Wed Oct 20 10:08:22 2021
    Yes and the really ad hoc nature of access services proves once again that disabled people are not thought of at the inception, just as the lack of AD
    on many streamed versions of films and tv show by not having them available. The films out there that are in the cinema all have captions and AD these
    days, and as long as the cinema equipment what works so does the system, as they are created by the content provider. If that is the case how come so
    many fairly modern films shown on British TV just do not have any of these services and notably the AD?

    It is not rocket science if the content has already been made is it?


    Those stations like CBS Great and Paramount channels need to get their
    fingers out of their arses and get with the programme!

    Brian


    --

    This newsgroup posting comes to you directly from...
    The Sofa of Brian Gaff...
    briang1@blueyonder.co.uk
    Blind user, so no pictures please
    Note this Signature is meaningless.!
    "Tweed" <usenet.tweed@gmail.com> wrote in message news:skn304$kkp$1@dont-email.me...
    https://www.theregister.com/2021/10/19/c4_subtitles/


    Confusion continues to reign in the world of television, including UK national broadcaster Channel 4, weeks after a broadcast centre cockup
    wrought havoc upon servers.

    Things went horribly wrong at Red Bee Media's broadcast centre back on 25 September. Yes, that was the weekend before we ran an accidentally appropriate episode of Who, Me?

    A fire suppression was triggered and severely damaged a lot of critical hardware. The net result was that a number of UK television channels (including the BBC as well as Channel 4) suffered a wobble. While others
    have recovered, Channel 4 remains unable to provide accessibility
    services,
    such as subtitles or audio description.

    A bit of background: an industry insider (who asked to remain anonymous) explained to The Register that TV companies generally provide several
    items
    to playout providers, usually file-based. Video and audio tend to be
    batched together in one MPEG stream. Audio Description (AD) and subtitles arrive in another. And so on.

    "In the 'olden' days of 2006, we manually got all of the different
    components and placed them into the various different output servers," explained our insider.

    The playout server described in our Who, Me? story would grab the video, audio, AD and subtitles - which explains its sudden shutdown in our story, that caused similar problems.

    Automation took over in the last decade, meaning that skilled intervention was perhaps not so essential. Up until everything went wrong.

    Red Bee posted a Twitter thread last week, apologising for the situation,
    but insisting it was "getting back on track."

    As soon as we receive the media (the programmes), our teams proceed to
    create and add pre-recorded subtitles, audio descriptions and signing.
    This
    process has been disrupted since the incident in the Broadcast Centre in London on 25 Sep, but we are getting back on track. (2/3)

    - Red Bee Media (@RedBeeMedia) October 15, 2021
    We don't know how far away those tracks are. Presumably quite some
    distance
    if, after three weeks, Red Bee Media has yet to haul itself back onto
    them.

    Our source noted that it was not the first time the fire alarm had gone
    off, but was the first time that nobody had managed to get to the override
    in time. The thinking behind the system was apparently "kill the flames
    and
    sod the consequences." As for those consequences: "Now we know... it kills servers."

    As for the hardware, hot-swap spares are likely limited and there is every possibility that parts of the kit could be quite difficult to find
    nowadays. Red Bee Media got the pictures and audio up and running again quickly, but the ancillary systems (such as the ones dealing with
    subtitles) are clearly proving problematic.

    We have asked Red Bee Media to comment but it has yet to respond. A spokesperson told the BBC:

    "Things are improving every day and we are able to deliver more and more accessible programmes, but we are unfortunately still experiencing issues with receiving the media for which our access teams create pre-recorded subtitles, audio descriptions and signing."

    As for Channel 4, it fired up an emergency backup but reported that the system supposed to provide subtitles failed. The building of an entirely
    new system is now apparently underway, according to a release emitted by
    the broadcaster.

    "Not only will this enable our channels to move back out of disaster recovery," it said, "but it also means we will be able to provide
    subtitles, audio description and sign language services as well."

    It's not great, and there is every chance that well over a month will have passed before full recovery. According to Channel 4, AD and sign language services were "irretrievably lost during the incident," making one wonder exactly what backup strategy was in place. We asked the broadcaster to explain how its backups worked (or if it was something it had outsourced)
    but have yet to receive a response.

    The ongoing woes are a salutary, and highly public, reminder to all administrators of organisations large and small. Check your backups.
    Rehearse your disaster recovery. Customers have very, very long memories


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Roderick Stewart@21:1/5 to usenet.tweed@gmail.com on Wed Oct 20 10:11:11 2021
    On Tue, 19 Oct 2021 18:32:36 -0000 (UTC), Tweed
    <usenet.tweed@gmail.com> wrote:

    The ongoing woes are a salutary, and highly public, reminder to all >administrators of organisations large and small. Check your backups.
    Rehearse your disaster recovery. Customers have very, very long memories

    The BBC used to have routines for this kind of thing. Once while
    exploring the rabbit warren that was Lime Grove Studios, a colleague
    and I discovered the legendary Emergency Presentation Suite. I'd
    previously read about it but never quite believed it was real, and
    here it was. According to what I'd read, it was supposed to be kept
    operational at all times in case some catastrophe should occur that
    would disable Television Centre but somehow miraculously spare Lime
    Grove. I've no idea what kind of catastrophe they had in mind, but
    they evidently took it seriously because somebody was supposed to
    switch it on and test it on the third Sunday of every month (or maybe
    it was the second Sunday) to make sure they would always have the
    ability to broadcast to the nation no matter what.

    Rod.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From J. P. Gilliver (John)@21:1/5 to briang1@blueyonder.co.uk on Wed Oct 20 13:30:23 2021
    On Wed, 20 Oct 2021 at 10:08:22, "Brian Gaff (Sofa)"
    <briang1@blueyonder.co.uk> wrote (my responses usually follow points
    raised):
    Yes and the really ad hoc nature of access services proves once again that >disabled people are not thought of at the inception, just as the lack of AD >on many streamed versions of films and tv show by not having them available. >The films out there that are in the cinema all have captions and AD these >days, and as long as the cinema equipment what works so does the system, as >they are created by the content provider. If that is the case how come so >many fairly modern films shown on British TV just do not have any of these >services and notably the AD?

    It is not rocket science if the content has already been made is it?

    I think those who think they know better than us, think we wouldn't
    accept American AD. Certainly, I don't _think_ I've ever actually heard
    any, even on US programmes - except those where the AD is clearly
    synthesized speech, which tends to be American.

    Those stations like CBS Great and Paramount channels need to get their >fingers out of their arses and get with the programme!

    Agreed. (And news channels need to think about moving the default
    subtitle position from where it was set up about 40 years ago.)

    Brian


    --
    J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

    "To YOU I'm an atheist; to God, I'm the Loyal Opposition." - Woody Allen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From J. P. Gilliver (John)@21:1/5 to rjfs@escapetime.myzen.co.uk on Wed Oct 20 13:33:19 2021
    On Wed, 20 Oct 2021 at 10:11:11, Roderick Stewart
    <rjfs@escapetime.myzen.co.uk> wrote (my responses usually follow points raised):
    []
    The BBC used to have routines for this kind of thing. Once while
    exploring the rabbit warren that was Lime Grove Studios, a colleague
    and I discovered the legendary Emergency Presentation Suite. I'd
    previously read about it but never quite believed it was real, and
    here it was. According to what I'd read, it was supposed to be kept >operational at all times in case some catastrophe should occur that

    Did you get the impression that it was indeed operational, or was it
    covered in (at least metaphorical) cobwebs?

    Was it full of - usable, but quaint - equipment, or similar to the
    then-current such suites?
    []
    (I can't help thinking of the spaceport in THHGTTG [radio version].)
    --
    J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

    "To YOU I'm an atheist; to God, I'm the Loyal Opposition." - Woody Allen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mark Carver@21:1/5 to All on Wed Oct 20 13:37:15 2021
    On 20/10/2021 13:30, J. P. Gilliver (John) wrote:


    It is not rocket science if the content has already been made is it?

    I think those who think they know better than us, think we wouldn't
    accept American AD. Certainly, I don't _think_ I've ever actually
    heard any, even on US programmes - except those where the AD is
    clearly synthesized speech, which tends to be American.

    There's another complication. Ofcom have strict rules regarding
    compliance to UK broadcasting standards for subtitles, AD etc. The
    broadcasters are not permitted to 'cut and paste' blindly pre existing
    'data' and broadcast it

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Max Demian@21:1/5 to All on Wed Oct 20 17:40:47 2021
    On 20/10/2021 10:08, Brian Gaff (Sofa) wrote:

    Yes and the really ad hoc nature of access services proves once again that disabled people are not thought of at the inception, just as the lack of AD on many streamed versions of films and tv show by not having them available. The films out there that are in the cinema all have captions and AD these days, and as long as the cinema equipment what works so does the system, as they are created by the content provider. If that is the case how come so many fairly modern films shown on British TV just do not have any of these services and notably the AD?

    It is not rocket science if the content has already been made is it?

    Why do they have to separate the AD and subtitles in any case? My PVRs
    store programmes in .ts files (there are also .hmt and .nts files - not
    sure what is in them - the former has programme descriptions). I can
    choose AD and subtitles on playback.

    --
    Max Demian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From MB@21:1/5 to All on Wed Oct 20 17:57:49 2021
    On 20/10/2021 13:33, J. P. Gilliver (John) wrote:
    The BBC used to have routines for this kind of thing. Once while
    exploring the rabbit warren that was Lime Grove Studios, a colleague
    and I discovered the legendary Emergency Presentation Suite. I'd
    previously read about it but never quite believed it was real, and
    here it was. According to what I'd read, it was supposed to be kept operational at all times in case some catastrophe should occur that

    Someone once claimed to have seen firearms in a locked cupboard on one
    site. Seems unlikely but who knows in the past when rules were much
    less restrictive.

    Though I was told of a new arrival at Penmon during the war, being a bit shocked when someone grabbed a rifle and started firing through the open
    window - he had seen a rabbit!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From charles@21:1/5 to MB@nospam.net on Wed Oct 20 18:39:13 2021
    In article <skphog$dqb$1@dont-email.me>,
    MB <MB@nospam.net> wrote:
    On 20/10/2021 13:33, J. P. Gilliver (John) wrote:
    The BBC used to have routines for this kind of thing. Once while
    exploring the rabbit warren that was Lime Grove Studios, a colleague
    and I discovered the legendary Emergency Presentation Suite. I'd
    previously read about it but never quite believed it was real, and
    here it was. According to what I'd read, it was supposed to be kept operational at all times in case some catastrophe should occur that

    Someone once claimed to have seen firearms in a locked cupboard on one
    site. Seems unlikely but who knows in the past when rules were much
    less restrictive.

    Though I was told of a new arrival at Penmon during the war, being a bit shocked when someone grabbed a rifle and started firing through the open window - he had seen a rabbit!

    Certainly things were alightly like that at Fort William: Duck, rather than rabbit and a shotgun, not a rifle.

    --
    from KT24 in Surrey, England
    "I'd rather die of exhaustion than die of boredom" Thomas Carlyle

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From J. P. Gilliver (John)@21:1/5 to charles on Wed Oct 20 18:51:11 2021
    On Wed, 20 Oct 2021 at 18:39:13, charles <charles@candehope.me.uk> wrote
    (my responses usually follow points raised):
    In article <skphog$dqb$1@dont-email.me>,
    MB <MB@nospam.net> wrote:
    On 20/10/2021 13:33, J. P. Gilliver (John) wrote:

    (Actually, I _didn't_ write this next bit. No matter though.)

    The BBC used to have routines for this kind of thing. Once while
    exploring the rabbit warren that was Lime Grove Studios, a colleague
    and I discovered the legendary Emergency Presentation Suite. I'd
    previously read about it but never quite believed it was real, and
    here it was. According to what I'd read, it was supposed to be kept
    operational at all times in case some catastrophe should occur that

    Someone once claimed to have seen firearms in a locked cupboard on one
    site. Seems unlikely but who knows in the past when rules were much
    less restrictive.

    Though I was told of a new arrival at Penmon during the war, being a bit
    shocked when someone grabbed a rifle and started firing through the open
    window - he had seen a rabbit!

    Certainly things were alightly like that at Fort William: Duck, rather than >rabbit and a shotgun, not a rifle.

    (-:
    --
    J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

    Where [other presenters] tackle the world with a box of watercolours, he
    takes a spanner. - David Butcher (on Guy Martin), RT 2015/1/31-2/6

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From MB@21:1/5 to charles on Wed Oct 20 20:07:02 2021
    On 20/10/2021 18:39, charles wrote:
    Certainly things were alightly like that at Fort William: Duck, rather than rabbit and a shotgun, not a rifle.

    Our rigger handyman came back from a trip into town one day with a
    couple of ducks or geese that he had shot down by the river.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From charles@21:1/5 to MB@nospam.net on Wed Oct 20 21:12:05 2021
    In article <skppao$4c8$1@dont-email.me>, MB <MB@nospam.net> wrote:
    On 20/10/2021 18:39, charles wrote:
    Certainly things were alightly like that at Fort William: Duck, rather
    than rabbit and a shotgun, not a rifle.

    Our rigger handyman came back from a trip into town one day with a
    couple of ducks or geese that he had shot down by the river.

    and left one on Roy's desk - so Roy told me.

    --
    from KT24 in Surrey, England
    "I'd rather die of exhaustion than die of boredom" Thomas Carlyle

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From MB@21:1/5 to charles on Wed Oct 20 22:48:14 2021
    On 20/10/2021 21:12, charles wrote:
    and left one on Roy's desk - so Roy told me.



    That's the one.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Roderick Stewart@21:1/5 to G6JPG@255soft.uk on Thu Oct 21 09:09:54 2021
    On Wed, 20 Oct 2021 13:33:19 +0100, "J. P. Gilliver (John)"
    <G6JPG@255soft.uk> wrote:

    On Wed, 20 Oct 2021 at 10:11:11, Roderick Stewart ><rjfs@escapetime.myzen.co.uk> wrote (my responses usually follow points >raised):
    []
    The BBC used to have routines for this kind of thing. Once while
    exploring the rabbit warren that was Lime Grove Studios, a colleague
    and I discovered the legendary Emergency Presentation Suite. I'd
    previously read about it but never quite believed it was real, and
    here it was. According to what I'd read, it was supposed to be kept >>operational at all times in case some catastrophe should occur that

    Did you get the impression that it was indeed operational, or was it
    covered in (at least metaphorical) cobwebs?

    Was it full of - usable, but quaint - equipment, or similar to the >then-current such suites?
    []
    (I can't help thinking of the spaceport in THHGTTG [radio version].)

    It looked a bit antiquated, like a lot of stuff at Lime Grove that was
    still being used in the 1970s, but reasonably clean, and nothing
    appeared to be missing or to have been plundered for parts. You get a
    feel for whether an installation has actually been abandoned, or is
    simply not in frequent use, and this was definitely the latter. I
    guess they were taking seriously the possibility of an emergency of
    some sort disabling Television Centre.

    Rod.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From charles@21:1/5 to Roderick Stewart on Thu Oct 21 09:30:28 2021
    In article <no62ngpv5jgepv2poba71mj8j43ujrt1ja@4ax.com>,
    Roderick Stewart <rjfs@escapetime.myzen.co.uk> wrote:
    On Wed, 20 Oct 2021 13:33:19 +0100, "J. P. Gilliver (John)" <G6JPG@255soft.uk> wrote:

    On Wed, 20 Oct 2021 at 10:11:11, Roderick Stewart ><rjfs@escapetime.myzen.co.uk> wrote (my responses usually follow points >raised):
    []
    The BBC used to have routines for this kind of thing. Once while >>exploring the rabbit warren that was Lime Grove Studios, a colleague
    and I discovered the legendary Emergency Presentation Suite. I'd >>previously read about it but never quite believed it was real, and
    here it was. According to what I'd read, it was supposed to be kept >>operational at all times in case some catastrophe should occur that

    Did you get the impression that it was indeed operational, or was it >covered in (at least metaphorical) cobwebs?

    Was it full of - usable, but quaint - equipment, or similar to the >then-current such suites?
    []
    (I can't help thinking of the spaceport in THHGTTG [radio version].)

    It looked a bit antiquated, like a lot of stuff at Lime Grove that was
    still being used in the 1970s, but reasonably clean, and nothing
    appeared to be missing or to have been plundered for parts. You get a
    feel for whether an installation has actually been abandoned, or is
    simply not in frequent use, and this was definitely the latter. I
    guess they were taking seriously the possibility of an emergency of
    some sort disabling Television Centre.

    a colleague, who was in the TA, said 2 properly placed 2lb charges of
    gelignite would be enough. Luckily he was on 'our' side.

    Rod.

    --
    from KT24 in Surrey, England
    "I'd rather die of exhaustion than die of boredom" Thomas Carlyle

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From MB@21:1/5 to charles on Thu Oct 21 12:21:48 2021
    On 21/10/2021 09:30, charles wrote:
    a colleague, who was in the TA, said 2 properly placed 2lb charges of gelignite would be enough. Luckily he was on 'our' side.

    With Red Bee, who needs gelignite? :-)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)