[continued from previous message]

(pseudo-)random numbers produced from it in future. Any kid who's played Minecraft is familiar with pseudo-randomness because Minecraft gives you the seeds it uses to generate the random worlds, so by sharing seeds you can
share worlds.

Clearly, the documentation wants us to think that, given a starting seed,
the model will always produce the same results.

Investigation reveals the truth: the code produces critically different results, even for identical starting seeds and parameters.

I'll illustrate with a few bugs. In issue 116 a UK "red team" at Edinburgh University reports that they tried to use a mode that stores data tables in
a more efficient format for faster loading, and discovered -- to their
surprise -- that the resulting predictions varied by around 80,000 deaths
after 80 days:

https://lockdownsceptics.org/wp-content/uploads/2020/05/image.png

That mode doesn't change anything about the world being simulated, so this
was obviously a bug.

The Imperial team's response is that it doesn't matter: they are "aware of
some small non-determinisms", but "this has historically been considered acceptable because of the general stochastic nature of the model". Note the phrasing here: Imperial know their code has such bugs, but act as if it's
some inherent randomness of the universe, rather than a result of amateur coding. Apparently, in epidemiology, a difference of 80,000 deaths is "a
small non-determinism".

https://github.com/mrc-ide/covid-sim/issues/116#issuecomment-617304550

Imperial advised Edinburgh that the problem goes away if you run the model
in single-threaded mode, like they do. This means they suggest using only a single CPU core rather than the many cores that any video game would successfully use. For a simulation of a country, using only a single CPU
core is obviously a dire problem -- as far from supercomputing as you can
get. Nonetheless, that's how Imperial use the code: they know it breaks when they try to run it faster. It's clear from reading the code that in 2014 Imperial tried to make the code use multiple CPUs to speed it up, but never made it work reliably. This sort of programming is known to be difficult and usually requires senior, experienced engineers to get good results. Results that randomly change from run to run are a common consequence of
thread-safety bugs. More colloquially, these are known as "Heisenbugs".

But Edinburgh came back and reported that -- even in single-threaded mode -- they still see the problem. So Imperial's understanding of the issue is
wrong. Finally, Imperial admit there's a bug by referencing a code change they've made that fixes it. The explanation given is "It looks like historically the second pair of seeds had been used at this point, to make
the runs identical regardless of how the network was made, but that this had been changed when seed-resetting was implemented". In other words, in the process of changing the model they made it non-replicable and never noticed.

Why didn't they notice? Because their code is so deeply riddled with similar bugs and they struggled so much to fix them that they got into the habit of simply averaging the results of multiple runs to cover it up... and
eventually this behaviour became normalised within the team.

In issue #30, someone reports that the model produces different outputs depending on what kind of computer it's run on (regardless of the number of CPUs). Again, the explanation is that although this new problem "will just
add to the issues" ... "This isn't a problem running the model in full as
it is stochastic anyway".

Although the academic on those threads isn't Neil Ferguson, he is well aware that the code is filled with bugs that create random results. In change #107
he authored he comments: "It includes fixes to InitModel to ensure deterministic runs with holidays enabled". In change #158 he describes the change only as "A lot of small changes, some critical to determinacy".

Imperial are trying to have their cake and eat it. Reports of random
results are dismissed with responses like "that's not a problem, just run it
a lot of times and take the average", but at the same time, they're fixing
such bugs when they find them. They know their code can't withstand
scrutiny, so they hid it until professionals had a chance to fix it, but the damage from over a decade of amateur hobby programming is so extensive that even Microsoft were unable to make it run right.

No tests. In the discussion of the fix for the first bug, Imperial state the code used to be deterministic in that place but they broke it without
noticing when changing the code.

Regressions like that are common when working on a complex piece of
software, which is why industrial software-engineering teams write automated regression tests. These are programs that run the program with varying
inputs and then check the outputs are what's expected. Every proposed change
is run against every test and if any tests fail, the change may not be made.

The Imperial code doesn't seem to have working regression tests. They tried, but the extent of the random behaviour in their code left them defeated. On
4th April they said: "However, we haven't had the time to work out a
scalable and maintainable way of running the regression test in a way that allows a small amount of variation, but doesn't let the figures drift over time."

Beyond the apparently unsalvageable nature of this specific codebase,
testing model predictions faces a fundamental problem, in that the authors don't know what the "correct" answer is until long after the fact, and by
then the code has changed again anyway, thus changing the set of bugs in
it. So it's unclear what regression tests really mean for models like this
-- even if they had some that worked.

Undocumented equations. Much of the code consists of formulas for which no purpose is given. John Carmack (a legendary video-game programmer) surmised that some of the code might have been automatically translated from FORTRAN some years ago.

For example, on line 510 of SetupModel.cpp there is a loop over all the "places" the simulation knows about. This code appears to be trying to calculate R0 for "places". Hotels are excluded during this pass, without explanation.

This bit of code highlights an issue Caswell Bligh has discussed in your
site's comments: R0 isn't a real characteristic of the virus. R0 is both an input to and an output of these models, and is routinely adjusted for
different environments and situations. Models that consume their own outputs
as inputs is problem well known to the private sector -- it can lead to
rapid divergence and incorrect prediction. There's a discussion of this
problem in section 2.2 of the Google paper, "Machine learning: the high interest credit card of technical debt".

Continuing development. Despite being aware of the severe problems in their code that they "haven't had time" to fix, the Imperial team continue to add
new features; for instance, the model attempts to simulate the impact of digital contact tracing apps.

Adding new features to a codebase with this many quality problems will just compound them and make them worse. If I saw this in a company I was
consulting for I'd immediately advise them to halt new feature development until thorough regression testing was in place and code quality had been improved.

Conclusions. All papers based on this code should be retracted immediately. Imperial's modeling efforts should be reset with a new team that isn't under Professor Ferguson, and which has a commitment to replicable results with published code from day one.

On a personal level, I'd go further and suggest that all academic
epidemiology be defunded. This sort of work is best done by the insurance sector. Insurers employ modelers and data scientists, but also employ
managers whose job is to decide whether a model is accurate enough for real world usage and professional software engineers to ensure model software is properly tested, understandable and so on. Academic efforts don't have these people, and the results speak for themselves.

My identity. Sue Denim isn't a real person (read it out). I've chosen to
remain anonymous partly because of the intense fighting that surrounds lockdown, but there's also a deeper reason. This situation has come about
due to rampant credentialism and I'm tired of it. As the widespread dismay
by programmers demonstrates, if anyone in SAGE or the Government had shown
the code to a working software engineer they happened to know, alarm bells would have been rung immediately. Instead, the Government is dominated by academics who apparently felt unable to question anything done by a fellow professor. Meanwhile, average citizens like myself are told we should never question "expertise". Although I've proven my Google employment to Toby,
this mentality is damaging and needs to end: please, evaluate the claims
I've made for yourself, or ask a programmer you know and trust to evaluate
them for you.

------------------------------

Date: Thu, 7 May 2019 19:18:07 -0800
From: "Peter G. Neumann" <neumann@csl.sri.com>
Subject: German contact-tracing app to be rolled out in mid-June (Politico)

Janosch Delcker, Politico, 7 May 2020
German contact-tracing app to be rolled out in mid-June

BERLIN -- Germany will roll out its national smartphone application tracing potential coronavirus infections in mid-June, according to high-ranking officials involved in developing it. The launch will be flanked by a broad advertising campaign aimed at convincing as many Germans as possible to use
the voluntary app, the officials said.

The underlying software, which is built by German technology companies SAP
and Deutsche Telekom, analyzes Bluetooth signals between mobile phones to
alert people who have been close enough to infect each other. It is based on what's known as a decentralized software architecture, with information
about interactions saved only on users' phones for up to three weeks -- an approach championed by both privacy advocates and U.S. tech giants Apple and Google.

After initially favouring a different architecture where information would
have been stored on a central server, German Chancellor Angela Merkel's government changed course in late April and said it would adopt such a decentralized approach, following an announcement by Apple and Google to
unlock their smartphones' Bluetooth capabilities to allow outside
developers to build interoperable apps.

The first version of the German app expected for mid-June, which will be available for both iOS and Android operating systems, will trace
interactions even while running in the background of other applications, according to the officials.

An updated version, set to be launched later this year, will also offer a voluntary option of donating the information that is collected to
Germany's national disease control center for research purposes.

------------------------------

Date: Thu, 7 May 2019 18:12:21 -0800
From: "Peter G. Neumann" <neumann@csl.sri.com>
Subject: Digital immunity passport is `the lesser of two evils' (Politico)

Vincent Manancourt, Politico, 7 May, 2020

Millennial founder of tech firm Onfido is in talks with the UK government
about system that could help ease lockdown restrictions -- but admits the
idea is controversial.

As the U.K. rolls out a coronavirus contact-tracing app, its government is already considering another technological tool to help loosen lockdown restrictions -- the immunity passport.

The idea behind so-called digital passports is that they would allow people
who have recovered from the coronavirus to signal their immunity and thus
move around freely, enabling economies to open up.

But there are fears such a system, which is at a preliminary stage of discussion with the developer, could lead to discrimination, create perverse incentives to get infected, and violate privacy.

The scheme also relies on reliable antibody testing and enough kits for large-scale testing -- neither of which exist, yet. Not to mention the fact that health experts don't know whether immunity to the coronavirus even
exists and, if it does, how long it lasts. In late April, the World Health Organisation (WHO) warned against a passport scheme on the basis that
``there is currently no evidence that people who have recovered from
Covid-19 and have antibodies are protected from a second infection.''

But governments are facing pressure to unshackle their economies, and any
ideas that allow them to do so without endangering public health are up for discussion. Immunity passports -- however pie in the sky at this stage --
fit the bill.

One firm that is proffering its expertise to help the U.K. government design such a scheme is British start-up Onfido, which last month secured $100
million in funding in part to help it develop its health certificate
offering.

The company usually helps businesses like banks and car rental firms verify identity digitally, but is now turning its tech to the fight against the
virus.

``Our approach is to bind your digital identity to your test results at the outset, and help you prove it on an ongoing basis,'' says the start-up's millennial founder Husayn Kassai on a video call with Politico.

Onfido submitted proposals to the U.K. parliament just over a week ago, and
is now in the brainstorming stage with the government, according to Kassai.

``The first area of focus for everyone is very much test kits, that comes first, and then there are a range of options that the governments and other governments want to have. So this health or immunity passport is just one of them that's being explored,'' Kassai said.

A U.K. government official told Politico that though there is interest in
the idea of introducing some form of immunity verification, there is no
formal plan yet due to the ongoing uncertainty around immunity.

Onfido's technology would work by first verifying someone's identity -- by comparing a picture or video of their face against a picture of their
identity card -- and then linking that to a coronavirus test result. People would then be able to bring up a QR code on an app or a browser signaling
their immunity status just by taking a picture of their face.

The advantage of a digital system, says Kassai, is that it is continuous and live, and can be adjusted as new evidence over immunity comes to light.

``The problem with a physical health pass is, if after six months it
transpires that [immunity] may only last for six months it's hard to go back and recall those passes. Whereas if it's digital, if in January I'm tested [and] I've proven that I've recovered from the virus and in March, it
suddenly transpires that immunity was only for three months, then every time the test result is called upon the results can suddenly be switched from
green to an amber for instance.''

If the scheme ever does see the light of day, Kassai envisages that it is
most likely that it will first be used in the workplace.

`Extremely high risks'

While the technology promises reprieve for economies hard hit by the
lockdown, it does not come without controversy.

A report last month by the AI research body the Ada Lovelace Institute said immunity passports would ``pose extremely high risks in terms of social cohesion, discrimination, exclusion and vulnerability.''

Speaking to Politico at the time, the institute's director Carly Kind said
that using such a scheme would raise difficult questions about how it is
used to allow access to spaces.

``In some way, we can imagine a world in which immunity becomes a protected characteristic like ethnicity or race, and we need to think about how to put
in place a structure to ensure that discrimination on the basis of that characteristic isn't enabled,'' Kind said.

There are also fears an immunity passport scheme could create incentives to become infected, and that it poses risks to privacy and data protection.

The European Data Protection Supervisor Wojciech Wiewi=F3rowski called the
idea extreme, and he has repeatedly expressed alarm at the idea during
online webinars. ``Even the name disgusts me a little bit,'' he said at one.

For Onfido's San Francisco-based founder, mitigating concerns is a matter of ``proceeding with caution.''

``The fact of the matter is, there's no two-tier system that would be
different to what we have already. We already have two categories of people Category A and Category B. Category A have had the virus and have recovered, Category B have not had the virus yet. So the question becomes, how are they able to signal it?'' Kassai says.

The fact that immunity passports may encourage people to become infected to
get back to work is ``very much a risk. So you're weighing two risks. And
this is a lesser of two evils, not to want to call it that. Group A and
Group B exists and that is just a fact. Some have now recovered and others
have never had the virus -- Group A are able to effectively signal that they have a period of immunity and Group B are not yet immune, is there an
incentive or risk for Group B to want to become Group A,'' he says.

As to privacy, Kassai says data in Onfido's system would be stored on ``a private server for an individual,'' [which] can only be accessed with the user's face.

``No private business, no government should really be, there's no need for
them to hold your personal data. You as a consumer should,'' he says.

Janosch Delcker and Jack Blanchard contributed reporting.

------------------------------

Date: Thu, 7 May 2020 13:41:19 -1000
From: geoff goodfellow <geoff@iconia.com>
Subject: Flu vs. COVID-19

"34,157 Americans died from influenza last year.
35,361 Americans died from COVID-19 in the last 18 days.

61,000 Americans died from the worst influenza in the last 50 years.
60,939 Americans died from COVID-19 in the last 30 days."

https://twitter.com/numbers_truth/status/1258525261672284160

------------------------------

Date: Thu, 7 May 2020 12:04:03 +0300
From: Amos Shapir <amos083@gmail.com>
Subject: Re: Visualization shows droplets from one cough on an airplane
infecting large number of passengers (RISKS-31.80)

The visualisation provided by Fox News was made for a study by Purdue University of the 2003 SARS virus. The caption says "The model is based on
the assumption that the 2003 SARS virus was airborne."

But recent studies of the current COVID-19 corona virus shows that it is not
so airborne. The WHO site states about its transmission by cough drops:
"These droplets are relatively heavy, do not travel far and quickly sink to
the ground".

https://www.who.int/emergencies/diseases/novel-coronavirus-2019/question-and-answers-hub/q-a-detail/q-a-coronaviruses

IOW, the Purdue model does not fit what is known about the newer virus.

------------------------------

Date: Thu, 07 May 2020 22:10:42 +0100
From: Chris Drewe <e767pmk@yahoo.co.uk>
Subject: Re: What the Coronavirus Crisis Reveals... (RISKS-31.79)

This reminded me of a similar example -- this may have been mentioned
before. A few winters ago in Britain, there were worries about the threat
of an outbreak of severe influenza. This was just the regular winter type, nothing like Covid-19; as I understand it, this is only a health threat to
the very young, the very old, or those with existing health problems,
otherwise it leaves people feeling terrible for a week or two but is no big deal apart from them being temporarily incapacitated. The main threat was
not streets full of dead bodies, but lots of "Tesco's truck drivers"
suddenly being off sick (Tesco being the country's major supermarket chain, though many industries would be affected).

Like many businesses, the grocery trade runs on a just-in-time basis like
the vehicle parts supplies mentioned in the RISKS article. Obviously the trucking industry has resources for normal sickness and vacation absences,
but suddenly having many drivers out of action would cause major supply problems, especially severe for groceries where many food items have short shelf lives (few days) so little opportunity for building stockpiles; not
only would supermarkets quickly run out of stock for the shelves, but food suppliers may have problems in dealing with items ready for despatching to stores. (In Europe we also have the complication of foods having 'best by', 'use before', 'sell by', etc. date markings.)

In the UK, gaining a Heavy Goods Vehicle driver's licence requires a
rigorous training and testing session (no idea what exactly it entails, but
as far as I know not a quick process, and trainers and test examiners have
to be available too), and then rookie drivers would need to familiarise themselves with the individual supermarkets' transport systems so would not
be fully effective straight away. By the time that this has happened, the original influenza outbreak would probably be over anyway.

In the end, the dreaded influenza outbreak never happened and pharmacies
were left with huge stocks of unused vaccine, but obviously something
similar could happen again. Not sure what lessons to learn; as ever, it's a balancing act between efficiency and having costly back-up resources lying
idle in reserve.

------------------------------

Date: Mon, 14 Jan 2019 11:11:11 -0800
From: RISKS-request@csl.sri.com
Subject: Abridged info on RISKS (comp.risks)

The ACM RISKS Forum is a MODERATED digest. Its Usenet manifestation is
comp.risks, the feed for which is donated by panix.com as of June 2011.

SUBSCRIPTIONS: The mailman Web interface can be used directly to

subscribe and unsubscribe:
http://mls.csl.sri.com/mailman/listinfo/risks

SUBMISSIONS: to risks@CSL.sri.com with meaningful SUBJECT: line that

includes the string `notsp'. Otherwise your message may not be read.
*** This attention-string has never changed, but might if spammers use it.

SPAM challenge-responses will not be honored. Instead, use an alternative

address from which you never send mail where the address becomes public!

The complete INFO file (submissions, default disclaimers, archive sites,

copyright policy, etc.) is online.
<http://www.CSL.sri.com/risksinfo.html>
*** Contributors are assumed to have read the full info file for guidelines!

OFFICIAL ARCHIVES: http://www.risks.org takes you to Lindsay Marshall's

searchable html archive at newcastle:
http://catless.ncl.ac.uk/Risks/VL.IS --> VoLume, ISsue.
Also, ftp://ftp.sri.com/risks for the current volume
or ftp://ftp.sri.com/VL/risks-VL.IS for previous VoLume
If none of those work for you, the most recent issue is always at
http://www.csl.sri.com/users/risko/risks.txt, and index at /risks-31.00
Lindsay has also added to the Newcastle catless site a palmtop version
of the most recent RISKS issue and a WAP version that works for many but
not all telephones: http://catless.ncl.ac.uk/w/r
ALTERNATIVE ARCHIVES: http://seclists.org/risks/ (only since mid-2001)
*** NOTE: If a cited URL fails, we do not try to update them. Try
browsing on the keywords in the subject line or cited article leads.
Apologies for what Office365 and SafeLinks may have done to URLs.

Special Offer to Join ACM for readers of the ACM RISKS Forum:

<http://www.acm.org/joinacm1>

------------------------------

End of RISKS-FORUM Digest 31.81
************************

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)