The Post Office prosecutions scandal is the United Kingdom’s greatest mass miscarriage of justice of our times.
The scandal, however, is also difficult to write about.
Partly this is because many of the personal and systemic failures in the scandal are so maddening that any attempt at objective explanation and detached commentary can quickly become a rant.
And it is partly because the matter is so complex that very few will have mastery of all the legal and other documents and evidence. For example, the key 2019 judgment of Mr Justice Fraser – a judgment which also happens to be one of the greatest forensic exercises undertaken by any modern judge – is over a thousand paragraphs long, even without its appendices.
Nonetheless, there have been some outstanding accounts and analyses of this sorry situation. In particular, the journalist Nick Wallis has produced a book which should be read widely on the mess. There is also now a statutory inquiry which is seeking to get to the bottom of what happened, and why it happened, and how it should not happen again.
The focus of many of the accounts and much of the commentary has, rightly, been on the numerous personal and systemic failures – especially those of the Post Office management and their lawyers, and those of the software provider Fujitsu.
Those personal and systemic failures are central to what happened: none of the miscarriages of justice would have occurred without decisions by individuals (and groups of individuals) which could and should have been made differently.
And some of those decisions are such that the individuals involved should themselves be prosecuted.
But this post – and the posts which will follow this, as part of a series – is on another failure which was part of the mix.
This is the failure of the law itself and of the procedures of the courts.
And if anything, this failure of the law itself and of the procedures of the courts makes the individual decision-makers more culpable – for they knew (or should have known) how harsh the applicable law and procedure would be on the defendants, but the defendants would be prosecuted anyway.
Nothing in an account and explanation of the applicable law and procedure should be taken to limit the culpability of the Post Office management and their lawyers, and of those at the software provider Fujitsu.
*
In this first post let us start with what lawyers called a “presumption”.
The classic statement of this presumption is as follows:
“In the absence of evidence to the contrary, the courts will presume that mechanical instruments were in order at the material time”.
Here “mechanical instruments” include computers.
So, in other words, computers are presumed to be operating correctly, unless there is evidence to the contrary.
As a “presumption” this does not mean that the court will take this view each and every time, regardless of circumstance.
It is instead a starting-point which can be rebutted.
It is what the court will take to be the state of affairs, unless it is satisfied by evidence that it is not the state of affairs.
*
There is nothing inherently wrong about a court using presumptions: indeed, without presumptions, the courts could not properly operate.
Presumptions keep almost all legal cases manageable. For example, a contract will be presumed not to be a fraudulent instrument, unless it is shown to be a fake; or a defendant may be presumed not to be insane, unless shown to be insane; and so on.
Presumptions tell us what will be taken to be the state of affairs – and which party has the onus of showing whether that state of affairs is not correct.
The problems with any presumption are in what it presumes, and in what is needed to rebut it.
If the presumption is unrealistic in and of itself, or if rebuttal is unrealistic, then the presumption converts from being something that assists the course of justice to something that causes miscarriages of justice.
*
The presumption that computers are presumed to be operating correctly, unless there is evidence to the contrary is what lawyers call “a presumption of evidence”.
This means that a court can be satisfied that a relevant fact can be established just by computer records, unless there is evidence that the computer is not working properly.
And so when the computer record shows, for instance, a financial shortfall by postmaster or postmistress, the court will accept that as evidence of an actual shortfall – unless the defendant can show that the computer was not operating correctly.
In short, when the computer record is the essence of a prosecution case: computer says guilty.
*
This evidential presumption has not always been part of English law.
In 1984 a law was passed which pointed this presumption in the opposite direction.
Section 69 of the Police and Criminal Evidence Act 1984 provided:
This provision pointed the presumption in the other direction: it was for the prosecution to show that the computer was operating correctly, and not for the defendant to show that computer was not operating correctly.
This section 69 replaced the old common law position where, as stated above, in the absence of evidence to the contrary, the courts will presume that mechanical instruments were in order at the material time.
Had section 69 still been part of the law when the Post Office brought its prosecutions of post-masters and post-mistresses then the course of those cases may well have been different.
Section 69, however, was repealed in 1999:
The effect of this repeal was that the old common law presumption returned, meaning that is was again for the defendant to show that a computer was not operating correctly, rather than for the prosecutor to show that the computer was operating correctly.
How this repeal came about, and whether the reasons for that repeal were sound, will be the subject of the next post in this series.
**
Sources
Considerable assistance for this series of posts has come from the following articles:
The Law Commission presumption concerning the dependability of computer evidence (2020)
2 important tenets for computers:
Trust but verify
To err is human, to really cock things up requires a computer.
People forget that the programming is carried out by humans and these fallible creatures can and do make mistakes. When a government procurement is thrown into the mix the scope for error (and the project cost) increases exponentially.
See also the racial bias that crept in on facial recognition projects which will be the next big miscarriage of justice in due course.
The onus on the defence to prove a computer is not operating correctly puts quite a high barrier in place. Those who developed and maintain any computer system, and fixed defects that emerged during testing and operation will be in a good position to know whether the computer system operates correctly. And in fact they’ll know how many outstanding defects there are, and how many lower-severity defects they’ll have decided not to fix for commercial reasons (it’s very common in the software industry for low-severity defects not to be fixed).
So it appears to me that everyone involved with any computer system knows that it’s not working correctly, but the defendents must prove this without access to the information that the operators and developers of the system have.
That’s where the duty of disclosure comes in. One of the major issues with the prosecutions brought by the Post Office was lack of disclosure of evidence they held (I believe it was also an issue in the civil cases and they are still having problems with disclosing all relevant information to the statutory inquiry).
Although the inquiry hearings have yet to deal with the process of their prosecutions it’s pretty clear from information already available that the Post Office had a long-standing (and very possibly illegal) policy of not disclosing potentially exculpatory evidence when carrying out criminal investigations.
Indeed. I believe lack of disclosure is at the heart of this problem and likely would have undermined the presumption of mechanical soundness of the system. As another comment has highlighted, the PO was, possibly still is a highly politicised ex-civil service organisation where the technical aspects of computerised financial systems might not be fully appreciated. However that wouldn’t be true of their supplier where the coincidences of multiple allegations of fraud would have been very noticeable and actionable. The PO should rightly carry the can for these malicious prosecutions; ignorance is no defence. However, I believe their supplier has equally important questions to answer.
They may. Or the developers may have been dismissed, their list of bugs filed, and maintainers put in place to keep it working. Neither gets to talk to customers.
Proprietary, secret, computer code is neither more nor less likely of itself to be correct, or have mistakes, than software licenced and disseminated under one of the Open Source (etc) licences, or published without a licence to use it, but in the latter case nobody who is interested can be prevented from looking to see if it has a fault.
In the present state of software for large organisations that prevention is a major effort of sub-optimal managements.
What happens when AI is introduced into the mix: arguably another layer of veils mystifying the link between the human judgement of programming and its digital representation? What would constitute evidence and how would it be assessed?
Indeed, what happens if AI is introduced into the work of the courts, perhaps with the aim of assessing evidence and arguments – about the legitimacy of another AI system?
My father always said that computers were perfect – at doing what they are programmed to do.
He was in sales and during a stocktake one year there was some commotion regarding his figures. He knew his sales and margins etc and argued the company figures were flawed. “But the computer says …..” came the reply from management.
“The computer does as it’s told” he replied.
Eventually, the clerical and management teams found their (computer) mistake and apologised.
Not nearly as awful and life affecting as the post office, but a lesson to a young teenager at the time.
Thank you David for this brilliant article. It looks as if the 1984 law should never have been repealed. Just one point: you talk here about the ‘culpability of the Post Office management and their lawyers, and of those at the software provider Fujitsu.’ There’s no mention of the culpability of a large number of judges. The Father of the House, Sir Peter Bottomley, said about the PO miscarriages of justice, ‘How is that not a single judge asked “where were the fruits of SPMs embezzlement’. No grand houses, no flash cars, no huge amounts of money stashed in bank accounts or off-shore havens nor any significant investments. We tend to take for granted that judges can be relied upon to interrogate questionable assumptions made by lawyers and institutions/companies.
Justice Fraser, now newly appointed Chair of the Law Commission I believe, is an exception – a big exception to the tenet of your quite justified comment. He wrote a massive judgement after the Common Issues trial – 1000 paragraphs, 440 pages and an attempt had been made to recuse him by Lord Grabiner acting for the Post Office on the advice of a former President of the Supreme Court, Lord Neuberger.
Maybe my understanding is wrong, but I thought the problem in the Post Office case was something different? I thought it was a software issue?
If I’ve got that right, then the computer’s functioning as a mechanical instrument wasn’t the problem. The issue was between the intended (and asserted) effect of the software and it’s actual effect.
That seems to me a bit more like setting out to write a book promoting racial harmony and accidentally writing ‘Mein Kampf’. The published book would still be functioning correctly as a mechanical instrument (you could turn the pages, read the text and imbibe its meaning, stand it on your bookshelf etc.). The problem would be in the actual effect of the writing, which didn’t match the intended effect. That’d obviously be further compounded if you continued to assert that your book in fact did promote racial harmony in the face of (or oblivious to) its actual effect.
The functioning of any computer system covers the combination of both hardware and software. As far as I am aware the old “mechanical instrument” approach was never restricted to considering operation of the hardware alone but was (and is) concerned with the overall system.
Yes, I suppose the question is really what the system does when it is in order. And that is surely not merely dependent upon the assertion or intention of anyone (whether the designer/operator or not), but is a question of fact?
Is a particular computer system a device that, when in order, produces accurate financial records? Or is it a device that, when it is order, produces a string of random numbers? (or of course more insidiously, produces a string of random numbers with enough casual similarity to accurate financial records that it might easily be mistaken for them)
Surely that matter of fact is something that needs to either be agreed by the parties or decided in court?
With an actual mechanical instrument (rather than one legally defined as such!), a mutual agreement of it’s orderly function should be relatively easy. What do these scissors do when in order? What about this industrial press? When you include computer software, the function of the system when otherwise unimpeded by any malfunction may vary wildly with a single changed line of code in an update.
While the code writers may have a uniquely privileged knowledge about the *intended* orderly function, they have no inherently privileged knowledge about it’s *actual* orderly function (indeed, a software user may in many cases have much better handle on the actual function than the designer, through greater experience of its use).
Exactly. It seems neither the legal proponents of the repeal nor the Government legals, or judges seemed to understand the difference between software or hardware issues. A computer hinge failure causing an electrical fault and a blank screen seemed to be the sort of thing they were thinking about – not a computer software system working as in this case on multiple tasks with an umbrella role as an electronic (software) double entry book-keeping system.
I’m afraid I don’t agree with a need for a court to understand or even appreciate the difference between hardware and software.
A computer is just a chunk of sophisticated parts strung together in some semblance of order. It will run its own diagnostic programs (software) to ensure it’s mechanisms are functioning correctly; but beyond that it is a pile of parts and nothing else. Likewise the intended software programs are collections of coding stored on a disk somewhere until they’re loaded onto their hardware hosts, initiated and running.
When its software programs are loaded onto the hardware to perform the intended tasks such as an accounting task, the “computer system” operates as a unit. The hardware and software cannot function alone.
In terms of accountability, the onus is on the operator to assure the system is functioning correctly which it would appear in this case is the PO. If it were proven the software was at fault, that would be a matter for the PO to address with its supplier.
There has been a big shift in the practice of computer programming in the last 25 years. In the “old days”, software was delivered on floppy disks or tape, and pretty much had to be correct first time.
Nowadays, everything is on the web, “agile” means the functionality gets delivered incrementally in 2-week sprints. It is regarded as ok to leave out the difficult cases or ship bugs, provided they don’t affect too many users too much of the time. In other words, the software is never really finished.
With that in mind, it becomes simple to rebut the presumption of correctness: if the company’s tech organisation describes itself as “agile”, using “scrum” etc, its software – by design – contains bugs and omissions, both known and unknown.
Fortunately, modern development practices means evidence for this will be in the dev tickets in GitHub, Azure DevOps or whatever other systems their dev teams use.
When I worked at Bath University computer centre, we were a disaster recovery site for DVLA. This is by the by, by the way.
DVLA at the time used ICL System 4 computers. So did we. As another by the by, Fujitsu took over ICL at a later date.
Reports were coming in the driving licence renewals were somewhat randomly removing licence categories. DVLA flatly denied this, stating the computer doesn;t make mistakes.
In one case I clearly remember, a motorcycle instructor had no choice but to take his motorcycle test again, so he could continue in work He becomae a “new” rider, which impacted his insurance, but it was all he could do. DVLA were adamant, intransigent, immovable – the computer never makes mistakes. But at least he never got sent to jail.
Just to be clear, Fujitsu owned part of ICL with whom the Major Government were liaising on the IT project for the Post Office and the DSS as it was (DWP) and took it over completely by 2000 when it was rolled out into post offices across the land – if only Subpostmasters had known what doom this presaged for many of them. At around the same time I believe, although this needs checking, Subpostmasters ‘feudal’ contracts made them liable for all financial shortfalls/errors and for signing accounts.
To be clear, ICL was wholly owned by Fujitsu by the time of the Pathway contract (1996). The contract was a PFI contract from inception and was awarded by the Major government.
When the BA (DSS/DWP) later decided to pull out of the project, the Blair government decided to continue with the project – despite clear contemporaneous evidence that there were serious problems with the implementation of certain aspects of the software written for the contract.
You ought to take a look at the various hearings around NCA Operation Venetic – allegations of top-level serious organised criminals dealing in narcotics and using a difficult-to-penetrate encrypted smart phone system called EncroChat. The cases rely on messages acquired by a Dutch-French team and supplied to the NCA via Europol. The French, in operational lead, said that while they were happy to provide the message content they would not say how they achieved it, citing national defence secrecy.
The NCA and CPS, who could have decided to use the material as intelligence only, decided to tender the messages (and photos) as evidence. Two problems: is the result reliable and how does this fit with the peculiarity of English law that intercept evidence is inadmissble (s 56 IPA 2016)?
Read the judgements and look at the intellectual leaps made to admit the evidence – French witness admitted as hearsay but prosecution witnesses agreeing they don’t know how the data acquisition and subsequent processng took place.
Later computer-aided analysis of the delivered messages shows internal inconsistencies and obvious but unexplained gaps – thus indicating the Dutch-French acquisition execise was, at best, only partially successful.
Against this – many of the allegations are very obviously of wholesale narcotics trafficking and in some instances of conspiracy to murder and the obtaining of firearms. The message content is, often, “gold dust”, provided of course it is reliable and complete.
Some of the cases are ongoing and sone of the arguments quite complex – around disclosure, PII, candour in obtaining necessary warrants.
But take a look at how judges, at first instance and at appeal, made various findings of fact.
It sounds like the newer law was written by ‘law people’ or government people who didn’t really understand computers. As has been commented already, there are many more factors than the ‘computer’ which generally refers to the hardware. And what is the definition of ‘working’? Beta versions of applications can be ‘working’ in the sense of doing its main job, but that does not mean there are no bugs that can occasionally screw things up.
I think the law needs to be rewritten again (if it hasn’t already) with computer experts advising on the potential issues – e.g. where A.I. is taking us, as suggested elsewhere.
The change wasn’t about computers. It was about the need to rely on phone, tablet, laptop and computer storage as evidence in prosecutions. If the prosecution had to prove that the devices were working as intended, it would have been ‘unworkable’. Easier to put the burden on the defendant to prove otherwise.
I suspect that the repeal of Section 69, and the acceptance by the legal profession that this meant that not only hardware – which might by several stretches of the imagination almost be called mechanical – but also software would be presumed infallible was in part at least a consequence of the woeful ignorance of technology endemic in the political and legal worlds at that time. I also suspect that this position has not changed vey much.
To treat the operation of an extensive software system as being a mechanical device is as gross a category error as treating an airliner as being merely a collection of nuts and bolts. In either case we can satisfy ourselves that the elementary parts are performing within specification without having gained any confidence that the complete assemblage will function properly under all conditions to which it will be reasonably exposed; indeed we won’t even be confident that we understand what it was exposed to. Even showing that it has passed a collection of tests isn’t adequate: one needs to see what unfixed issues are being registered against it and how they are being resolved (fixed? Swept under the carpet to keep the numbers below since arbitrary threshold?) The opposing side in case hinging upon accuracy of computer records should have the right to access the issue database, and absence of a well-run system or a large number of issues in it (or evidence that issues have been “scrubbed”) should be taken as a strong argument that the computer records are unreliable.
Much could be learned by the software industry from the culture of aviation safety, where even minor incidents are followed up by detailed investigations intentionally run in a non-judgemental way and the findings distilled into public reports and recommendations. Similarly, one of the best legal developments in recent years has been the increasing tendency to publish detailed judgements.
Regarding aviation safety, a few years ago there was an excellent talk at a software conference called “Who Destroyed Three Mile Island”.
https://www.youtube.com/watch?v=1xQeXOz0Ncs
It’s about the famous nuclear disaster and how it was investigated.
The speaker does a very clever thing: in the first half of the talk, he goes through the sequence of events, in a linear fashion, tension building towards the inevitable catastrophe, and you are left thinking “how could the plant operators have been so stupid? they should have just done X, Y and Z.”.
Then he goes back over it in a second pass, revealing the thought behind the plant’s design, key background details about the staff’s training, the limited information they had access to at each moment, the constraints they were operating under, the other disasters they were trying to prevent at the same time, all under incredibly stressful conditions. And then you realize: they could hardly have done any better in the situation.
He does this to contrast two cultures of investigation. One is judgemental, looking for someone to blame, is “spoiled” by knowing the bad outcome ahead of time and interpreting every decision in that light. It assumes the system basically works, and that problems only come from “human error”, which is considered a sufficient explanation.
The other tries to understand what happened, tries to figure out how the *system as a whole* went wrong rather than individual people, not just on that day but in the months and years leading up to it. It treats human fallibility as an unavoidable fact of life, and hence “human error” is never on its own a satisfactory explanation; it can only be the beginning of an investigation, not the conclusion.
In a blame culture, people cover up their own mistakes and those of others, they don’t speak out when they see dangerous practices, close-calls go unreported, they don’t question how things are done. Everything seems fine, until one day it isn’t. Then a suitable scapegoat is found, things go on as they were, and nobody learns anything.
The airline industry were among the first to react against this culture. They realized that blaming “pilot error” for a crash just doesn’t work; you have to look at the process, the checklists, the training, the competing priorities, the user interface of the instruments, what information was available and when. The whole system, not just the person.
The book “A Field Guide to Understanding Human Error” is an excellent introduction to these principles, with lots of case studies. I strongly recommend it. The lessons (e.g. “blameless postmortems”) have slowly found their way into software development cultures, at least in good companies.
I worked in IT around this time and came across a few senior-ish ex Post Office types. I got the impression the culture was very very political and very very defensive. I suspect not a pleasant place to work.
I look forward to the upcoming episodes.
The Law Commission presumption concerning the dependability of computer evidence (2020)
In list of documents referenced this one-the law commission presumption of 2020, when I click on it comes up with the 2022 document-the legal rule. So that is listed twice. Do you have The Law Commission presumption note of 2020 to upload here?
https://committees.parliament.uk/writtenevidence/7839/pdf/
I have found Stephen Mason’s paper which must be the 2020 note referred to in reference notes at the bottom of the article here.
I am still not clear on the Law Commission’s ‘reasons’ in 1997 to repeal Section 69 of the Police and Criminal Evidence Act 1984, other than some general awareness that 1997 was the real dawn of the computer age, and it was a principal thrust of the Labour Party in its manifesto for Government therefore around the same time. Horizon was not rolled out until 2000. Prosecutions swiftly followed where shortfalls appeared on Horizon ‘accounts’ to to the bafflement of Subpostmasters from 2000 onwards. It was a useful coincidence for the prosecutors that the presumption of proof feel upon the shoulders of the defendant due to the Law Commission’s proposal and subsequent repeal in law.
I should have said : ‘fell’ upon the shoulders not ‘feel’. There is no edit button here.
The rationale was to be able to prosecute benefit fraud (etc) more easily once the evidence was all tied up in a computer system.
The computer itself may be operating correctly, but the argument that the software output is therefore unimpeachable does not follow. And in any case it’s highly questionable whether software is a “mechanical instrument” since it is just a transformation of a human-written text.
Not to mention that the output relied on by the court depends on human data input, which even the most technologically ignorant knows is prone to error.
A long time ago I worked for an organisation that boasted the biggest concentration of computer power in Europe.
Even then there was this rule, very simple and easy to understand, that simply stated: ‘Rubbish In – Rubbish Out’.
Yes. The ability to rebut a presumption of reliability depends on the candour of disclosure. My own experience as an expert is that in both civil and criminal proceedings there will be considerable resistance. It is based partly on interpretations of the formal procedural rules, partly on claims of commercial confidentiality (civil) and partly on public interest immunity (criminal). Where the computer systems are partly outside the immediate jurisdiction is another avenue for dispute. A practical problem is often inequality of arms – a big organisation versus an indvidual or small organisation – and the costs of finding some-one to frame, test and comment on a disclosure exercise. That was one of many issues facing the sub-postmasters in the Horizon cases (plus the attitude of the Post Office).
But a promising way forward and not reviving s 69 PACE is the use of compulsory questionnaires to those who wish to rely on computer-originated evidence. There are provisions under Civil Procedure Rule Practice Direction 31B (https://www.gov.uk/government/publications/form-n264-electronic-documents-questionnaire-civil-procedure-rules-practice-direction-31b). And also in the US: https://www.justice.gov/atr/questionnaire-electronically-stored-information. Don’t be put off by the lugubrious titles of these documents but see how they work.
I think that formulation of the presumption – “In the absence of evidence to the contrary, the courts will presume that mechanical instruments were in order at the material time” – comes from Castle v Cross [1984] 1 WLR 1372, an appeal by way of case stated from magistrates in Hatfield to a divisional court of the Queens Bench Division (as it then was), in drink-driving case. That case was about whether the print-out from a breath-testing machine at a police station – demonstrating that the accused failed to give enough of a sample for testing four time – was admissible in evidence to prove the offence of failing to provide a specimen (the magistrates ruled it was not, and found no case to answer).
The divisional court seems to base its decision on the maxim “omnia praesumuntur rite esse acta” (that is, the necessary legal formalities are presumed to have been complied with, absent evidence to the contrary), with a reference to the 5th edition of Cross on Evidence, which in this connection refers to stopwatches and speedometers and traffic lights.
This is just an example of a basic presumption of regularity – that, absent evidence to the contrary, simple machines of this nature are working properly. But it is some way from that presumption to an assumption that all computer systems – or indeed any particular systems – have no faults.
Particularly as the reliability of the Horizon system was a live matter in many of the cases. For example the rather appalling Castleton civil case in 2007, which depends on the evidence from Fujitsu that “there was no evidence whatsoever of any problem with the system” along with “Since Mr. Castleton accepts the accuracy of his entries in the accounts and the correctness of the arithmetic, and since the logic of the system is correct, the conclusion is inescapable that the Horizon system was working properly in all material respects, and that the shortfall […] is real, not illusory.” https://www.bailii.org/ew/cases/EWHC/QB/2007/5.html
There are some interesting parallels with the court’s approach to evidence in drink-driving cases after the adoption of the breathalyser under the Road Safety Act 1967. A ruling by Melford Stevenson (about whom much more could be said) that a breath-testing machine’s instructions needed to followed properly for the evidence of the test to be admissible was overturned by the House of Lords (!) in DPP v Carey [1970] AC 1072, with Lord Diplock saying “provided that [a police constable] acts bona fide in the conduct of the test and in the evaluation of its result, it matters not that it may subsequently be proved at the trial that the person to whom the test was administered that through inadvertence or ignorance of some relevant circumstances the instructions were not fully complied with.” (In that case, that the test results would be invalidated if the test subject had drunk alcohol in the 20 minutes before the test was administered.)
That has been followed quite recently in Scheiner v DPP [2006] https://www.bailii.org/ew/cases/EWHC/Admin/2006/1516.html – a case where the court decided to ignore the manufacturers precaution that mobile phones and radios should be switched off as they could interfere with the test results. Perhaps the courts need to take a robust approach to this sort of thing, but there are considerable risks.
Hey Andrew, you have more-or-less anticipated the content of my next few posts!
Oh sorry – I’ll just let you get on with that, and no doubt much better than I could!
I should have added that this comes from the rule against hearsay evidence, and whether the evidence is admissible at all. Perhaps we should be talking about what weight should be put upon such evidence, but even Lord Hoffman in DPP v McKeown and Jones [1997] seemed to think it would be obvious when a computer is not performing properly.
There are some interesting points to consider about who should bear any burden of proof to rebut any presumption, but ultimately the prosecution should be proving a criminal case beyond reasonable doubt, not relying on presumption of guilt.
I’ll end by mentioning the sorry tale of John Munden, a police officer convicted in 1994 of attempting to obtain money by deception after he queried some ATM withdrawals from his Halifax account when he came back from holiday in Greece (in reality, likely an ATM fraud by a third party). Although the building society was supported by a thick report from a big accounting firm on the robustness of their systems, his appeal was eventually allowed several years later, after (I understand) the building society would not comply with a court order to allow his computer expert to examine their computers.
Interesting re John Munden the Police Officer (awful) and the drink driving issue – failing to provide a proper breath test means not blowing into the breathalyser properly so a reading can not be obtained. Most people who do that and are charged would get a fine and a long driving suspension not get off lightly.
Anyway – what I really wanted to thow into the mix was the impact of all this on human beings -this foolish indefensible presumption on the defendant to prove that a computer was not working properly. How would he/she know? In the Castleton case, his belief was that his shortfalls were caused by software flaws. He could provide no KEL’s (Known Error Logs) because they belonged to Fujitsu. He had no access to them. The Post Office pretended they did not know if KELs actually existed or not, and anyway, they maintained no one could access the Subpostmaster terminal remotely to investigate, do a fix or anything. In solicitor Stephen Dilley’s evidence last week, (Bond Pearce, now Wombles Bond Dickinson) the data on numbers of Subposmasters complaining about Horizon errors was claimed to be too costly to provide . This ‘other’ data denied was the noise from 15000 calls received by help desk from worried subpostmasters about Horizon , information that Dilley and Talbot knew about but which was hidden from Castleton. (Excuse was that as Dilley said: “It would have cost £3000″) This money, as the enquiry was told was too much. But compare it to the £300,000 it cost POL to conduct the case to make Castleton an example to others –
and put the fear of God in Subposmasters who wished to take on the Post Office over computer ‘robustness’, as they called it. This was £300,000 plus fees paid to Wombles Bond Dickinson and to the KC for the Post Office (what one might call a Presumption of a Large Payment evidenced by loads of money going into lawyers bank accounts). This too was the ‘charge’ placed on Castleton when he lost the case and it bankrupted him. In 2014 I think it was, Dilley was seconded to the office of litigation within POL with Mandy Talbot, a solicitor who worked for POL and gave as little help to the Inquiry as she possibly could last week. It was Mandy Talbot who wrote in an email to colleagues that it was irrelevant if Lee Castleton’s children would not have a roof over their heads, and delighted in the ‘good news’ of bankrupting him. Following this secondment, Dilley returned to Bond Pearce (now Wombles Bond Dickinson) and in 2016 penned a published paper about his legal advice to companies that include such things as ” it might be better not to write things down”. One can certainly understand why provision of information to a defendant, which is supposed to happen in a court of law was already curtailed deliberately in POL prosecutions. As far as Help Desk data was concerned, this was held back and made worse by the Presumption of computers being reliable. This s 69 repeal benefited the prosecution by 100 per cent and the defendant, not at all. To have judges unable to understand what computers going wrong actually meant in practice was an additional insult to the appalling harm done to innocent families. (lee has a son and a daughter and the impact statemnt on the daughter is unbrearable to read. The invisible worms of wrongly inputted code or glitches hidden from view but known to some at Fujitsu, meant as little to Lee Castleton’s Judge Havery as integrity, honesty and compassion meant to Mandy Talbot, the Post Office Board and anyone else at POL involved in this appalling saga.
Is the source code (the language rather like English describing instructions) of a computer program a document?
If so, why is it not disclosed?
“Source code” is generally an unintelligible listing of alphanumeric characters and symbols which if published for disclosure would be meaningless.
The evidence required would generally be that of how it performs within the system for which it’s designed. This would include outputs, reports, error logs etc.
“Source code” is the written instructions that a computer executes, albeit by way of a translation into the binary code actually executed.
It is intelligible to computer scientists and perhaps partly so by non-professionals who are mathematically literate.
The evidence would need to be a combination of the source code, plus proof of test coverage (itself never, ever 100%), test results, reported errors and corrections mitigations in place.
Proper expert witnesses know the above. The courts should too.
file:///C:/Users/Rosie/OneDrive/User/Downloads/DEESLR_2023_Vol20pp62-95_Advanced_Article6_christie%20(1).pdf
There is also this 2023 article by James Christie of great interest about the repeal.
NOTE TO DAVIDALLENGREEN
Have you noted that two of the reference docs below your peie are the same docs of June 2022. I have inserted in one of my comments here the correct 2020 version that you have as the title for the first reference.
Thank you, I shall look.
Barrister Paul Marshall who represented several Postmasters in the GLO under Justice Fraser, had this (below) to say on Linked In, following the report to the Horizon Post Office Inquiry under Sir Wyn Williams, and the report given by Duncan Atkinson KC on the seeming malfeasance of the PO investigators from day one in 2000, breaking the law, refusing disclosure and aided and abetted by lawyers.
‘THE SPREADING STAIN OF THE POST OFFICE ON ENGLISH JUSTICE’
‘⚖️The most intrusive power of the state over its citizens, absent war, is to imprison them.’
⚖️’In 2020 I was invited to give evidence to the Justice Committee. I suggested there was a systemic problem with private prosecutions by the Post Office. My evidence was rejected.’
⚖️Duncan Atkinson K.C., a criminal barrister, recently gave evidence to the Post Office IT Inquiry that basic procedural safeguards imposed upon the Crown Prosecution Service, as the state prosecuting authority, simply didn’t exist for the Post Office between 1999-2014 when it prosecuted hundreds of postmasters for offences of alleged dishonesty, resulting in the most widespread miscarriage of justice in English legal history.
⚖️The Post Office had no policy for “third party disclosure”; it had no formalised procedures for considering lines of inquiry that led away from, rather than toward the suspect; it conflated the roles of investigator and prosecutor – ordinarily subject to separate procedural safeguards. It was also ‘victim’.
⚖️There were other failings. In 2019 the Post Office told the High Court judge that the “Horizon” computer system Known Error Log, maintained by Fujitsu from 1999, of all bugs, failures and their fixes (Fujitsu supplied the Post Office’s IT system) was “a red herring” and irrelevant and anyway wasn’t in the Post Office’s power to disclose (i.e. produce).
⚖️That was false and misleading. The KEL was of decisive importance in exposing the Horizon system to be unreliable.
⚖️That was 20 years after the Post Office began its vicious campaign of prosecutions.
Between 2000 and 2014 the Post Office acted as a ‘cowboy’ prosecutor, permitted by statute to assume the mantle of the state – but not subject to any regulatory supervision and answerable only to itself. Astonishing.
⚖️Most troubling, is that to date the Court of Appeal Criminal Division has set its face against exercising one of its two principal functions.
⚖️It has two functions, a “reviewing” function – considering the safety of convictions obtained in the lower courts, and a “supervisory” function – supervising the conduct of proceedings in the lower courts.
⚖️The CACD has exhibited no interest in why so many miscarriages of justice occurred in the same way.
⚖️That is unsatisfactory and extraordinary.
⚖️One consequence is that the CACD continues, questionably in my view, to uphold Post Office convictions when prosecutions fall outside the simplistic binary classification that it has adopted in Post Office cases.
⚖️ e.g. August 2023, Joanne O’Donnell v Post Office – the late Joanne O’Donnell was convicted in 2007 when it is increasingly apparent that the Post Office was a fatally flawed unregulated prosecuting authority – as now meticulously explained to Williams.
⚖️It abused and unscrupulously exploited privileges that for so long it was afforded for its own ends.
⚖️More requires to be done – not least to restore ebbing public confidence in the courts.’
POSTED ON LINKEDIN – 8th October 2023
The assertion “Computer says guilty” seems to be incompatible with article 22 of the GDPR, which says
Art. 22 GDPR Automated individual decision-making, including profiling
1. The data subject shall have the right not to be subject to a
decision based solely on automated processing, including profiling,
which produces legal effects concerning him or her or similarly
significantly affects him or her.
https://gdpr-info.eu/art-22-gdpr/
I think the UK GDPR and the preceding DPA have the same provision.
I note that these make an exception for law enforcement, but
that seems to make the assumption that legal investigations and
court proceedings will result in a fair outcome.
I was aware of this because I’ve been reading
https://walledculture.org/the-book/
which is about the analogous situation of the power of large
organisations versus the power of individuals in the context
of copyright enforcement.
Deep rabbit-hole warning – search for “Article 22” for the
paragraph I was looking at.
https://www.mixcloud.com/Resonance/playlists/password/
This is a very good listen and includes James Christie interview on computer evidence and the Law Commission who pushed for the repeal over 2 years ago, and who has written a brilliant paper which I will post, plus interview with Lord James Cellan Jones on AI and the Law Commission.
In my experience it’s straightforward to prove that no network of computers is ever “in order”. The challenge that the IT professional faces is one of mitigation of that fact. Systems administrators constantly worry about how likely the various problems they are aware of are to cause a material failure of the business that their network supports.
Every time you update your computer you are fixing potential problems that have been discovered since the last time you did it.
Good admins spend a lot of time contriving ways to test material assertions about the state of their systems that are functionally independent of it. The idea is that getting exactly the same wrong result in two different ways is very unlikely. In computer science, and indeed in science generally, there is no such thing as certainty and the onus is on whoever makes a claim to demonstrate how much doubt it is appropriate to have about it by means of a method that can be independently verified.
Assumptions that imply certainty of intended function are not scientifically acceptable in relation to this sort of equipment whatever the law says about it.
“Source code” is generally an unintelligible listing of alphanumeric characters and symbols which if published for disclosure would be meaningless.
The evidence required would generally be that of how it performs within the system for which it’s designed. This would include outputs, reports, error logs etc.
I believe you refer to “proprietary source code” as opposed to “open source code”. Shouldn’t a Judge be able to insist that software authors provide intelligible commented code complete with versioned history for consideration by expert witnesses if they want serious evidential weight to be placed on any output the contrivance happens to have produced in the past? A “Habeas codice” device that fetters the ability of corporations to benefit from untestable assumptions about the quality of their code is obviously in the public interest.
“Shouldn’t a Judge be able to insist that software authors provide intelligible commented code”
In the Post Office case, the issue is that “intelligible commented code” does not exist.
The requirement perhaps might be for a computer scientist to analyse the code to uncover defects. I note that many “expert witnesses” are not properly qualified to comment on software systems, let alone the code within them.
The Fujitsu / ICL software issues described elsewhere date from the mid to late 1990s and probably earlier. It would also appear that poor coding practice was the norm and standards were low so expecting code to be commented might be a stretch; open source or proprietary code is irrelevant.
Even if the millions of lines of code had been commented, what would a judge, jury or an expert witness be expected to do with it? Unravelling another person’s coding is hard; they’d stand a better chance if the Horizon system had been written in Klingon.
I feel I have to respond Peter as I disagree substantially with what you are saying.
To start, yes, the ICL Pathway software was written in the 1990s. This does not mean that it should have been done badly.
For context, during the 80’s, ICL spent vast sums implementing quality systems and procedures as it knew it needed to improve delivered quality of systems and applications. Much of the UK’s national infrastructure depended on these systems and procedures and ran – and still run in many cases – reliably with high integrity for decades. Think: HMRC core tax systems, DSS core systems, command and control systems for the RAF/Royal Navy, mortgage systems at the largest building societies, BACS … I could go on.
Some of these systems still run today.
It is wholly wrong to state that “poor coding practice was the norm” in ICL in the 90’s. Your comment “standards were low so expecting code to be commented might be a stretch”, I suspect reveals that you are not qualified even at an elementary level in computer science or software engineering? Comments are not required in code that is clear and easy (for a qualified expert) to understand. Especially if that code is written in the scope of a properly constructed and peer-reviewed design specification. Generally in ICL at the time, standards were not low at all. They clearly were, however, very low in the particular team that was implementing the cash accounting part of Horizon.
“Even if the millions of lines of code had been commented, what would a judge, jury or an expert witness be expected to do with it? ” An expert witness would very quickly be able to determine the overall standard of coding within the system. Indeed, there was evidence of poor coding in the Horizon system that was was highlighted by the technical team to management in the late 90’s. In particular, it was recommended by the technical team that the “cash accounting” module needed to be rewritten. That ‘management’ decided not to rewrite the code is not a failing of the technical team, is a simple failure of management.
I should add at this point that an expert would likely use a code complexity analysis tool to produce an automated report to allow examination of the code to be directed.
“Unravelling another person’s coding is hard; they’d stand a better chance if the Horizon system had been written in Klingon.”
This is simply not true. A properly qualified, experienced expert can unravel code quite easily.
For the record, I was employed by ICL from July 1986 until March 1998. I had nothing to do with ICL Pathway.
Apologies for the misunderstanding.
I was referring to the quality of work undertaken by the Horizon team specifically “as described elsewhere”.
This original post refers only to the PO Horizon scandal and Fujitsu’s role in that. None of it, to my knowledge refers to ICL or Fujitsu as a whole.
I hope this helps.
Hi Peter, apologies – I should have framed my comment more carefully. One important aspect that I failed to get across, I think, is that ICL knew how to built reliable, resilient and durable systems and had been doing so for a very long time – so, why was ICL Pathway different? Why did it not follow the best practice in place elsewhere in ICL? There are questions to be asked about this. This is quite apart from the specific issues identified during the project now enumerated in the public record. There also should be an examination of why the Post Office chose to deploy the system (even to pilot users) when they knew of fundamental problems in the software.
My vote for the ‘most forensic judgment’ goes to Mr Justice Gray in the Irving libel trial. That said, he did have Prof Richard Evans and his research assistants to hand. The 350 page judgment exceeded in length Evans’ own ‘Telling lies about Hitler’. If I were to be saddened about the state of English law (as opposed to its politics) I’d use this for reassurance.
Very much looking forward to the next post on this issue of law and the reason it was reversed!
I wrote an article (published on the Justice Gap website https://www.thejusticegap.com/the-justice-systems-role-in-non-disclosure-of-horizon-material/ ) in which I argue that PII rulings are the cause of non disclosure of Horizon related material. I’d be interested to know what contributors to this stream think.