The government’s 2016 consultation paper, Schools that work for everyone asked a very loaded question. “What is the best way to expand selection?” It did not seek any views on whether selection was desirable. Proponents argue it promotes social mobility and helps levelling up. However, data for the last nine years shows poorer children are consistently underrepresented at grammars in the first place. The Sutton Trust said, “the challenge … is how to ensure that those grammar schools are open to all, and are not simply the preserve of better off families who can afford private tutors or prep school fees.”

To morally justify selection, grammar schools know they need a “tutor-proof” test.

Theresa May claimed, without any evidence that, “many selective schools are already employing much smarter tests that assess the true potential of every child. So new grammars will be able to select in a fair and meritocratic way, not on the ability of parents to pay.”

In my article documenting grammars’ quest for a tutor-proof 11+, I review all the evidence and conclude that this resulted in less fairness. This article documents the legal battle, dating back to 2014, to get test provider CEM to disclose basic information about these tests.

Requested information.

In 2014 I asked CEM to disclose the anonymised raw and standardised scores used for grammar school admissions. The 11+ test varies between different admission authorities but always consists of short multiple-choice questions. Candidates have between 30 and 60 seconds to attempt each question. The marker awards one ‘raw’ mark for each correctly answered question .

After making small adjustments for candidates’ differing ages, the test provider works out the mean score for all candidates and each individual test taker’s score relative to this. Standardisation itself is a well-established statistical practice which originated from Carl Gauss (1777-1855). It results in a negative score for candidates whose scores are below the mean, so the final stage is to rescale the results on to a more convenient scale.

What might disclosure reveal?

In 2015, Schools Week reported an unsuccessful attempt by the Buckinghamshire campaign group Local Equal Excellence (LEE) to get CEM to disclose evidence to support the “tutor-proof” claims.

  • LEE told Schools Week, “CEM should either publish their evidence or stop making claims that have no basis in fact.”
  • The ICO’s position was, “There is public interest in protecting the methods by which CEM is attempting to deliver a fairer testing system.” The ICO recognised it was too early to ascertain whether CEM’s methods would be successful or not, but said they needed to be given a fair chance to succeed.
  • CEM said, “the university did not claim that its tests were “tutor proof” but developed to minimise the impact of “additional coaching.”

That decision established, in the Information Commissioner’s mind at least, that the 11+ is fairer if the content and structure of the test remain secret. CEM have told the courts, “Part of CEM’s strategy in making its tests ‘resistant to tutoring’ is to not publish past papers or practice papers, or the algorithms used to produce the standardised scores.” (Information Tribunal decision, §24). It is not clear how disclosing the scores past candidates would reveal the content and structure of the test, which CEM assert gives it a Unique Selling Point in tutor resistant tests.

In 2016, MP Catherine McKinnell summarised the evidence given to the Commons Education Select Committee that attempts to make the test tutor-proof had resulted in less fairness. Government Chief Scientific Advisor for Education, Dr Tim Leunig confirmed CEM’s attempts to create a “tutor-proof” test had failed and that the grammar schools had to withdraw the test. Despite this, CEM have successfully argued to both the Information Commissioner and successive Tribunals that they should not disclose their raw marks because this would prejudice their commercial interests.

A more plausible reason for non-disclosure.

One reason grammar schools and test providers may not want to disclose the raw marks is this allows them to change the standardisation process to pass more candidates without going through consultation. The outcome of a six month investigation by the Guardian was an article revealing “how grammar schools are expanding – by taking pupils who fail the 11-plus.”

Appealing a refused FoI request.

If a public authority refuses to disclose information, the requestor can refer this to the Information Commissioner to determine whether the public authority dealt with the request correctly. Should the ICO uphold the public authority’s decision, the requestor can appeal to the First Tier Tribunal (FTT). If the FTT dismiss the appeal, the requestor can appeal to the Upper Tribunal, then the Court of Appeal and finally the Supreme Court. (I am unaware of any Information Rights appeals making it to the Supreme Court.)

There is a high bar to any appellant court giving given permission for the appellant to appeal to them. The appellant must submit grounds which demonstrate that the lower court erred in law and that the error was material to the outcome.

I have made three requests for disclosure of raw marks between 2014 and 2020. These have been the subject of ten appeals (including permissions hearings.) I have summarised these in the following sections.

May 2016: Coombs v ICO EA/2015/0226

A footnote on page 5 of this ruling says, “CEM asserts that one of the benefits of its 11+ testing is that it is ‘tutor proof’, or more ‘tutor proof’ than the alternatives. That is to say that it is harder for pupils to be coached to perform well in its tests simply by approaching the test in a certain way. The claimed result is that the test is a better predictor of natural ability and the advantage gained by children in families that can afford private tuition is reduced.”

The decision was not unanimous. The majority ruled, “The merits of those [tutor-proof] assertions are not the issue for the Tribunal. (if they cannot be justified there may be an argument about advertising standards and/or what decision-making process schools used when determining which supplier to use).” [§21 third bullet]

The minority, Judge Hamilton, ruled that “… the claimed USP of tutorproofing was highly questionable and that the public interest warranted close examination of this claim which could only be achieved through the disclosure of the disputed material.”

CEM explained how tutors would benefit from disclosure in closed evidence. Judge Hamilton described this as, “highly technical and incomprehensible to anyone without a qualification in statistics.” I specialised in statistics in my Maths degree. The Tribunal refused to let me even see the claimed reasoning for withholding raw marks.

Aug 2018: Coombs v ICO and Durham University EA/2017/0166

The Freedom of Information Act adopts the default position that a public authority must disclose information unless there is good reason to not disclose it. The reasons, or exemptions, are all defined by law. CEM have relied on s.43(2) prejudice to commercial interests to refuse disclosure of raw test marks they awarded to past candidates.

The Court of Appeal’s judgment in Department for Work and Pensions v Information Commissioner and Zola [2016] EWCA Civ 758 (‘Zola’) provides an authoritative judgement on any prejudice-based exemption setting three tests.

  • Firstly, the Public Authority must identify the relevant interest which they would be prejudice by disclosing the requested information.
  • Secondly, there must be link between disclosure and the claimed prejudice; and
  • Thirdly, the ‘likelihood’ of prejudice must be real and significant.

By law, the first thing CEM must demonstrate is a commercial interest in providing “tutor-proof” tests. In its conclusions (at §36) the Tribunal recorded, “The Commissioner has further asserted that it was for neither her nor the Tribunal to look behind the University’s assertion that it has achieved considerable commercial success as a result of its USP, or to interrogate the commercial validity of this USP.”

Evidence given to this appeal gave lie to the probable true reason CEM do not provide practice material. CEM explained that they collate each test from a bank of questions. They use each question in multiple different tests. The Tribunal’s decision records, ‘[30] Past papers are not made available nor are practice materials sold to the public. This is done in an effort to “reduce the advantage that more affluent parents can obtain for their children by paying for private tuition”. [31] Releasing the information would make future tests more expensive by increasing costs for the University whilst simultaneously de-valuing the bank of test content it holds for re-use in future commercial opportunities.’

CEM never explained how disclosing the marks candidates achieve would reveal the actual questions themselves.

April 2019: Coombs v ICO and Durham [2019] UKUT 119 (AAC)

I successfully appealed the FTT’s on the grounds of inadequate reasoning. Judge Wikeley’s decision recorded (at §17) “Mr Coombs has submitted a detailed reply. I only need cite one passage, which in a sense reflects the lowest common denominator of the arguments in these proceedings: “I’ve read a number of FTT rulings and whilst this one summarises the parties’ submissions I struggle to find anywhere in its 42 paragraphs where the Tribunal explain why they came to their decision.”

Judge Wikeley set aside the lower court’s decision and ordered the FTT to set up a fresh Tribunal to rehear it.

March 2020: Coombs v ICO and Cambridge EA/2017/0166A

(Note. Durham University transferred CEM to Cambridge University. The second respondent changed although it remained CEM. Also, the remitted appeal reference has the letter ‘A’ appended.)

The Tribunal admitted evidence from CEM’s witness filed just before the hearing (at §26) that, “CEM considers its tests to be of a higher quality than its competitors but acknowledged that they were also more expensive, in part because the cost was not subsidised. The ‘tutor-resistance’ of CEM’s test was a strong USP, although some schools and local authorities inevitably chose cost over quality.”

Where admissions authorities have disclosed contract values CEM are invariable less expensive. However, collecting documentary evidence takes a long time. The Tribunal accepted witness evidence that CEM’s tests were more expensive than its competitor (at §45) and dismissed the appeal.

The Tribunal recorded (at §19) that I had “produced a substantial body of evidence in support of his appeal, including summaries from a House of Commons Select Committee in November 2016; media articles; academic research; other information about 11+ test outcomes and online material.” It specifically mentioned the Guardian article in which CEM’s director acknowledged they had withdrawn marketing claims to be able to assess natural ability and the Schools Week article saying the Bucks grammars had ‘sacked’ CEM and gone back to their previous provider.

CEM told the Tribunal it must disregard this wealth of evidence showing that from late 2016 onwards CEM’s which called into question their claimed tutor-proof tests. (In APPGER v ICO and FCO GIA/2230/2012 (AAC) the Upper Tribunal, whose decisions are binding, ruled that the ICO and FTT must determine whether information is exempt from disclosure at the time of its request.) The Tribunal’s finding of fact (at §49) was, “the withheld information was commercially sensitive at date of request.”

February 2021: Coombs v ICO and Cambridge GIA/1121/2020 (AAC)

One of my grounds of appeal to the Upper Tribunal was that the FTT had insufficient evidence to reach its findings of fact that CEM’s tests were more expensive. Counsel for CEM said CEM’s witness statement was the evidence. It seemed to me that the witness could claim that the moon was made of cheese and as that was the only evidence on the constituents of the moon, the Tribunal would base its finding of fact on this. The Upper Tribunal refused permission to appeal.

March 2023: Coombs v ICO and Cambridge EA/2021/0127

The Upper Tribunal’s decision in APPGER was that “it seems to us that Parliament would have intended that the requester should make a further request if he wished to rely on changes over time to the public interest factors.”

I submitted a third request for raw marks. They refused. I referred it to the ICO who upheld the decision, as did the Tribunal. At every stage I explained that the decision maker had to consider whether the information was exempt at the time of request (2020) and consider the overwhelming evidence which emerged in late 2016 and early 2017 debunking CEM’s claimed USP in providing tutor resistant tests.

In setting out the factual background, as is common, the Tribunal replayed the facts surrounding my previous appeal and said they saw no reason to go against that decision.

October 2023 Coombs v ICO and Cambridge UA-2023-001019-GIA

On 17 October, the Upper Tribunal heard an oral hearing to decide whether to grant me permission to appeal the FTT’s decision. As these proceedings are still live it would be inappropriate to comment on them (although that never stopped Donald Trump.)

I can disclose that I am partially relying on another Upper Tribunal decision, Coombs v Information Commissioner and University of Durham: [2019] UKUT 119 (AAC). The Upper Tribunal have promulgated that decision and published it on the government website. In his decision, Judge Citron said the Tribunal did not adequately explain how disclosing the requested information would benefit tutors or why it rejected my challenges to the evidence.