Following Louise Tickle’s piece in the Guardian, there has been widespread speculation regarding my motivation for wanting 11+ test marks to be made public. Contrary to some speculation, I am not trying to get my 16 year old son into year 7 at the local grammar. That would be silly.
This is purely about whether information should or should not be disclosed.
Why is this important?
To understand why releasing the information is important requires an understanding of standardisation although ironically, if the information was released everything would be crystal clear.
As any statistician will explain, a standardised score measures the individual relative to the population. What 11+ providers don’t tell you is, behind the scenes, they interpret population as just one year’s cohort. Given a set of test marks, a desired pass mark and the proportion needed “pass” in order to fill the school, it takes less than half an hour to ensure your grammar school standard selects the correct number of applicants to the exact child. To me it is deceptive to refer to something as standardised when it is anything but. However, given the amount of trouble I’ve had explaining this I made a five-minute film explaining it the issues.
Click on Snow White, all will be revealed and you’ll only be five minutes older.
The Freedom of Information Act operates on the principle that information should be disclosed unless there is a reason not to. In this case Durham have said that release of this information would be commercially damaging. The court now need to decided whether overall, it is in the public interest for the information to be released.
There is some confusion about what constitutes public interest. Whilst the public are probably quite interested in how much Manchester United paid for Cristiano Ronaldo it’s not in the public interest in the same way that it would be useful to understand what exiting the EU actually means. In the same way, an individual parent knowing why their child did/n’t get offered a place at a given school isn’t in the public interest in the same way that knowing how schools admissions operate in general. There follows a few reasons why disclosure of this information is in the public interest.
Reasons for Disclosure
If there is a downturn in overall admissions numbers then, like the self-levelling suspension on those funny old Citroëns, grammar school standard is automatically lowered. That’s good for the grammars as they remain full and therefore fully funded but no school is an island. In Kent, which is fully selective, this must come from taking pupils who in other years would have attended the neighbouring secondary moderns. Those schools are faced with dwindling numbers and loss of funding. The heads can’t complain about moving goal posts because they don’t even get to see the goal posts (raw marks).
The law states, “Admission authorities must ensure that their arrangements will not disadvantage unfairly, either directly or indirectly, a child from a particular social or racial group.” GLA have told the Schools Adjudicator that Verbal Reasoning tests are biased against EFL children so it would be in the public interest to understand what weighting each authority gives to each discipline (Maths, Verbal Reasoning, Non-Verbal Reasoning). That way any particular social or racial group who may feel they are unfairly prejudiced can challenge these arrangements but these groups don’t get to see the goal posts either.
Reading School’s 2015 entry set a cut-off mark of 110. The standardised results show that one boy scored 110.00 whilst another only 109.99. To quote the school’s published admissions criteria the second was, “deemed not to benefit from the style of education provided at Reading School.” It continues, “The following criterion will be used to allocate places as between borderline candidates who cannot be distinguished in terms of academic ability. With regard to the historic links with the Borough of Reading, eligible boys living nearer to the School will be accorded a higher priority in the allocation of day-boy places.” This tie-breaker has never been used!! If a child in Newbury scores 0.01 higher than one in Newtown he’s given the place. Data from North York, where children sit two tests a week apart show the residual variance (‘noise’) is about 20 – 30% so a 95% confidence limit would put the true scores of these test takers somewhere in a band of about 20 marks, but the results are recorded to two decimal places! It’s in the public interest to understand that this ‘distinction’ is entirely synthetic so they can question the decisions made by what is an increasing number of unaccountable governing boards of Academies who have been given the power to act as their own admissions authorities. Admissions law says, ‘Parents should be able to look at a set of arrangements and understand easily how places for that school will be allocated’ but the basis on which schools are currently ‘distinguishing’ between candidates is completely hidden.
When the location of the itinerant goal posts is a closely guarded secret, there isn’t much incentive for the referee to pay attention. One school ‘standardised’ six candidates, who sat a late test, as an entirely separate population. As standardisation selects a proportion of the population regardless of absolute attainment, this blunder guaranteed two of these candidates a place. The candidate originally ranked 100th (for 100 places) in the main test scored 113/150 but was pushed down two places by applicants with raw scores of 87/150 and 76/150. Standardisation might come across as a bit of a black art but if you check the raw marks, this gaffe stands out like a sore thumb. This information was not released until the First Tier Tribunal ordered it but if it had been routinely released the parents of the two children who missed out on a place as a result would have had no trouble whatsoever picking up on the mistake.
The 2016 standardised scores for Bucks county included some negative scores. Durham explained this was the result of, “… adjusting the standard deviation to account for a fixed qualifying score of 121 and a year-on-year increase in the number of candidates.” If that’s not moving goal posts I don’t know what is. The scores for the 2016 Verbal Reasoning test consisted of over six thousand different standardised scores. I’d like to understand how slightly over a half an hour’s multiple choice questions can differentiate candidates to such a degree. I can conceive some particularly ingenious mathematical ways to generate such arbitrary precision but it’s about as smart as making a sundial so big you are able to read the time to the nearest second. (Who needs a space elevator when you have a Durham ™ sundial?)
There is also that claim floating around that Durham have somehow managed to invent a test which can’t be tutored. In Coombs vs ICO (2016) Judge Hamilton considered, “… the appellant had provided evidence that the claimed USP of tutorproofing was highly questionable and that the public interest warranted close examination of this claim which could only be achieved through the disclosure of the disputed material.” So at least one Tribunal judge thinks that it would be in the greater interest to be able to challenge such a claim.
There is general public interest in public authorities being transparent in their decision making. Durham’s income from the 11+ in 2016 was £815,801 but this is only part of the picture. These tests are used to decide which children benefit by attending one of 163 grammar schools. Estimating their the annual budget as £5m per school, the best part of a billion pounds of public funding each year depends on the outcome of 11+ tests. Transparency specifically over whether the allocation of school places is based on sound decision making is in the public interest.
Reasons for non-disclosure
Having listed a few reasons why it would be beneficial to release the information it’s important to put the other side of the arguments. The reasons the Information Commissioner gave for non-disclosure are:
- It is in the public interest to develop tutor proof tests.
- The Information Commissioner says that CEM’s rival is not subject to Freedom of Information thus placing CEM at a commercial disadvantage.
- The revenue from these tests helps fund Durham Vice Chancellor’s £470,000 basic salary.
Regarding the first of these points, the Commons Select Committee of Education found, “the Government has yet to demonstrate how an admissions system could be designed in a manner which would be immune to gaming, or down to the ability to pay.” Giving evidence, the Chief Scientific Advisor for Education Tim Leunig said, “Buckinghamshire had attempted to come up with one [a tutor-proof test] but it had not worked.” and schools minister Nick Gibb has likened this objective to the quest for the holy grail.
Regarding the second, GLA are indeed a private company, but the information itself is held by schools. If, for example, I thought GLA were selling tutor proof tests or snake oil I’d ask the schools for information.
With the last point the Information Commissioner’s exact words put a bit more lipstick on this particular pig but still boils down to not releasing information because Durham are making money from something. Exactly what, has me confused. Each time I mentioned “tutor proof tests” during the hearing I was reprimanded by the judge who tells me Durham’s unique selling point is that their tests are seen to be more resistant to tutoring and that it works hard to design tests that are as resistant to tutoring as possible. C’mon that’s one hell of a mouthful!
The hearing lasted about six hours without even managing to establish that commercial damage would result from the release of the information. As we seemed to be in total deadlock over this point I thought it worth conceding so we can get on to the question of which is most important; Durham’s profits or opening up the can of worms.