Research

Grammar Schools deny that the effect of tutoring on 11+ test exists.  I want to measure it.

Having completed my BSc Hons in Maths, Statistics and Pedagogy I’m looking for support to undertake a PhD research to investigate, “Why grammar schools are stuffed full of middle class kids”.  This an online version of my research proposal.

background

Former chief inspector of schools, Sir Michael Wilshaw, is on record as saying Grammar Schools are full of middle class children1.  This is at odds with the key claim made in support of selective education; that it promotes social mobility.  Addressing the House of Commons, Prime Minister Theresa May said, “If we look at the attainment of disadvantaged and non-disadvantaged children, we see that the attainment gap in grammar schools is virtually zero, which it is not in other schools.” (HM Government 2016)2.  The Department for Education clarified that Mrs May was comparing the proportions of disadvantaged and non-disadvantaged pupils achieving 5+ GCSEs at A* to C, (Full Fact 2016)3.  Disadvantaged pupils are generally defined as those in receipt of Free School Meals (FSM) and figures produced by Andrews et al (EPI 2016)4 appear to support her position.

5+ A* – C % Non-FSM FSM Gap
Selective Schools 96.8% 92.5% 4.3%
Non-selective schools in non-selective LAs 60.9% 35.0% 25.9%

Proponents of selection hold this up as evidence of disadvantaged children doing better than expected at grammar schools but any narrowing of the gap might equally be caused by more affluent children performing less well once other factors have been controlled for.

It is common knowledge that children are tutored specifically to pass the entrance test and access to this tutoring is based on parents’ ability to pay.  My hypothesis that any tutoring towards the 11+ test also inflates KS2 results which are taken the same school year but that after five years of secondary education the effect of any would have diminished.  I believe Andrew’s results taken out of context by Mrs May are more likely evidence of affluent tutored children underperforming rather than disadvantaged children benefiting.

A second potential barrier to social mobility has come about through a combination of the 1989 Greenwich judgement5 and the Academy program which hands control of admissions to schools themselves.  Ironically Greenwich ruled that children should be able to their nearest primary school but has been used by schools to justify the exact opposite; extending admissions to a wider catchment in order to increase prior attainment of their intake.  The single most significant explanatory factor controlling GCSE results is prior attainment meaning selective schools can become ‘better’ simply by widening their catchment and calling this parental choice.  This choice isn’t available to families who cannot afford the expense of sending their children long distances to school.

The effects of Greenwich and the Academy program are likely to reduced social mobility.  Andrews et al (ibid) found pupils travel twice as far to grammars and the government have announced they intend to open new state-funded grammar schools and allow non-selective schools to convert to selection (POST 2016)7.  It is imperative that a systematic analysis is carried out to quantify the combined effects of tutoring, the Greenwich ruling and de-regulation of admissions on access to this type of education system.

Methods

Design

The research into KS4 attainment gaps will be primarily based on longitudinal census data from the National Pupil Database (NPD).  Previous studies have broadly categorised pupils as living in either selective, partially selective or non-selective LAs.  I will go into much finer detail by combining pupil data with geospatial information from the Ordnance Survey to work out actual distance to the nearest grammar school as a continuous variable to investigate the effect of what Allen (2016)8 refers as the porous local authority borders.

Parents in both grammar school and non-grammar school areas will be surveyed to establish the relationships between affluence, proximity to selective schools and the amount of tutoring children receive.

Kent County Council’s (KCC) admissions team have already expressed an interested in helping with research into the effects of tutoring.  Kent is a fully selective county which tests 15,000 pupils each year and can provide these data with the NPD unique pupil reference (UPR) enabling 11+ test results to be linked all of the metrics captured in the NPD.

The raw 11+ test marks will be obtained to build up a picture of the comparative supply and demand for different selective schools across the country.

The two main 11+ test providers (GLA and CEM) will be approached to ask if they would be prepared to set controlled tests in primary schools parts of the country which are far enough from grammar schools as to preclude the possibility that any of the children have been tutored.

Data from the National Travel Survey will be used to estimate the overall effect of the Greenwich ruling on travelling distances in general.  National Schools Census and Ordnance Survey data will be used to break this down by school type (Grammar/Non-Grammar, Academy/ Maintained).

Analysis

A number of separate studies will be conducted and the overall results combined.

Study 1 – ‘the KS2 boost’

The first test will examine the main hypothesis, that tutoring children to perform well in the 11+ test also boosts their KS2 SATs taken shortly afterwards but by the time they take GCSEs, six years later, this ‘boost’ effect has diminished.  This study will take a single cohort (approx. 500,000 pupils) from the NPD and enrich these data with three new derived pupil level linked variables:

(* using the Ordnance Survey codepoint dataset)

Multiple linear regression will then be carried out to investigate the model

Prais (2001)9 found grammar schools to have better value-add but this study did not account for progress they made between KS1 and KS2.  One explanation for these results would be that those children who make more progress between KS1 – KS2 may be predisposed to faster progress due to influences independent of their schooling (genetics, supportive home environment, better access to enrichment activities etc.)  By including KS1 – KS2 progress in the model I will isolate non-school influences.

Study 2 – parental surveys

Schools in both selective and non-selective and asked for help in involving parents in a very simple survey providing:

  • Level of tutoring using multiple choice categorical variables
  • Gross family income using categorical variables to select bandings.
  • Postal sector (first five digits of the post code covering ~2,600 households to ensure anonymity. Source: UCL’s Centre for Advanced Spatial Analysis)

The postal sector will be converted to an approximate distance to nearest grammar school using Ordnance Survey CodePoint data and used to investigate the model

Study 3 – Kent

Most grammar schools (>95%) are academy status so responsible for setting their own admissions.  As a result “The 11+” can vary enormously from school to school.  Kent is a ‘fully selective’ authority which administers the testing of 15,000 children on behalf of all the schools in the county.  They cross reference these results against the NPD meaning it is possible to validate the assumed correlation between 11+ test results and KS2 SATs.

The objectives of studying these data are somewhat open ended at this stage but these open up the possibility to investigate other potential factors such as

  • Gender
  • Ethnicity
  • IDACI
  • Age within the year group

Data retention policies might mean that linking these date to final GCSE results may have to wait until those children for whom there is data to complete their GCSEs.

Study 4 –controlled exam

Grammar schools claim that no preparation is necessary for the 11+ tests.  To investigate the effects of taking the test with no preparation, primary schools in areas far enough from grammar schools to be confident that the children have not been tutored will be asked to take sit typical 11+ tests.  Propensity matching based primarily on KS1 APS will be used to compare these test takers against those in Kent.

Greenwich, the Academy program and affluence.

To investigate the effects of legal rulings and government policy on schools admissions, data from the National Travel Survey will be used to model any change in overall school commute distances.  Data from NPD will be used to look for relationships between IDACI score and the distance children travel using a block design categorised by home environment (urban, suburban rural etc.) school type (selective, non-selective).  Raw test marks will be obtained to build up a comparative picture of how difficult it is to get into a given school by comparing results to nationally standardised (within each test provider) marks to generate a ‘heat-map’ showing the degree of exclusivity for each school.

Expected Outcomes and Objectives

Cribb et al (2013)10 observed free schools meals children, frequently used as an indicator of social deprivation, are proportionally underrepresented at grammar schools.  FSM is a binary indicator.  When Coombs (2016)11 looked at the Income Deprivation Affecting Children Index (IDACI) it is clear that the bias towards more affluent children being admitted to grammar schools extends across the whole socioeconomic spectrum.  The outcome of Cribb’s research has been the government threatening to remove schools’ selective status unless the admit more FSM children but this positive discrimination does nothing for those children who are just above the threshold to qualify for FSM.

Smith-Woolley et al (2018) found the genetic makeup of test takers was influential in final outcomes and the government have claimed that 11+ tests are now less susceptible to tutoring (Greening 2016)12 (Nash 2016)13.  In the absence of quantitative analysis which controls for the external factors such as Smith-Wooley found, proponents of the selective system can argue that Cribb’s findings correlating FSM to admissions does not prove causality; grammar schools are full of affluent children simply because they are the intelligent offspring of intelligent, and therefore affluent, parents.  I would like to bring some objective research to this question.

I have discussed my ideas with the former Government Chief Scientific Advisor for Education, Dr Tim Leunig who responded positively to a preliminary study drawing parallels with other research showing that private school students do less well at university than their A-level results would indicate when compared to state school students (Rodeiro & Zanini 2015)14.  When the ‘push’ to do well at A-level is removed, these students drop back down to what might be described as their ‘normal’ level of attainment.  I recommended that 11+ tests should be regulated and the results included in the National Pupil Database and whilst I cannot claim any connection a month later the national press reported the government were considering introducing national 11+ tests (Daily Telegraph 2017)15.

Acknowledgements

I’ve am indebted to the support, encouragement and useful suggestions from Prof Rebecca Allen, Director of UCL’s Centre for Education Improvement Science (CEIS).  My research proposal has improved greatly from discussing it with her although the errors in this proposal are all my own work.

References

1 Daily Telegraph, Ofsted chief: grammar schools ‘stuffed full’ of rich pupils https://www.telegraph.co.uk/education/educationnews/10518893/Ofsted-chief-grammar-schools-stuffed-full-of-rich-pupils.html
2 May T., Britain, the great meritocracy: Prime Minister’s speech.  9 September 2016.  Published by HM Government, https://www.gov.uk/government/speeches/britain-the-great-meritocracy-prime-ministers-speech
3 Full Fact (independent fact checking charity) Poor children at grammar schools seem to do well.  Sept 2016.  https://fullfact.org/education/poor-children-get-grammar-schools-seem-do-well
4 Andrews J., Hutchison J., Johnes R., Grammar Schools and Social Mobility, 23 September 2016.  Education Policy Institute.  http://epi.org.uk/report/grammar-schools-social-mobility
5 Regina v Greenwich London Borough Council, ex parte John Ball Primary School (1989) 88 Local Government Reports 589 [1990] Fam Law 469.
6 Thompson D., Why do pupils at schools with the most able intakes tend to make the most progress? 2015, https://educationdatalab.org.uk/2015/05/why-do-pupils-at-schools-with-the-most-able-intakes-tend-to-make-the-most-progress/
7 Parliamentary Office of Science and Technology (POST), Academic Evidence on Selective Secondary Education, 2016 http://researchbriefings.parliament.uk/ResearchBriefing/Summary/POST-PB-0022
8 Allen R, Grammar schools contaminate comprehensive schooling areas, Education Datalab 2016 https://educationdatalab.org.uk/2016/08/grammar-schools-contaminate-comprehensive-schooling-areas/
9 Prais, S. (2001) Grammar schools’ achievements and the DfEE’s measures of value-added: an attempt at clarification, Oxford Review of Education, 27, 1, 69-73
10 Cribb J., Jesson D., Sibieta L., Skipp A., Vignoles A. Poor Grammar, Entry into Grammar Schools for disadvantaged pupils in England.  The Sutton Trust, 2013, https://www.suttontrust.com/wp-content/uploads/2013/11/poorgrammarreport-2.pdf
11 Coombs J, Will poorer families benefit from the return of grammar schools.  TRAK 2016 Published online http://trak.org.uk/will-poorer-families-benefit-from-the-return-of-grammar-schools/
12 Greening J., House of Commons debate, Schools that work for Everyone, 12 September 2016.  Published by Hansard, https://hansard.parliament.uk/commons/2016-09-12/debates/9FBA99CB-1059-4F5D-903B-D2C7509BF46E/SchoolsThatWorkForEveryone
13 Nash, House of Lords debate on grammar schools.  Hansard, 07 September 2016Volume 774 http://hansard.parliament.uk/Lords/2016-09-07/debates/93231CA0-E307-446F-A310-5A2016585BB1/GrammarSchools
14 Rodeiro C. L. V., & Zanini N., The role of the A* grade at A level as a predictor of university performance in the United Kingdom,   Oxford Review of Education Vol. 41, Iss. 5, 2015
15 Daily Telegraph, ‘Test tourism’ for grammar schools to be replaced by national exams, ministers hint, 2017, http://www.telegraph.co.uk/education/2017/02/08/test-tourism-selective-schools-replaced-national-exams-grammar/
16 The Independent, Grammar schools could admit just the cleverest 10% of pupils under new plans

Education chiefs considering ‘national selection test’ to clamp down on exam ‘tourism’ 2017 http://www.independent.co.uk/news/education/grammar-schools-admit-cleverest-10-per-cent-pupils-new-plans-gsha-education-department-national-a7570761.html

17 Smith-Woolley E., Pingault J., Selzam S., Rimfeld K., Krapohl E., Von Stumm S., Asbury K., Dale P., Young T, Allen R., Kovas Y. & Plomin R. (2018) Differences in exam performance between pupils attending selective and non-selective schools mirror the genetic differences between them, Nature magazine