The problems behind the recent exam results chaos illustrate a more general need to rethink which factors shape public policy, writes Stephen Muers. He discusses the role of legitimacy, context, and symbolism in UK policymaking.

It is too early to know the full impact and the full lessons we should learn from the exam system problems of August 2020. It is always easy to jump to conclusions about what should change without full information or a proper perspective. But the problems do illustrate the more general need to put culture and values at the heart of policymaking.

The news, politics, and the education system were dominated in mid-August by chaos around exam results. Thousands of students and their families suffered days of disappointment and uncertainty. All the four governments in the United Kingdom ended up making dramatic late changes to the way exams were graded. By any reasonable measure, this was a significant failure of public policy, creating clearly avoidable emotional, social, and financial costs. There will undoubtedly be detailed investigations to find out what went wrong. My recent work has focused on the place of culture and values in policymaking. I believe that we can draw lessons from the exam grades experience about the importance of putting those factors at the heart of our approach to policymaking.

Legitimacy

The first lesson is about legitimacy. To be successful, a policy needs not only to achieve its objectives but to do so in a way that is seen by citizens to be legitimate. Some policies, especially those that require public consent to operate, will simply not work if they are not recognised as legitimate by a large majority of the population. The restrictions on personal liberty introduced to control COVID-19 are an obvious recent example. However, legitimacy goes further than this: it is valuable in and of itself.

The original system for awarding exam grades in 2020 did succeed in one sense: it achieved an aggregate distribution of grades that was roughly in line with previous years. However, it was not seen as legitimate because of the way it treated a minority of individuals, especially from schools with poorer academic records. Policymakers seem to have failed to understand what the public value in an exam system is, and what people see as fair. It became clear very rapidly once results were released that fairness at an individual level was fundamental to the legitimacy of the system, and trumped the aggregate distribution. If the government had started from a strong understanding of how exams fitted in to underlying value-judgments in society about what was fair, they would surely have designed the system differently.

Data and context

One of the immediate debates prompted by the A-level situation is the role of data and particularly algorithms in policymaking. The exam results were generated by an algorithm combining teacher assessed grades and historical performance. There is nothing inherently wrong with using algorithms in public policy, but the A-level experience shows it is essential to think about the cultural context in which they are being deployed. Putting an understanding of culture at the heart of the policymaking process helps us to recognise the heterogeneity of public service delivery systems. Even systems that are on paper highly centralised, for example the Jobcentre Plus network, have been shown to exhibit wide variation in practice on the front-line due to different delivery cultures in different offices across the country.

The school system is several degrees more decentralised. Schools are in fact encouraged by the inspection regime to develop strong and distinct cultures and sets of values. A single algorithm with a centralised set of data produces results that mean very different things in those different contexts. Had Ofqual taken draft exam results and interrogated them from a bottom-up perspective, looking at the implications for individual schools with their different approaches, cultures, and contexts, they surely would have discovered the problems that later undermined the whole system. Future policymakers would do well to combine sophisticated data analysis with a decentralised approach, understanding the realities in different places and different contexts.

The symbolic power of policy

Culture is carried in part by symbols and stories. Every culture through history has had particular events, myths, individuals and practices that convey something about what is important in that place and that society. In my time as a senior policymaker, calling a policy ‘symbolic’ was often seen as derogatory, meaning that it wasn’t a serious attempt to have a real-world effect. I believe this attitude is mistaken. Symbols do matter, and government can have long-term substantial effects by effectively (or not) using the symbolic power of particular policies.

The exam results U-turn showed the power of symbols very clearly. Individual students with particular stories dramatically symbolised the unfairness of the whole results regime in a way that statistics simply could not. The student who told the Schools Minister on national television that he had ‘ruined her life’, or the student who told Newsnight his story about hoping to be the first from his state school to go to Oxbridge had a direct impact on policy change through the symbolic power of their stories. Governments have to take the symbolism of their policy (and its obvious consequences) seriously if they want to succeed.

My previous work on culture and values in policymaking concluded that the way these factors intersect with the political system makes effective accountability for the failure of specific policies unlikely. There is considerable evidence that voters do not make choices on the basis of an understanding of policy proposals and an assessment of whether or not they have been delivered. Such a position would require voters to have both an unrealistically high level of policy understanding and a belief that governments do successfully implement their policy promises. It is more plausible, both theoretically and empirically, to argue that much voting behaviour is driven by a broad sense of values-alignment with a given party or candidate. In this model, policies become important more for what they stand for, and what message they send about values and approach to governing, than their real-world impact.

The implication is therefore that accountability for real-world policy failure is likely to be limited. Rather than looking at whether a policy has failed and then voting for or against the party responsible accordingly, voters start with a predisposition to align with a particular party for deeper reasons, and interpret information about the impact of a policy in a way that aligns with that. I am fundamentally much more likely to think a policy is a success if it is introduced by a party I identify with in terms of fundamental values than if something identical was done by their political opponents.

It is clearly too early to tell if any of the governments in the UK will be held to account for the problems with exam results, to the extent of suffering falling popularity and electoral consequences. My hypothesis, based on the above, is that they will not. The possible counter-argument would be that these U-turns could combine with other errors or changed decisions to create a deeper sense that a government is not competent or doesn’t care about people. That would be enough to start to shift those deeper values-based alignments that seem to matter more to voting behaviour. So, exam chaos will not bring down a government (or even, probably, an individual minister) but it might, over time, be part of a slower change in perceptions that does have consequences.

It is too early to know the full impact and the full lessons we should learn from the exam system problems of August 2020. It is always easy to jump to conclusions about what should change without full information or a proper perspective. But the problems do illustrate the more general need to put culture and values at the heart of policymaking.

_________________

About the Author

Stephen Muers is a Policy Fellow at the Institute for Policy Research at University of Bath. He has held a number of senior policy-making roles in central government and the non-profit sector and is currently interim CEO at Big Society Capital. His book Culture and Values at the Heart of Policy Making was published by Policy Press in July 2020.