Important new education research

Why has large-scale standardised testing attracted such a bad press? Why has pedagogic benefit to be derived from test results been downplayed? The paper investigates this question by first surveying the pros and cons of testing in the literature, and goes on to examine educators’ responses to standardised, large-scale tests in a sample of low socio-economic status (SES) schools in the Western Cape, South Africa. The paper shows that teachers and school managers have an ambivalent attitude to tests, wary of the reputational costs they can incur, but also curious about the differentiated picture test results can give them as they learn to ‘read’ the underlying codes embedded in the results. The paper concludes that a focus on what tests make visible and a recognition of the pedagogic agency of teachers points to potential pedagogic benefits of systemic tests.

Part of Craig’s PhD abstract:”Rural South African schools face a complex mix of challenges, which make improvement a daunting task. Not only do schools deal with the time, place and space issues that face rural schools worldwide, but in addition they contend with a legacy of severely deprived schooling under the apartheid system. Using the framework of the Five Essential Supports, developed by the Consortium on Chicago School Research, together with Bourdieu’s notions of habitus and doxa, this thesis examines what improvement might mean in this deeply disadvantaged context. The five supports – leadership, learning climate, school-community ties, ambitious instruction and professional capacity – are contextualised to account for both the rural setting and the peculiarities of education in South Africa’s former homeland communities. Alongside this largely quantitative framework, Bourdieu’s conceptual tools are brought to bear, offering an alternative perspective that makes sense of the complex forces produced by history and rurality

If one were to take this approach of “rigorous evidence” at face value then there is rigorous evidence that nothing in the conventional wisdom actually works. There is rigorous evidence that giving out textbooks doesn’t matter, there is rigorous evidence pay for performance doesn’t matter, there is rigorous evidence that class size doesn’t matter. Of course there is also rigorous evidence that all these elements of the conventional wisdom do matter. The usual approach of doing a “systematic review” of the literature that simply counts studies (in a quality weighted basis) is not at all helpful. Suppose that context A is a system coherent for learning—so that teachers know what students should learn, that learning is measured on a regular and reliable basis and teachers are motivated to achieve high student learning—and class size is reduced. Let’s assume that learning improves (as there is RCT evidence from the USA, for instance, that this is true). Context B is a system coherent for schooling only. Class size is reduced. Let’s assume learning doesn’t improve (as there is RCT evidence from Kenya, for instance, that this is true). Suppose the only two studies in the systematic review were USA and Kenya. Then the conclusion would be that “class size improves student learning in 50 percent of the studies.” Now suppose that 8 more rigorous studies were done in the USA so that a systematic review would conclude “class size improves student learning in 90 percent of studies.” Suppose, in contrast, 8 more studies were done in Kenya. Then a systematic review of the rigorous evidence would conclude “class size improves student learning in 10 percent of the studies.” All three statements are equally worthless. The (assumed) truth is that “class size improves performance in context A but not in context B” and hence unless one knows whether the relevant context is A or B the systematic review finding of impact in 50 percent, 90 percent or 10 percent of the studied cases is irrelevant.