Category Archives: Research

This week we published 3 Open Access books on Early Grade Reading, Mathematics and Interventions in SA :)

There are lots of things that are wrong with South Africa at the moment. As I type this our parliament is in the process of deciding whether President Ramaphosa should be subject to an impeachment inquiry. But even when things are dark (literally), and the prospects for improvement are dim, South Africa still has a lot of things going for it. I’ll mention just one of them: a thriving community of scholars who give a shit about the country, who want to do impactful research, and who really care about improving the life chances of poor kids. Not many people know that before COVID hit, South Africa’s learning outcomes were improving quickly. That is to say they were both improving and terrible at the same time – they are not mutually exclusive. This week Oxford University Press published three volumes on (1) Early Grade Reading, (2) Early Grade Mathematics, and (3) Early Grade Reading and Mathematics Interventions. A few years ago a few of us were discussing progress in reading and mathematics outcomes and we all agreed that we know so much more today than we did in 2010. In some areas (TVET and housing, for example) we know almost nothing more in 2022 than we did in 2010. Things are bad and we don’t really know how to improve them at scale. In early grade reading and mathematics things are different. We have much more data now, we have a whole wave of new research and (excitingly) new researchers. We decided that we should write a book to try and document what we now know at the end of the first decade. We quickly realised that it wouldn’t fit into one volume, and so now we have three! Although I suppose I was the chief cat-herder across the volumes, the entire series was a collaborative effort. The folk at Oxford University Press (Megan, Marisa, Ashley and the copy-editors) were efficient, professional, flexible and passionate about the project. Our excellent editorial assistant (Jess Qvist) made sure that no ball remained dropped for too long. My co-editors Lilli, Stephen, Hamsa and Nicky were exactly what you want in academic colleagues – smart, funny, friendly, punctual, and always ready to call you on your bullshit. It’s such a nice way to close off the year to see the books in print, and also as Open Access ePDFs which are free to download. Of course nothing is ever the final word and research is an iterative process so we look forward to everyone’s comments and critiques and will most certainly have an in-person launch event in the new year!

For now you can download all three volumes here.

Preface

“Collectively the three books bring together 77 authors from disciplines including economics, linguistics, literacy studies, mathematics education, teacher education, and policy studies. Although their domains and methods of analysis may differ, all authors grappled with the same underlying question: why is it that so few young children in South Africa acquire the building blocks of reading and mathematics in the first years of school? While international large-scale assessments have drawn increasing attention to learning outcomes at the primary school level, there is now a broad-based consensus that the roots of the problem lie even earlier than upper primary school. International assessments like PIRLS and TIMSS show that 60–80% of Grade 4 and 5 learners cannot read for meaning or calculate using the four operations, but emerging research documented in these volumes highlights that more than 50% of learners at the end of Grade 1 do not know all the letters of the alphabet, and cannot add and subtract single-digit numbers.

It is this challenge that animates the research across these three volumes, with an analytic focus on lessons learnt in the last decade (2010–2022). While learning outcomes in South Africa before the Covid-19 pandemic were improving quickly by international standards, the chapters included here present evidence for both optimism and alarm. Optimism because system-wide improvements do not happen accidentally or in a vacuum. Alarm because in 2022 it is still the case that the dignity and life-chances of millions of children in South Africa are foreclosed because they do not learn to read for meaning, or do mathematics with understanding in the first three years of school.

As a group of scholars committed to understanding and documenting the roots of both blockages and breakthroughs in reading and mathematics, it is our hope that you, the reader, find this new research interesting, helpful, generative, and challenging.”

“Girls Do Better” (Our new journal article on the gender gap in SA)

I was happy to see an early Christmas present in my emails today. A paper I wrote with my co-author, Dr Nwabisa Makaluza, was released online today! The full ungated article is available HERE. We tried to take all of the education data that we could get our hands on from 1995 – 2018 and see where and when girls do better or worse than boys. This is one of those empirical questions where we don’t have to rely on the unsubstantiated claims of every Tom, Dick and Harry around the braai. The evidence is pretty clear: Given the way girls are, the way they are socialised, the way we organise schools and the way we assess kids, girls do better than boys.

Screen Shot 2019-12-13 at 16.54.38

 

Screen Shot 2019-12-13 at 16.59.50

Screen Shot 2019-12-13 at 17.00.13

Screen Shot 2019-12-13 at 17.00.27

Screen Shot 2019-12-13 at 17.01.25

Screen Shot 2019-12-13 at 17.02.07

Screen Shot 2019-12-13 at 17.02.21

 

Links I liked…

Screen Shot 2019-10-20 at 06.14.03

Some new research I liked…

mg graph

If you’ve come across interesting articles please post them in the comments!

 

Education research…

barca

This week was the 2018 AEDE conference in Barcelona and Prof Lorraine Dearden gave the keynote address: “Higher Education Funding, Access and Returns: Policy Lessons from England” which was so relevant for South Africa given all the recent decisions about free higher education for poor and working class students. I include some slides from her presentation:
Screen Shot 2018-06-29 at 15.46.29
Screen Shot 2018-06-29 at 15.46.37
Screen Shot 2018-06-29 at 15.46.52
Screen Shot 2018-06-29 at 15.47.04
—————————————————————————-
Some other great papers (and abstracts) from the conference:
Screen Shot 2018-06-29 at 10.15.44

 

screen-shot-2018-06-29-at-15-41-21.png

 

Reading in African Languages: An Annotated Bibliography 2004-2017 (Pretorius, 2018)

reading

I think there is now broad-based agreement that most South African children do not acquire the skills and dispositions they need to read for meaning and enjoyment. The PIRLS Literacy 2016 results show that 78% of Grade 4 students couldn’t read in any language. The way forward is therefore to ‘get reading right.’ Given that more than 70% of South African children learn to read in an African language in Grades R-3, we need to understand more about how children learn to read in these languages, and that inevitably involves research. Do children from different language groups learn to read in different ways? The language structures (orthographies) of South African languages are quite different to each other. As we’ve pointed out in some of our earlier work, the same sentence in different languages looks very different:

reading2

Should we be teaching Nguni languages (isiZulu, isiXhosa, SiSwati etc.) and Sotho languages (SeSotho, Setswana, Sepedi) in different ways? Or are these just peripheral differences that don’t change the overall approach. At the moment there is not a large body of research on this. However, Prof Lilli Pretorius has recently published an annotated bibliography of 40 studies (2004-2017) titled Reading in African Languages an Annotated Bibliography 2004-2017 under the PRIMTED banner. I include their blurb below:

“This annotated bibliography was compiled by Professor Lilli Pretorius of UNISA as part of the Primary Teacher Education Project (PrimTEd). It gives a summary account of research that has been done on reading in African languages from 2004 to 2017, more specifically on languages belonging mainly to the family of Southern African Bantu languages. It comprises over 40 annotated entries, mainly research articles from accredited journals, chapters from books and postgraduate dissertations or theses, and also lists several other sources closely related to reading in the African languages. Although it was originally compiled in 2017, it is designed in such a way that new entries can be added to it as new research emerges, and it will be regularly updated.”

Screen Shot 2018-05-22 at 11.20.43

Screen Shot 2018-05-22 at 11.20.59

This is a great resource both for those just starting out in the field, but also for established researchers looking for an overview of what’s out there.

Many thanks to the Lilli and the PRIMTED team for doing and initiating this important work. I believe the English-as-First-Additional-Language (EFAL) annotated bibliography is soon to be released.

For those interested here is my Q&A with Lilli from 2014.

The Comprehension Iceberg: Developing reading benchmarks in African languages

The-Iceberg-Illusion-SOURCE-openlab.ncl_.ac_.uk_

For the last few years we have been working on a project exploring reading in three African languages; isiZulu, Xitsonga and Northern Sotho (Sepedi). This was part of an ESRC-funded study looking at “Leadership for Literacy” and schools in challenging contexts. This week we published the first of a series of working papers on some of our results. The paper is titled “Investigating the comprehension iceberg: Developing empirical benchmarks for early grade reading in agglutinating African languages  (2018) and is co-authored with colleagues and friends Prof Elizabeth Pretorius (UNISA) and Mpumi Mohohlwane (DBE), who is now also doing her PhD at RESEP.

In the paper we argue here that we need to move beyond a repetitive focus on low comprehension outcomes; this is simply the tip of the iceberg. Below the surface there is widespread evidence that most children have not acquired the basic ‘tools’ for reading success – the ability to accurately and fluently decode letters and words and move from an effortful activity to an automated skill. Knowing that 78% of Grade 4’s cannot read for meaning in any language is important and sobering information, but probably not as helpful as knowing which components of reading children are struggling with and why teachers are struggling to teach them.  If children and teachers are ‘falling at the first hurdle’ (Taylor, 1989) does it make sense to focus on the seventh or eighth hurdle and ask why learners and teachers are not making it over?

One of the important contributions of the paper is that we try to estimate benchmarks for these three African languages. While many benchmarks exist in English there are currently no benchmarks in African languages and unfortunately you cannot just version the English benchmarks for the African languages because the language structures (orthographies) are so different. This is easiest to explain using an example:

Screen Shot 2018-03-13 at 11.08.10 AMSo in isiZulu the first sentence is only 3 words while in Northern Sotho (Sepedi) it’s 13 words. We go on to show that there are also large differences in ‘required fluency’ across these languages and that accuracy rates and speed differ between conjunctive and disjunctive languages – see below.

Screen Shot 2018-03-13 at 10.54.39 AM

“Comparison across the languages shows that accuracy seems to be more important for fluent reading in isiZulu than in Northern Sotho or Xitsonga. The isiZulu learners reading at 21 wcpm or faster are reading with 95% accuracy or higher. In contrast, 95% accuracy is only associated with reading at 51 wcpm or faster in Northern Sotho and 31 wcpm or faster in Xitsonga. One of the reasons why Decile-1 learners are reading so slowly is that they are making mistakes on every second or third word. The fastest Northern Sotho readers (wcpm=107) and Xitsonga readers (wcpm=91) in the sample made no mistakes whatsoever.” (p12)

This can also be seen visually:

Screen Shot 2018-03-13 at 10.55.10 AM

We then use this information and some rudimentary comprehension outcomes to benchmark the different languages:

Screen Shot 2018-03-13 at 10.54.55 AM

We’d encourage those interested in this type of work to read the full paper – if you have any comments or suggestions we’d love to hear from you! Please include them in the comments below.

 Abstract

The importance of learning to read in mother-tongue is widely acknowledged in the linguistics literature yet reading acquisition in African languages remains under-researched and under-theorized. While numerous studies have highlighted the low levels of comprehension among learners reading in African languages in South Africa, little has been done to understand what lies beneath this ‘comprehension iceberg.’ In this paper we present new empirical evidence on reading outcomes and the subcomponents of reading for 785 Grade 3 learners across three languages (Northern Sotho, Xitsonga and isiZulu), drawn from 61 primary schools in South Africa. This is the largest sample of such learners to date. Using an adapted EGRA-type assessment we assessed letter-sounds, single-word reading, non-word reading, oral reading fluency and oral comprehension. From this data we present results on fluency, accuracy and comprehension and how these might relate to each other in these morphologically rich agglutinating languages. We also show that there are large differences in reading subcomponents between languages with conjunctive and disjunctive orthographies. Our results suggest that there are minimum thresholds of accuracy and oral reading fluency in each language, below which it is virtually impossible to read for meaning. These are 52-66 wcpm in Northern Sotho, 39-48 wcpm in Xitsonga and 20-32 wcpm in isiZulu. We argue that there is a strong need for empirical language-specific norms and benchmarks for indigenous African languages and present our benchmarks for these three languages as a move in that direction.

My ‘Lead the Change’ Q&A with AERA SIG

See below for a Q&A session I did for the Augest edition of “Lead the Change” run by the AERA SIG on Educational Change. PDF here.

Screen Shot 2017-08-02 at 8.55.00 AM

Screen Shot 2017-08-02 at 8.55.12 AM

Screen Shot 2017-08-02 at 8.55.22 AM

Screen Shot 2017-08-02 at 8.55.31 AM

New education research (SA)

Screen Shot 2016-02-21 at 1.33.10 PMImportant new research article by Stephen Taylor and Marisa von Flintel (2016) titled “Estimating the impact of language of instruction in South African primary schools: A fixed effects approach

“For many children around the world, access to higher education and the labour market depends on becoming fluent in a second language. In South Africa, the majority of children do not speak English as their first language but are required to undertake their final school-leaving examinations in English. Most schools offer mother-tongue instruction in the first three grades of school and then transition to English as the language of instruction in the fourth grade. Some schools use English as the language of instruction from the first grade. In recent years a number of schools have changed their policy, thus creating within-school, cross-grade variation in the language of instruction received in the early grades. Using longitudinal data from the population of South African primary schools and a fixed-effects approach, we find that mother tongue instruction in the early grades significantly improves English acquisition, as measured in grades 4, 5 and 6.”

“A rising number of school leadership changes have been occurring in South African schools as a large proportion of incumbent principals near retirement age. While this presents opportunities to replace weaker school principals with better performing ones, these changes may also destabilise school environments and impede on learning. This paper explores how these principal change events affect school performance in the context of South Africa using a unique administrative dataset constructed by linking payroll data on the population of public school principals to national data on schools and matriculation examination outcomes. Exploiting the panel structure of the data, a school fixed effects strategy suggests that principal changes are indeed detrimental to school performance especially when leadership changes are due to principals exiting the public education system. These results are robust to using an alternative estimation strategy proposed by Heckman, Ichimura and Todd (1997) which combines propensity score matching with a difference-in-difference estimation strategy. The paper also considers two mechanisms through which school leadership changes may impact on school performance, namely through rising promotion rates and teacher turnover”

  • Two new policy briefs released by RESEP (1) “Increasing the supply of teacher graduates” (Hendrik van Broekhuizen) and (2) “Education datasets in South Africa” (Chris van Wyk).

    In the first of these, Hendrik van Broekhuizen describes teacher graduate production in South Africa in the context of existing teacher shortages and finds that universities are still not producing enough teacher graduates  to satisfy the demand for qualified new teachers in South African schools.   Importantly he also stresses the importance of UNISA in the production of new teachers –  between 2008 and 2013 UNISA accounted for half (48%) of all new enrolments in initial teacher education. Despite rising enrolments in teacher training programmes over the past decade, enrolment levels remain low. This is exacerbated by the fact that too few students complete their training programmes and too few new teacher graduates become teachers. His research highlights the need to (1) promote the teaching profession, (2) increase the absorption of new teacher graduates, (3) expand targeted funding for African-language teaching students, and (4) place greater focus on UNISA’s increasingly important role in teacher graduate production.   The full research paper can be found here. The graph below summarises the UNISA situation well…

Screen Shot 2016-02-21 at 1.29.12 PM

Important new education research

holy

    • Ursula Hoadley and Joe Muller have just published their important paper looking at assessment in South Africa “Visibility and differentiation: Systemic testing in a developing country context” (Curriculum Journal, 2016)- I prefer their earlier title “Testing testing: Investigating the epistemic potential of systemic testing” (Un-gated draft-version of that paper here).
    • Why has large-scale standardised testing attracted such a bad press? Why has pedagogic benefit to be derived from test results been downplayed? The paper investigates this question by first surveying the pros and cons of testing in the literature, and goes on to examine educators’ responses to standardised, large-scale tests in a sample of low socio-economic status (SES) schools in the Western Cape, South Africa. The paper shows that teachers and school managers have an ambivalent attitude to tests, wary of the reputational costs they can incur, but also curious about the differentiated picture test results can give them as they learn to ‘read’ the underlying codes embedded in the results. The paper concludes that a focus on what tests make visible and a recognition of the pedagogic agency of teachers points to potential pedagogic benefits of systemic tests.

    • Craig Paxton has finally finished his PhD thesis “Possibilities and constraints for improvement in rural South African schools” (UCT, 2015). This is on my to skim/read list together with Eric Schollar’s PhD (see below)
    • Part of Craig’s PhD abstract:”Rural South African schools face a complex mix of challenges, which make improvement a daunting task. Not only do schools deal with the time, place and space issues that face rural schools worldwide, but in addition they contend with a legacy of severely deprived schooling under the apartheid system. Using the framework of the Five Essential Supports, developed by the Consortium on Chicago School Research, together with Bourdieu’s notions of habitus and doxa, this thesis examines what improvement might mean in this deeply disadvantaged context. The five supports – leadership, learning climate, school-community ties, ambitious instruction and professional capacity – are contextualised to account for both the rural setting and the peculiarities of education in South Africa’s former homeland communities. Alongside this largely quantitative framework, Bourdieu’s conceptual tools are brought to bear, offering an alternative perspective that makes sense of the complex forces produced by history and rurality

    •  Pritchett’s new (2015) RISE Working Paper “Creating Education Systems Coherent for Learning Outcomes.” This has been quite an influential paper for me. Although in the South African context I would almost always add “Capacitation” to his four criteria Delegation, Financing, Information, Motivation. Lant also has a great (and scathing) critique of meta-analyses of quantitative studies:
    • If one were to take this approach of “rigorous evidence” at face value then there is rigorous evidence that nothing in the conventional wisdom actually works. There is rigorous evidence that giving out textbooks doesn’t matter, there is rigorous evidence pay for performance doesn’t matter, there is rigorous evidence that class size doesn’t matter. Of course there is also rigorous evidence that all these elements of the conventional wisdom do matter. The usual approach of doing a “systematic review” of the literature that simply counts studies (in a quality weighted basis) is not at all helpful. Suppose that context A is a system coherent for learning—so that teachers know what students should learn, that learning is measured on a regular and reliable basis and teachers are motivated to achieve high student learning—and class size is reduced. Let’s assume that learning improves (as there is RCT evidence from the USA, for instance, that this is true). Context B is a system coherent for schooling only. Class size is reduced. Let’s assume learning doesn’t improve (as there is RCT evidence from Kenya, for instance, that this is true). Suppose the only two studies in the systematic review were USA and Kenya. Then the conclusion would be that “class size improves student learning in 50 percent of the studies.” Now suppose that 8 more rigorous studies were done in the USA so that a systematic review would conclude “class size improves student learning in 90 percent of studies.” Suppose, in contrast, 8 more studies were done in Kenya. Then a systematic review of the rigorous evidence would conclude “class size improves student learning in 10 percent of the studies.” All three statements are equally worthless. The (assumed) truth is that “class size improves performance in context A but not in context B” and hence unless one knows whether the relevant context is A or B the systematic review finding of impact in 50 percent, 90 percent or 10 percent of the studied cases is irrelevant.

  • Glewwe & Muralidharan’s new (2015) RISE Working Paper “Improving School Education Outcomes in Developing Countries” they find that:
  • Interventions that focus on improved pedagogy (especially supplemental instruction to children lagging behind grade level competencies) are particularly effective, and so are interventions that improve school governance and teacher accountability