How to raise the ‘real’ matric pass rate [my Africa Check article on Matric 2014]

matric 2

[This article first appeared on Africa Check on the 13th of January 2014]

The release of the 2014 matric results last week followed the now familiar routine of focussing primarily on the pass rate and how it changed, both nationally and provincially.

The notion of the matric pass rate is one that is deeply rooted in the South African psyche and seen as perhaps the most important indicator of education in the country. This is extremely short sighted.

The public, and it would seem the Minister as well, believe that if the pass rate goes up then the quality of education is improving, and if it goes down then this is an omen of deteriorating quality, or – as was the case in 2014 – due to some other factor like a change in the curriculum.

The problem is that the matric pass rate is a function of the students that actually reach and write matric. More and more commentators and critics are beginning to understand why the pass rate seen in isolation is problematic and in many instances misleading.

To illustrate; of 100 students that started school in 2003, only 49 made it to matric in 2014, 37 passed and 14 qualified to go to university. The pass rate is calculated by dividing the number of students that pass matric (37 of the 100) by those that wrote matric (49 of the 100), yielding 76% in 2014.

However, a more appropriate measure would be to calculate what proportion of a cohort that started school 12 years ago passed matric – what some people are calling the ‘real’ matric pass rate or the throughput-pass-rate. That would be about 36% for 2014, down from 40% in 2013.

(For those interested in the actual numbers the calculation is 403,874 students who passed matric in 2014 divided by the 1,085,570 students that were enrolled in grade 2 in 2004. I use Grade 2 figures as the proxy for the true size of the starting cohort because there is excess grade repetition in Grade 1, leading to an overestimate of cohort size if one uses Grade 1 enrolments.)

 As a nation with a skills shortage and a below average proportion of youth that complete upper secondary school, we should be very concerned that there were fewer students reaching and writing matric than last year.

The table below shows that although there had been a general increase in the throughput rate from 2010 to 2013, the rate came down in 2014. This year there were 29,252 fewer matric candidates than in 2013. Part of this is that 2014 was a smaller cohort than 2013, but the lower throughput rate is also part of the story.

Table 1: Students writing and passing matric relative to initial cohorts at public ordinary schools only (Education Statistics at a Glance and Matric 2014 Technical Report)

  Grade 2 students 10 years earlier (i.e. in 2000, 2001, 2002, 2003, 2004) Students writing matric in 2014 Students passing matric in 2014 Traditional matric pass rate Throughput pass-rate from grade 2 to matric (proportion of grade 2’s 10 years earlier who passed matric)
2010 1,071,053 537,543 364,147 68% 34%
2011 925,761 496,090 348,117 70% 38%
2012 992,797 511,152 377,829 74% 38%
2013 1,087,933 562,112 439,779 78% 40%
2014 1,085,570 532,860 403,874 76% 37%

What causes fewer students to reach matric in one year relative to the previous year? It could be external changes that affect cohort sizes (like the change of age-of-entry policy in 1999). But it can also be through the direct actions and influences of principals, teachers and district officials. Given the attention and emphasis on the matric pass rate (for schools, districts and provinces), there are a number of people who have an incentive to ensure that the matric pass rate goes up, irrespective of how this is achieved.

For example in October 2014 the MEC for Basic Education in Gauteng, Panyaza Lesufi, explained to his district officials that “Any district that drops, even if it’s by 0.01 percent, before you give me the results, put the resignation letter on top.” This kind of approach to the matric pass rate introduces severe unintended consequences like teachers encouraging weaker students to leave school or repeatedly failing them in grade 11. Or, as was the case in some schools in the country, resorting to cheating and conspiring with students. I am all for more accountability in education in South Africa (including in the bureaucracy), but what exactly can a district official do in October to improve the outcomes in matric one month later? By all means set performance targets, but set the right ones. Like measuring the throughput pass-rate from grade 8 to matric, a far more sensible metric to use for accountability purposes (i.e. what proportion of grade 8 students in 2010 passed matric in 2014 relative to the throughput pass rate the previous year?) This is not an especially new argument. Nick Taylor, the CEO of the National Education Evaluation and Development Unit (NEEDU) commenting in 2011 explained the tension between quantity and quality as follows:

“Because the pass rate is a ratio consisting of two numbers—numbers of passes as a fraction of numbers of candidates—it can be improved by changing either or both these quantities. In the period 1999 to 2003 the one that was changed was the number of candidates: fewer children were given the opportunity to write matric whereas the number of passes stayed about the same. The result was that the pass rate went up and the government claimed victory… Ironically, although the 1999 to 2003 period received public approval for its increased pass rate, this was a period of declining quality that was achieved in two ways: encouraging candidates to register at the easier standard-grade level and lowering standards by making the examination papers easier, focusing largely on cognitive skills of an elementary nature at the expense of the higher-order processes of analysis and interpretation. In short, improved efficiency can be achieved by restricting opportunity or by compromising quality, or both, and this is what happened at the time.”

In light of the above statistics, it seems logical to ask not only how many students drop out (answer: 550,000), but also why they drop out. Perhaps the most comprehensive analysis of dropout is the article by Dr Martin Gustafsson titled “The when and how of leaving school.” In it he explains the reasons why South African students drop out, and in which grades they do so. Household surveys show that when youth are asked why they dropped out of secondary school, the four most prominent reasons were (1) lack of financing, (2) wanting to look for a job, (3) failing grades, and (4) pregnancy (for female learners). I would like to focus briefly on the last of these – pregnancy as the major cause of dropout for female students.

In 2010 there were 480,157 female students enrolled in grade 8, but by matric 2014 there were now only 289,795 female students. So 190,362 dropped out between 2010 and 2014. We know from household surveys that that 42% of female students that dropped out of school listed pregnancy as the reason. So a few back of the envelope calculations reveals that 79,952 female students dropped out of school between 2010 and 2014 because they fell pregnant (a figure close to the Department’s own estimates). This cause of dropout for girls is well defined and relatively well understood.

Despite being against policy, excluding pregnant students from school is widely practiced in South Africa, both formally and informally. In 2008 and 2009 school governing bodies at Welkom High School and Harmony High School in the Free State adopted pregnancy policies for their respective schools that allowed for the automatic exclusion of any learner from school in the event of her falling pregnant. In July 2013 the Constitutional Court ruled that this was unconstitutional and that pregnancy policies which exclude pregnant girls from attending class are prima facie a violation of pregnant learners rights to equality, basic education, human dignity and privacy.

While this cause of dropout primarily affects female learners, the fact that it is so well defined and measurable means that it is highly actionable from a policy perspective. There need to be better advocacy campaigns directed at youth – about sex, contraception, teenage pregnancy and the constitutional right to education of pregnant students. But there also needs to be tighter enforcement of policies that prevent unfair discrimination against pregnant girls. Perhaps most importantly the Department needs to develop workable solutions to facilitate the re-integration of school-age mothers after giving birth.

Decreasing pregnancy related dropout is likely to make a large dent in the 550,000 students that drop out of school before matric. Of course there are larger causes of dropout – notably the low quality of primary and secondary education and the lack of vocational opportunities – but these are notoriously difficult to understand and remediate. The fact that so little is done to firstly prevent teenage pregnancy, secondly to support pregnant learners, and thirdly to re-integrate school-age mothers, means that there is huge potential to improve the educational outcomes and life chances of thousands of girls in South Africa, and decrease dropout in the process.

//

For further reading on matric see the following articles:

 

 

Education woes go far deeper than matric pass rate [my Sunday Times article on Matric 2014]

matric2-602x400

[This article first appeared in the Sunday Times on the 11th of January 2014]

It’s at times like these that I sympathise with the Department of Basic Education and Minister Motshekga. Like the Goldilocks problem, it seems that nothing can be ‘just right.’ If the matric pass rate goes up, then standards are falling, but if it goes down then interventions are failing. Yet with the new, more rigorous CAPS curriculum we did expect the 2014 matric results to come down slightly. Yet there are many other problems we should be discussing. This year, as with previous years, not enough official attention was given to the high dropout rate. Of 100 students that started school in 2003, only 48 wrote matric in 2014, 36 passed and 14 qualified to go to university. I’ve been told by some that now is not the right time to talk about this. But when is the right time to talk about dropout? June? September? It’s never comfortable or convenient to talk about half a million children dropping out of school and facing unemployment or menial work – something that happens year in and year out. And lest you think these students are going to FET colleges or vocational training, let’s look at the stats. Household surveys show that only 1% of youths who did not hold a matric certificate held some other non-Grade 12 school certificate or diploma issued by an FET college for example. The rest have no educational qualifications whatsoever. It is highly problematic that around 60% of South African youth end up with no national or widely recognised educational qualification, despite spending a relatively high number of years in education. To be clear, the aim of education should not be to get everyone to matric. Rather we need trustworthy and credible exams at the grade 9 level, and legitimate vocational options with clear occupational roles that students are being prepared for.

This year we were also made aware of a surge in matric cheating with 5305 candidates implicated in 2014, more than ten times as many as in 2013 (473). Furthermore, the findings of ‘group copying’ by Umalusi (the quality assurance body) raises serious concerns about the involvement and complicity of teachers, departmental officials and examinations officers.

Language

Much has already been said in the media about the drop in mathematics performance and the mathematics crisis in South Africa. Let me rather talk about another subject that should be receiving as much attention: English First Additional Language (EFAL). In South Africa students take at least one home language and one first additional (i.e. second) language. EFAL is the largest single subject in matric with 81% of all matric students writing the exam in 2014. One might expect weak performance in this subject given that most international assessments that South Africa participates in show that our students perform two to four grade levels behind their peers in reading literacy. However, the 2014 pass rate for English First Additional Language was 98%. This is largely because EFAL is set at the same standard as all the other First Additional Language subjects which are relatively easy and prioritize communication. Yet, as the 2014 Ministerial Task Team on the NSC identifies, “EFAL does not and cannot fulfil the same purpose in the curriculum as the other 10 First Additional Languages.” This is primarily because the purpose of EFAL for most students is not only communicative efficiency, but also to prepare students to learn all their other subjects in English (their second language) and to prepare them for the world of work. The Task Team report goes on to explain that most of these students are only ‘semi-lingual’ in either their home language or in English. One only needs to look at the EFAL curriculum and the EFAL exams to know why. In 2010 the EFAL exams were reviewed by a number of international benchmarking authorities. The Cambridge International Examinations body concluded that “reliance on testing memorisation and recall, rather than critical thinking and analytical and evaluation skills” was a major problem. The Australian Board of Studies New South Wales did not mince their words when they explained that “The cognitive levels assessed in the examination questions are heavily weighted towards lower-order skills…The grammatical activities themselves are meaningless and reflect a drill and practice approach to language learning which does not support the need to develop students’ language for work and participation in the broader community.” These are the same sentiments that are repeatedly expressed by business leaders and those in higher education institutions in South Africa.

The Task Team report also highlighted the low levels of English proficiency among teachers for whom English is a second language, a severe problem that is widely acknowledged in the research literature. Yet interventions to improve teacher subject knowledge in English are meagre and wholly inadequate. During the course of 2013 South African teachers who have English as a second language had a maximum of three hours of English training, and in four provinces they had none. You do not become proficient in a language with 2-3 hours of training. This is not learning how to play Sudoku. The two main reasons for the low levels of in-service teacher training are firstly that there are so few high-quality training programs available to teachers (none of which are properly evaluated), and secondly that teacher training is seen as too expensive for the Department. This is largely because many teachers, vigorously backed by their union, refuse to attend training courses unless there is additional pay for it. This makes training inordinately expensive. Alternatively the training must happen during school hours, which is basically standard practice across the country (despite it being against policy). Yet all of this is quite ridiculous and unnecessary. All South African teachers are already being paid for 80 hours of professional development per year as part of their existing employment contracts (see Government Gazette, Notice 222 of 18 February 1999, Chapter A, Number 3.2, Section D). Yet nationally representative data show that the average South African teacher spends less than 40 hours on professional development per year?

More questions than answers

We need to end our infantile obsession with the matric pass rate and move on to talking about the real issues affecting education. Poor performance in matric is rooted in weak foundations in grades 1-3. Rather than frown about the two percentage point drop in the pass rate, we should be asking why only one in three students who took maths or science scored above 40% in either subject in 2014? Or why so few take these subjects. Or why 40% of our matrics are taking Business Studies and 20% are taking Tourism, when in reality these are empty subjects that are ill conceived and prepare them for nothing? Researchers at Wits have highlighted this problem before, with Stephanie Allais concluding that, “Vast numbers of our children enrol for semi-vocational subjects that are not teaching them either robust academic skills by building concepts and knowledge, nor preparing them for work in any meaningful way.” Is there any plan to reform these curricula and the way that they are taught? Is there any commitment from the Department that from next year they will report the ‘real’ matric pass rate (the throughput rate from grade 8) in addition to the traditional matric pass rate? No single number can capture the health of our education system, the sooner we realise this the better.

//

Some other links to comments I’ve made on Matric 2014 results:

Q&A with Paula Ensor

Paula Ensor picThe aim of the Q&A series is to get an inside look into some of South Africa’s leading education academics, policy-makers and activists. This is the nineteenth interview in the series. Paula Ensor is Professor of Education at the UCT School of Education.

1)   Why did you decide to go into education and how did you get where you are?

I left school with the intention of becoming an economist. My involvement in political activity, initially as a student and later as an activist in the liberation movement, cut across that. After some years of more or less full-time political engagement I realised I needed to qualify myself for a job, and so going into teaching was initially a fairly pragmatic decision. I trained in London to teach in further education, and taught mathematics for eight years at Botswana Polytechnic (it is now part of the University of Botswana), after that for 18 months in an inner London secondary school, and since my return to South Africa in 1991, I have been involved in higher education, first at UWC then UCT.

2)   What does your average week look like? 

 I have been on sabbatical leave for the past 12 months, as my term of office as Dean of Humanities [UCT] ended in 2013. So my average week this year has been quite unlike any weeks of the previous 10 years! I have spent most of my time reading and writing.

3)   While I’m sure you’ve read many books and articles in your career, if you had to pick two or three that have been especially influential for you which two or three would they be and why?

Karl Marx’s four volumes of Capital probably shaped my thinking more than anything else I have read. It is an extraordinary intellectual accomplishment, in that from the notion of commodities and commodity exchange he builds an analytical framework to account for the workings of the capitalist system and the production and reproduction of inequality. One can be fully mindful of the critiques of this work but still appreciate the brilliance of his argument and of his literary style. It is very difficult to identify other texts which have had the same impact on my outlook. But in terms of thinking specifically about education, I would add Foucault’s Discipline and Punish, Althusser’s Ideology and Ideological State Apparatuses and Fanon’s Black Skins, White Masks.

4)   Who do you think are the current two or three most influential/eminent thinkers in your field and why?

This is a difficult question to answer. For me the most influential thinkers in education are deceased – Piaget, Vygotsy, Marx, Durkheim, Weber, Bourdieu, Foucault and Bernstein, for example. This year I have spent a great deal of time revisiting these foundational theorists, as well as engaging with more contemporary thinkers such as Amartya Sen (he reminds us so well how crucially important education is for development and for human flourishing), Randall Collins (I am interested particularly in his work on ritual), Henri Lefebvre (through her work on pedagogy Heather Jacklin introduced me to him and his work on rhythmanalysis) and Judith Butler (whose work on performativity in relation to gender has helped me to think about pedagogy as performance).

5)   What do you think is the most under-researched area in South African education?

This is a tricky question as one could answer this question in so many ways, depending on the subfield of education one works in – ECD, primary and secondary schooling, further and higher education. I want to get a better grip on how education as a system articulates. Is there any research out there that provides guidance on this? I have been involved (either through research, or teaching) in different levels of the formal education system, from Foundation Phase through to higher education, but it is not clear to me how, and to what extent (if at all), government policy grasps education as a system rather than as a number of quite distinct silos. OBE had a disastrous impact on schooling; the NQF arguably has had a similarly disastrous effect on post-school (vocational and adult) education and Stephanie Allais’s recent book on the NQF and its effects is a must-read in this context. The parlous state of post-school education impacts negatively on schooling, and on higher education. So an interesting question for me (I wouldn’t claim it as the “most” under-researched area) is how (if at all) government policy understands and promotes the interconnection of the system as a whole. At the level of more personal interest, I want to understand better the regularity of pedagogic practice over time and place, and the difficulties of changing this, a question which for the moment I am placing under the working title of “ritual in pedagogy”.

6)   What is the best academic advice you’ve been given?

 To respect the importance of empirical data in educational research, to respect the discipline it imposes, and to understand that working rigorously with data is both demanding and richly enabling.

 7) You have been involved in teacher education at UCT for a long time – in your experience what are the two or three areas that students struggle with most when they become teachers and start teaching in schools?

My PhD focused on a PGCE mathematics method course, the experiences of a group of students on this course and their encounters as they entered schools as beginning teachers. I found that the experiences of beginning teachers, and the challenges they faced, were shaped by three factors – access to the principles which framed the teacher education programme they completed at university; educational biography (that is, their own experiences of schools as learners); and the organisation of the school setting they entered as beginning teachers and the level of support they received there. Effective school governance and ongoing collaboration and support amongst teachers was the most crucial factor in assisting beginning teachers plan curriculum coverage, gain access to materials and other resources, organise assessment and feel confident about issues of control. I have read other work since then, especially in the context of mathematics education, and I think these findings have been broadly confirmed.

8)   If you weren’t in education what do you think you would be doing?

Well, I think I might have enjoyed being an economist. But I would have been very happy as an historian as well.

9)  You have recently returned to the UCT School of Education after being in management at UCT for some years. Have you noticed any changes in the field of education in SA compared to five or ten years ago?

 It is difficult for me to make claims about changes in the field as a whole. With regard specifically to schooling, well, we said goodbye to OBE, which inflicted severe damage on our system, and I think some headway has been made in schools with the development of CAPS and the recognition of the need for high quality textual resources (such as text books and work books) in classrooms. There is now regular, wide-scale assessment (such as ANAs) but is not clear to me what the pedagogic effects of these assessment practices are. In many respects the issues are the same as a decade ago – improving initial and inservice teacher education, strengthening school leadership, changing pedagogic practices in classrooms, understanding better the relationship between home and school, the complexity of linguistic practices in classrooms and so on. So the issues appear to be broadly the same over time, but the precision with which they are articulated seems to be much sharper, and there is a wider array of theoretical resources that researchers are working with.

10)   You have been involved with activist organizations like Equal Education and were yourself part of the anti-apartheid struggle. What do you think is the role of these activist organizations in South Africa and what advice would you give to other similar organizations?

Civic organisations like Equal Education are crucial for mobilising citizens to act in their own interests, and to hold government to account for service delivery across a broad front of issues. An active citizenry is the ultimate defence we have against corruption, cronyism, authoritarianism and the wasteful inefficiency we see so much of at the present time. Civic organisations like Equal Education not only put pressure on government and other agencies to improve the quality of education, but they also build and strengthen civil society in defence of democracy.

11) Technology in education going forward – are you a fan or a sceptic?

I am not sure what you mean by technology in education – pens and pencils are already aspects of pedagogic technology! I assume you mean forms of technology which involve the use of computers and other electronic devices and make use inter alia of the internet and specialised educational packages. These offer new ways of accessing knowledge and new modes of communication which are now deeply embedded in contemporary (globalised) culture. As educators it seems to me that in the end we have no option but to give young people access to the opportunities this technology offers. I am sceptical about claims that use of such technology will miraculously transform teaching and learning – I have seen far too many education technology fads consigned to store rooms and cupboards. But it is an unavoidable part of life in a globalised world and potentially very empowering.

12) If you were given a R20million research grant what would you use it for?

R20 million is a great deal of money and we would want to make sure that such a funded research project would have maximum impact on policy and practice. Having read through some of your earlier interviews I am struck by how much more precise we have become in our diagnosis of educational problems and in identifying areas that require further research, and at the same time how broadly these questions cut across the whole system. So the first thing I would do is bring together a group of the most productive and interesting thinkers and researchers in education in South Africa and map out a project which we collectively believed would make the most impact.

//

Some of Paula’s research can be found here and an extended bio here. Some of the others on my “to-interview” list include Veronica McKay, Thabo Mabogoane, Andrew Einhorn, Maurita-Glynn Weissenberg, Shelley O’Carroll, Carole Bloch, Yael Shalem, Jill Adler, Linda Richter and Volker Wedekind. If you have any other suggestions drop me a mail and I’ll see what I can do.

Q&A with Lilli Pretorius

Screen Shot 2014-12-23 at 4.46.41 PMThe aim of the Q&A series is to get an inside look into some of South Africa’s leading education academics, policy-makers and activists. This is the eighteenth interview in the series. Elizabeth (Lilli) Pretorius is a Professor of Linguistics at UNISA.

(1) Why did you decide to go into education? 

Well, I originally come from Linguistics, not Education, I stumbled sideways into education. I majored in English and Italian at university and although in my early twenties I didn’t know what exactly I wanted to do with English and Italian besides waft around in Literature, I emphatically didn’t want to go into education! It was while I was doing honours in Italian that I encountered linguistics and fell in love with it. From my early encounter with Chomskian phase structure rules in Italian I moved into phonology and psycholinguistics – specifically first language acquisition and studies on language and the mind. I was hooked. In the meantime, I needed a job, and a post in teaching English to Grade 11 and 12 students in a township school came up. I did that for 4 years – and loved it, so I got hooked a second time. I was then offered a junior position in the Linguistics Department at Unisa so I left classroom teaching for academia. My four years of teaching English as a second language in a township high school stood me in good stead in my university teaching as it gave me insight into the daily struggles of students who do their schooling in a language that is not their home language and who come from high poverty homes with very little exposure to print-based literacy practices. Through Linguistics I became interested in how we construct meaning via language – I first studied constructions and perceptions of causality in discourse at Grade 5 level for my Masters, and then looked in the role of inferencing in the reading of expository texts by university students. After my doctoral research I worked with a colleague from Information Science, Myrna Machet, on a project that over a 3-year period took me into deep rural areas, where we piggy-backed on adult literacy NGOs, looking at the effects of storybook reading by neoliterate adults on children at preschool level. I then spent the following decade looking at various aspects of reading development in high poverty contexts, from preschool to university level. It was never a conscious decision to ‘go into’ education – my research lead me into education.

  (2)   What does your average week look like?

Rather messy – a combination of teaching, admin work, meetings, academic stuff – and trying to do research in between. I teach honours modules in Applied Linguistics (second language teaching and learning) and I have quite a few Masters and doctoral students who take up a lot of time. I am also the departmental co-ordinator for about 70 Masters and doctoral students and that takes up a lot of admin. I do article reviews, examine M and D theses, review ethics applications in our college, occasionally play an advisory role in literacy programs. I have collected a lot of reading data over the years but increasing administrative demands in the work place make it challenging to find time for research. I have come to the realisation that it is difficult to manage one’s time effectively in an environment where waves of work continuously crash over you. I’m trying to develop ‘work wave’ stamina and resilience…

(3)   While I’m sure you’ve read many books and articles in your career, if you had to pick one or two that have been especially influential for you which one or two would they be and why?

It’s interesting how some articles stand out among the many thousands one reads in the course of one’s academic career. I remember two articles in particular that affected me quite profoundly in my early research career – Keith Stanovich’s article on early reading development (Stanovich, KE. 1986. Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly 21, 360–406) and Meridith Daneman’s article on individual reading differences (Daneman, M. 1991. Individual differences in reading skills. In: Barr, R. et al. (eds) Handbook of Reading Research Volume II. London: Longman). I was impressed by two things: firstly, they asked different questions about reading. Changing the questions changes the way one looks at a phenomenon. Secondly, I was struck by the coherence of their articles – the clarity of their thoughts, the systematic nature of their arguments, and the rigour and compelling force of research evidence. We tend to associate aesthetic responses to texts with literary texts, but well argued and empirically well supported scientific texts have a beauty of their own too.

(4)   Who do you think are the current two or three most influential/eminent thinkers in your field and why?

Reading is a complex phenomenon, the domain of reading research is very wide and is informed by a range of disciplines – linguistics, psycholinguistics, cognitive science, neuroscience, psychology, sociology, social anthropology, semiotics, education and special needs education. Each of these disciplines brings to reading research their own particular interests and agenda, their own research approaches, and their own gurus. Who one considers important depends on which aspects of reading ‘float one’s boat’, so to speak.

There are so many people who’ve done – and continue doing – amazing work in the field of reading research, at both the micro and the macro levels of processing and in terms of teaching reading. Scholars such as Marilyn Adams, Michael Pressley, Catherine Snow immediately come to mind. Keiko Koda and reading researchers in Canada have done important comparative studies on bilingual decoding skills, Paul van den Broek and Charles Perfetti in the USA and Jane Oakhill and Nicola Yuill in the UK have done interesting work over the years on reading comprehension, Stanovich and Cunningham have done fascinating work on the effects of print exposure on reading, vocabulary and knowledge development in general. The affective dimension of reading (e.g. Rosenblatt) and the broader social context in which literacy is used and enacted are also important areas with their own influential thinkers such as Brian Street, Dave Barton and Paul Gee.

Because of my own interest in language and the mind, a lot of the research that I read about comes from the cognitive and neuroscience disciplines. There has been a lot of really interesting reading research coming from neurolinguistics and the brain sciences in the past 20 years. Stanislas Dehaene’s book Reading in the brain and Marianne Wolf’s Proust and the squid convey this complex research domain in an accessible style to the ordinary reading public.

Because so much of what we know about reading in the past 70 odd years comes mainly from North America and Europe, we need to continuously ask ourselves Ok, so how does this relate to reading and education in developing, middle income countries. There’s a lot we still don’t know.

(5)   What do you think is the most under-researched area in South African education?

Approaches to early reading instruction are based mainly on reading in analytic languages. African languages are agglutinating languages with rich and complex morphosyntactic structures. The Nguni languages in particular have very long word units. At the micro level we actually know very little about what really works when learning to read in the African languages, from a decoding point of view, so that is an area that calls for further research.

Another area that merits closer scrutiny is the development of academic language proficiency in bilingual education systems. There are quantitative and qualitative differences in the ways in which we communicate orally in everyday situations and the way we use language in more formal educational contexts. In written language – whether in paper or electronic form, whether in the home language or a first additional language – the locus of meaning shifts to the text itself, and the ability to construct meaning relies largely on the linguistic and textual cues in the written text and in the conventions associated with its use. In effect, since the advent of modern education, when we acquire our home language or learn another language, we learn oral and written ‘dialects’ or registers, and we switch between them, depending on the context in which we use language. Although oral proficiency is what we rely on most throughout our lives on an everyday basis, it is proficiency in using literate or academic forms of language that determine success in the educational context. Many of our children who pass through the education system acquire only rudimentary reading abilities and are exposed to teaching practices where information is parroted in a superficial manner, so they are thus denied opportunities to develop new ways of using and understanding language. Although they are orally very proficient and may even be marvellously multilingual, if they don’t acquire academic registers they will continue to struggle academically. Finding ways to develop and support academic language proficiency is important in our context.

(6)   What is the best academic advice you’ve been given?

Comments from past professors such as “Mmm, interesting ideas, but where’s your evidence?” or “Re-write this, it reads like a first draft” or “Let’s talk about this once you’ve done more reading”, though mortifying to a tender graduate student or junior lecturer at the time, were valuable academic lessons that helped shape the person I am today. I also had an interest in humour and at an early stage of my academic career I considered doing research on the use of language in humour. My husband said I wouldn’t be taken seriously if I analysed Monty Python scripts and suggested I do something more practical – so I continued with my research on causality in texts instead, and that drew me into reading research.

(7) You have been involved in language education and linguistics for a long time – in your experience what are the two or three areas that students struggle with most when they become teachers and start teaching in schools?

Being in Linguistics, I have not been as directly involved with teacher training as my colleagues in education faculties are. However, having worked in high poverty schools over many years, I’ve come to see how demotivating a dysfunctional school environment can be on teachers. Strong leadership and good governance at a school provide an enabling and supportive environment for novice teachers. It’s also important for new teachers to have as a mentor a teacher in whose classroom learning actually happens, not just someone who’s ‘experienced’ and has been teaching for many years. There are many teachers who have been teaching for years but whose kids are not performing. Who novice teachers have as role models is important.

(8) I recently read your excellent 2014 article titled ‘Supporting transitions or playing catch-up in Grade 4? Implications for standards in education and training.’ It was easily the most informative academic article I’ve read this year. If you were put in charge of a national reading campaign with a large annual budget (R200m) and total discretion on how to spend it, what would you spend it on and why?

The bottom line in reading development is fairly simple – reading only develops through reading. Throughout Africa, children are expected to become readers without having proper access to books and they attend schools where books are rare, and rarely used. This is akin to expecting kids to play football without a ball. This would be unthinkable in the football world. Using the same football analogy, it would be unthinkable in football circles to appoint coaches who knew nothing about football, yet we have teachers whose task it is to develop readers but who themselves do very little reading and who know very little about reading. From questionnaires I’ve administered to teachers in township schools over a 10-year period, I’ve found that around 70% of teachers reported that they have 10 or fewer books in their homes, and although they all have positive attitudes to reading, when asked to name a book they’ve recently read, most teachers give the name of a book prescribed in high school. This suggests that they don’t read voluntarily. This is a sensitive area and not much hard core empirical research has been done on this topic. Although many more teachers these days have better formal teaching qualifications than was the case two to three decades ago, there is quite a lot of indirect or anecdotal evidence of teachers’ poor reading abilities. The NEEDU report of 2012 identified poor content knowledge as one the areas in which teachers struggled. If teachers know very little about their content subject, this means they probably do very little reading in their content subject. So, finding ways to raise teachers’ literacy levels is paramount at this point in our education history.

Screen Shot 2014-12-23 at 5.54.38 PM

One of my favourite posters about reading is this one. Reading awakens possibilities, it gives one independent access to knowledge. So, if I were put in charge of a national reading campaign, I would adopt a two pronged approach to the reading campaign, with one strand focussing on making books accessible to school children and the second strand focussing on building teacher capacity with regard to reading.

Less than 10% of our schools have functional libraries, and this in our 21st century world with its knowledge economy, where most of our knowledge is stored, transmitted and updated via written language. Learners who come from poor homes and communities have little access to books and hence to knowledge beyond their immediate world, so schools in these communities should become rich and stimulating sites of literacy. But simply putting books in schools does not necessarily make reading happen. The affective, social aspects of reading should be promoted, learners should be constantly motivated to read, reading should become a high status activity and reading be made a cool thing to do. Learners should also have strong reading role models and reading and books should be visibly valued – which is where the teacher capacity building strand comes in. The DBE has actually done quite a lot about putting books in schools in the past few years, but the next step, that of maintenance and use, is neglected. I’ve been to many schools that have been given lots of books – and they are often stored away and gathering dust in a room somewhere, unopened and inaccessible. This suggests that schools do not know how to manage resources and how to use books effectively in their teaching. So this strand of the campaign would build up individual teachers’ reading levels, and also teach teachers how to explicitly teach reading strategies (and I’m referring to much more than skimming and scanning here), and how to manage, maintain and use book resources in their schools and classrooms.

(9)   If you ended up sitting next to the Minister of Education on a plane and she asked you what you think are the three biggest challenges facing South African education today, what would you say?

First of all, I’d thank her for arranging this sudden upgrade from economy to first class for me, for how else would I sit next to her? I’d also thank her for candidly acknowledging the reading crisis in our schools. I think she has quite a strong finger on the education pulse in that regard. Then, depending on how long the plane trip was, I’d highlight three challenges facing SA today, all of which revolve around literacy:

  1. Attention to changes in classroom literacy practices. After 70 years of reading research we have a pretty good idea of what works in early reading development and why a balanced approach to reading instruction is important. However, telling teachers what to do about reading and giving them resources with which to do it do not necessarily bring about educational reforms. Teachers’ perceptions, normative beliefs and knowledge influence their classroom practice. In order to get teachers to change ineffective classroom practices we need more research into theories of pedagogic change in schools and classrooms and how this can be done via in-service teacher training. We rely a lot on workshops to do this, but teachers often call these ‘hit and run’ affairs (facilitators come in, tell teachers what to do, then leave). How else can we support teachers in becoming more effective? All interventions should thus be informed by theories of change and be carefully monitored and evaluated so that factors that facilitate or hinder changes in teacher practice can be indentified and better understood.
  2. Attention to literacy in maths and science. In the past 20 twenty years millions of rands have been spent in this country on improving maths and science education, with very few dividends so far. The relationship between reading ability and maths and science learning needs to be given explicit attention. As long as we have low literacy levels in general, we will continue to have low maths and science performance. Students need to be explicitly taught how to read maths and science texts so that they can ‘read to learn’ more effectively from their textbooks.
  3. Attention to school libraries and post structures for school librarians. As mentioned above, very few schools in SA have functional school libraries, especially schools that serve poor communities, which is precisely where ready access to information and knowledge is most needed. Furthermore, school libraries need librarians, people who are specially trained to manage libraries and to help build a culture of reading and finding information. Teachers do not have the time or the expertise to run school libraries. This means that post structures for school librarians should also be created. Low quintile schools cannot afford to appoint school librarians because they would have to sacrifice a teaching post to do so. Although the DBE encourages schools to build up school libraries, in effect it does not provide the infrastructures to enable schools to do so. According to Equal Education’s statistics, by 2010 the state had spent a total of R13.6 billion on building ten new sport stadiums in SA. Surely providing easy access to knowledge via school libraries is a worthwhile investment too? It also helps to professionalise schools.

Just before we landed, I’d ask for more funding for raising teachers’ literacy levels in my national reading campaign. In light of the cost of sport stadiums, R200 million for a reading campaign that has the potential to improve teacher literacy seems rather stingy.

(10)   If you weren’t in education what do you think you would be doing?

 Linguistics? Researching language and humour?

(11)   Technology in education going forward – are you a fan or a sceptic?

Well, there are lots of new and wondrous things happening on the technology front every week – but my default position is that of scepticism. I first want to see evidence of their benefits. The assumption that technology automatically confers advantages is rather naive. Young children in particular need social interaction and mediation in language and literacy development – and also for socio-affective development; technology cannot replace that.

(12) If you were given a R15million research grant what would you use it for?

 I’d really like to explore ways of boosting the development of academic language proficiency during the Intermediate Phase. If children don’t learn to engage with texts and read at a deeper level, then they battle to ‘read to learn’ through the rest of their schooling and it’s really difficult for them to become independent learners. So I would use the funding to explore ways to develop such proficiency, using randomised-control studies with different interventions that are carefully monitored and evaluated. For example, something along the lines of a genre approach to expository/information texts similar to what Dave Rose has been doing in Australia and an intervention that focuses on explicit instruction in reading strategies. Pauline Gibbons has also done interesting work in this area. However, interventions are also only as good as the people who implement them, so the first year would be spent training teachers (language and content subject teachers) in these different approaches, building up their own literacy skills, developing their academic vocabulary and content-specific terminology, as well as their content knowledge and pedagogic knowledge and their understanding of the logic behind the specific interventions. Many teachers attend workshops where they are shown specific methods to use in the classroom, but they have very little understanding of why, what the bigger picture is, and their own pedagogic or literacy skills are not necessarily enhanced by implementing the method.

A list of some of Lilli’s articles can be found below:

Some of the others on my “to-interview” list include Veronica McKay, Thabo Mabogoane, Andrew Einhorn, Maurita-Glynn Weissenberg, Shelley O’Carroll, Carole Bloch, Yael Shalem, Jill Adler, Paula Ensor, Linda Richter and Volker Wedekind. If you have any other suggestions drop me a mail and I’ll see what I can do.

Q&A with Wayne Hugo

Wayne pic

The aim of the Q&A series is to get an inside look into some of South Africa’s leading education academics, policy-makers and activists. This is the seventeenth interview in the series. Wayne Hugo is an Associate Professor at the UKZN School of Education (CV here, personal website here). 

1) Why did you decide to go into education and how did you get where you are?

Four motivations that started the ball rolling – Firstly I was a working class boy with no means to fund higher education and the teaching bursary was very attractive. It actually gave me enough money to help my folks out a little, go out every night for a beer and some pool at the Dev, pay for a small flat in Braamfontein, and buy whatever books I wanted. Secondly, my favorite subjects at school were English and History, so it was easy to make these my majors. Thirdly, I had always loved explaining stuff to my school friends and found I had a knack for it. They are all now making a lot more money than me. Fourth, I have always had a strong desire to write and liked the idea of a half day job that would give me time. Those were the motivations pushing and pulling me into teaching. But then I stumbled into Education Studies at Wits with characters like Joe Muller, Mary Crewe, David Bensusan, Penny Enslin, Ian Moll, Pam Christie, Mike Kissack, Shirley Pendelbury, and Jill Adler. I was pursuing my own curriculum at Wits, spending time in the William Cullen library reading collected works of various luminaries. Joe and Mary Crewe, in particular, actively guided this process, pointing me towards Gramsci, Althusser, Saussure and others, whilst critically taking me on. Education Studies became a place of exploration that allowed me massive freedom along with critical comment. It helped that Lynne Slomimsky was with me for part of this process, as it gave me an intellectual partner who was happy to talk about Laclau and Mouffe, Post Modernism, Freud, dreams and whatever else was brewing. I then left the country to avoid army service and landed up in Wigan as a bouncer, but the massive bursary came back to haunt me as my parents could not afford to pay it back – so back I came into the army and into teaching. I loved teaching, especially as a young teacher, and am writing a book on it to try and capture all the energies and tensions a young male teacher experiences in high school. Anyway, whist teaching I carried on doing my honours and masters part time. A choice quickly arose between shifting into school management or taking a senior lecturing position at the Johannesburg College of Education, and I chose the latter. I will never forget getting my lecturing load and being astonished that I only had to lecture around twice a day. I loved JCE, became the coach of the rugby side, and pretty much carried on as I had at varsity. I did not have a PhD and when friends of mine in Grahamstown needed me to go down there for personal reasons, I decided to do my PhD there. So began a five year journey into the depths of educational thought. I went back to Plato and started to systematically read through all the classical educational texts, making it to Descartes and Hegel. Looking back now there has been an astonishingly consistent daily pattern that has consisted of morning and afternoon reading and writing, late afternoon intense exercise, and evening socialising. After getting my PhD the first post that became available was in Pietermaritzburg. It had academics I deeply respected – Ken Harley, Ben Parker (RIP), Volker Wedekind – so I was pleased to get a post there, and that is where I have settled until now, very happy with a sweet daughter and good friends.

2) What does your average week look like?

Days where I am free – Wake up at 5. Quick check of email, news, facebook. Start writing or reading or both until around 11. Go out for coffee and brunch. Carry on reading, writing and dealing with daily work requirements. Around 4 take the body out to play – go swim, touch rugby, cycle. Around 6 see friends, have a beer, supper, maybe yoga, go to sleep.

My daughter plays havoc with this routine, but then she is worth all of this and more.

3) While I’m sure you’ve read many books and articles in your career, if you had to pick two or three that have been especially influential for you which two or three would they be and why?

Plato’s Republic, Dante’s Divine Comedy

Plato exists for me as an unexplainable divine force at the fount of education. His work is beyond profound, continuously astonishes me, humbles me, enriches me.

Dante exists as a luminosity in my imagination – he is my muse and represents a teacher who was able to swallow his whole age and transmutate it into a pedagogic journey from the depths to the heights in a poem.

Below these two are Kant critiques, Hegel Phenomenology and Logic, Marx’s Kapital (vol1) and Piaget’s corpus.

The educational text that blew my mind open was Bourdieu and Passeron’s Reproduction in education, society and culture. I loved its axiomatic style that went back to Spinoza. Bernstein has been a growing love affair that has formatted my thinking.

4) Who do you think are the current two or three most influential/eminent thinkers in your field and why?

Joe Muller – Lucid thinker where others are struggling to find a path, nevermind understand it. My experience of him is that he is always on the boundary, pushing towards the unspoken, but able to articulate what the new is and why it is relevent. He is very generous with his insights and recommendations. Almost always I come away from a meeting with him with new thoughts and breakthroughs.

Karl Maton – Brave man with enormous energies and ambitions, complimented by one of the sharpest intellects I have come across. He is quickly developing a comprehensive analytical framework that is process oriented, research driven, and gets to the dynamics of education in ways that help students get to grips with curriculum and pedagogy. Legitimation Code Theory combines field insights of power with an internal analysis of educational events. It pushes Bernsteinian binary into process and flow.

5)   What do you think is the most under-researched area in South African education?

 We are way behind in Mind, Brain and Education research, and I feel this will give us strong insights into improving how learning works. We are clueless about Instructional Design, especially how it is currently playing out with new developments in technology and engineering. We are struggling with Big Data analysis. My basic take is that the social sciences have shifted sharply closer to the STEM subjects (Science, Tech, Engineering, Maths), and that education is not following suit. We need to strengthen our research by using insights and methods taken from Engineering, Economics, Computer programming, Biology, Geography and other ‘complexity’ sciences.

6)   What is the best academic advice you’ve been given?

From my dad – always act from a position of strength inside yourself. I have taken this to mean that you have to work at knowing so much more than what is expected. It is astonishing how an expanding and deep network consolidates and clarifies whatever it is you are working with. Everything lights up and is of interest because it places itself inside a glowing network.

 7) You are actively involved in teacher education at UKZN – in your experience what are the two or three areas that students struggle with most when they become teachers and start teaching in schools?

The transference relation between young teacher and learner. There are massive energies and emotions that cook under the surface when you start as a young teacher with learners who are only a couple of years younger than you. It’s a very difficult dynamic to talk about with students, because the unconscious works in such powerful and strange ways. Often naming and studying the process only makes it harder to control when encountered.

Over work. Young teachers are exploited and find themselves quickly stretched and exhausted, resulting in burn-out, drop out, and compromised standards

Too much coffee, too much alcohol

8)   If you weren’t in education what do you think you would be doing?

Education allows me the space to do all the things I want to do, so it’s hard to answer. I suppose a full time novelist. I enjoy trickery, so a magician would have been nice. At one stage I liked the idea of being a Psychoanalyst. A secular Monk of the Franciscan type would be my ideal.

9)   Technology in education going forward – are you a fan or a sceptic?

Big fan. I get the over hype and the over expectations and the dangers, but it is taking us into new spaces. It’s the connectivity and creativity it enables that astonishes me. Often we use technology just to replicate existing practices, but it is breaking these practices, changing them, moving us into a new terrain. Technology innovates faster than culture can keep up with, it is our job as the current generation of education academics to stay on the wave, learn to surf it, and articulate what its great strengths and dangers are. I love the post human stuff of Harraway and Crew, feel it speaks to a new mode of existence arising in and through us.

10) If you were given a R20million research grant what would you use it for?

Certainly not whole school improvement. I would like to set up an education website similar to Khan Academy but in education and then track and trace student responses in a way that improves the curriculum, pedagogy and assessment of the website material. The website would give the basic building blocks that specialise a student into education and provide a dialogic space for interaction and engagement that can be used across South Africa, outwards and onwards.

//

A full list of Wayne’s publications can be found here. He has also done a video lecture series on “Cracking the code“

Some of the other academics on my “to-interview” list include Veronica McKay, Elizabeth Pretorius, Jill Adler, Paula Ensor and Volker Wedekind. If you have any other suggestions drop me a mail and I’ll see what I can do.

Special issue of SA Journal of Early Childhood Education: Call for abstracts

call for abstracts

The relatively new open-access SA Journal of Childhood Education has recently put out a call for abstracts (see below) for their special issue on “Priorities and policy-making in South African Education” (Guest editors: Nick Taylor and Thabo Mabogoane). Given the policy relevance of this special issue, researchers at ReSEP (including myself) will be submitting a number of abstracts for work we are currently doing and intend to do. If you’re doing work in this field I’d encourage you to do the same, it’s likely to be a great issue!

//

Special issue: Call for Papers “Priorities and Policy-making in South African Education”

 Guest editors: Nick Taylor and Thabo Mabogoa

Despite considerable expenditures and efforts to improve performance and reduce inequality, there is limited evidence of substantial improvements in educational outcomes, or the equalisation thereof. Periodic reviews of the evidence have shown a number of recurring themes that are especially characteristic of schooling in South Africa. These include unequal access to socially, emotionally and cognitively stimulating environments (both in the home and at school), insufficient resources, low levels of curriculum coverage, low levels of teacher content knowledge, inadequate support and training opportunities for in-service teachers, challenges associated with learning and teaching in a second language, low levels of accountability and high dropout in upper secondary school (among many others).

While education officials are often aware of these challenges, most policy-makers find it difficult to synthesise this evidence, which is necessary for prioritisation and resource allocation. Making sense from research is particularly challenging  when it is presented in isolation from other problem areas and only speaks to other research within its ‘silo’.  It is now widely acknowledged that if government policies are to have the largest possible impact, they need to be based on rigorous evidence and peer-reviewed research. Furthermore the National Development Plan (the government’s guiding framework) has emphasised the need for the “process of prioritisation and sequencing” if the plan is to be implemented. Such a process of prioritisation and sequencing requires rich, inter-connected evidence on education in South Africa.

Consequently, this call for papers focuses on education research in South Africa that speaks directly to policy-making and prioritisation. Papers that synthesise existing evidence across research areas in education are especially welcome.

Instructions for authors: www.sajce.co.za

Journal administrator: childhooded@uj.ac.za

Online submissions and author registration: www.sajce.co.za

Deadline for abstract submission: 31 March 2015
Deadline for full papers (of accepted abstracts): 30 June 2015
Intended publication date: November 2015

The SAJCE is accredited by the Department of Higher Education and Training and  has applied, through the Academy of Science of South Africa  (ASSAF) for listing  on the open journals platform, ScIELO 

“Assessment results don’t add up” (my M&G article on the ANAs)

ANA 2015

(The article below first appeared in the Mail & Guardian on the 12th of December 2014. The original link can be found here.)

Last week, the minister of basic education announced the results of the Annual National Assessments (ANAs) for 2014. The ANAs test all children in grades one to six and nine, using standardised tests in mathematics and languages.

The problem is that these tests are being used as evidence of “improvements” in education when the ANAs cannot show changes over time. There is absolutely no statistical or methodological foundation to make any comparison of ANA results over time or across grades. Any such comparison is inaccurate, misleading and irresponsible. The difficulty levels of these tests differ between years and across grades, yielding different scores that have nothing to do with improvements or deteriorations necessarily, but rather test difficulty and the content covered.

Although the department of basic education tries to make the tests comparable across years, the way it goes about doing this (with teachers and experts setting the tests) means that in reality they are not at all comparable.

And the department knows this. On page 36 of its 2014 report, it states: “Even though care is taken to develop appropriate ANA tests each year, the results may not be perfectly comparable across years as the difficulty level and composition of the tests may not be identical from year to year.” Yet it then goes on to make explicit comparisons.

You can’t have it both ways. I can say categorically that the ANA tests are not at all comparable across years or grades. Despite this cloaked admission of incomparability, the report is full of the rhetoric of comparison, with scores reported side by side for 2012, 2013 and 2014, and 24 references to “increases” or “decreases” relative to last year’s ANA. Similarly the minister in her speech last week Thursday spoke about “consistent improvement in home language” as well as “an upward trend in performance”.

All of these statements are extremely misleading and factually incorrect. The ANAs cannot be compared across grades or years, at least not as they currently stand.

Those of us in the field of educational assessment have been saying this repeatedly for two years. Yet journalists continue to regurgitate these “increases” and “decreases” without any critical analysis, as if they must be true – but they are not. There are different ways of determining whether the quality of education is improving (primarily by using reliable international assessments over time) but the ANAs, in their current form, are not among them.

For tests to be comparable over time, one has to employ advanced statistical methods – for instance, item response theory, which essentially involves using some common questions across tests allowing us to compare performance on the common questions with performance on the noncommon questions within and between tests. This makes it possible to equate the difficulty of the tests (and adjust results) after they have been written. The common questions must also be used across grades and the ANA cycles.

This is standard practice around the world, and yet is not employed with the ANAs. Every single reliable international and national assessment around the world uses these methods if they intend to compare results over time or grades, but not the ANAs. There are no common questions used across any of the ANAs, either grade to grade within one year of an ANA, or between the ANA cycles. Using the ANA results to talk about “improvements” or “deteriorations” has no methodological or statistical justification whatsoever.

There is not a single educational statistician in the country or internationally who would go on record and say that the ANA results can be used to identify “improvements” or “deteriorations” over time or across grades.

Although the ANA report speaks about an “advisory committee” of “local and international experts”, it does not name them. These experts need to come forward and explain why they believe these tests are comparable over time, and if they do not believe they are comparable over time then the report should not refer to them.

On this matter, no one needs to take my word for it: the changes in results are so implausible that they speak for themselves. Take grade one mathematics, for example, where the average score was 68% in 2012, plummeted to 59% in 2013 and then soared to 68% in 2014. Very strange. Or, if we look at the proportion of grade three students with “acceptable achievement” (50% or higher) in mathematics, we have the fastest improving education system in recorded human history. The results went from 36% in 2012 to 65% in 2014. These changes are, educationally speaking, impossible.

Some of the provincial results are equally ridiculous. The average score for grade four home language in Limpopo doubled in two years, from 24% in 2012 to 51% in 2014. Given that the standard deviation for grade four home language in ANA 2012 was 26.5%, this amounts to a one-standard deviation increase in two years. For those who don’t know how large this is, it’s the same as the difference between township schools and suburban schools (mainly former Model C schools) in the 2011 study best known as “prePirls” (pre-Progress in International Reading Literacy Study), which recorded 0.9 standard deviations. There are clearly miracles happening in Limpopo.

I could go on and on and talk about other ridiculous changes, such as the national grade six mathematics average (from 27% in 2012 to 43% in 2014), grade five home-language increases in the North West (from 26% to 58%) or grade three mathematics increases in Mpumalanga (from 36% to 50%) or grade four home-language increases in KwaZulu-Natal (from 38% to 58%), and so on. These are all absolutely and unequivocally impossible, and have never been seen on a large scale anywhere in the world before. Ever.

Testing can be an extremely useful way to monitor progress and influence pedagogy and curriculum coverage, but only if it is done properly. Testing regimes usually take between five and 10 years to develop before they can offer the kinds of reliability needed to make claims about “improvement” or “deterioration”.

Test results send strong signals to students and teachers about what constitutes acceptable performance and whether things are improving or not. For example, the department assumes that 50% on the ANAs re-presents competent performance, but there is no rational basis for using this threshold as conceptually equivalent to “acceptable achievement”.

The overall decline in ANA achievement between grade one and grade nine is also extremely misleading, because it suggests that the problem lies higher up in the system. But all research shows that children are not acquiring foundational skills in grades one to three and that this is the root cause of underperformance in higher grades.

Testing children is a serious business that requires large teams of highly skilled professionals whose sole responsibility is to ensure the reliability and validity of the ANA results and process. This includes building a large bank of questions across grades, learning outcomes and subjects. It involves setting and moderating tests; linking and analysing test questions using item response theory; as well as reporting and disseminating results in ways that principals, teachers and parents understand. It needs intense collaboration across the curriculum and assessment branches of government and with those who develop the department’s workbooks. It requires a much longer planning, piloting and reporting cycle than the impossible time frames to which departmental officials are subject.

Let me be clear: the ANAs should not be scrapped – they are one of the most important policy interventions in the past 10 years. However, the first rule in educational assessment, as in medicine, is: “Do no harm.” Sending erroneous signals to teachers and students about “improvements” is extremely unhelpful. This makes it so much more difficult to really induce the improvement in behaviour at the classroom level that is central to real advances in learning outcomes.

In essence, the department needs to answer this: Are the ANA results comparable over time and across grades? If not, why are they being used as evidence for claims about “improvements” or “deteriorations” across grades or over time?

//

Previous posts on the ANAs:

Starting Behind and Staying Behind: Insurmountable learning deficits in mathematics (new Working Paper)

Screen Shot 2014-12-11 at 10.44.44 AM

A Working Paper that I co-wrote with Janeli Kotze was released today on the Stellenbosch Economic Working Paper site (available here). The paper has also been accepted for publication in the International Journal of Educational Development and should be out next year. I will include some excerpts from the paper for those who’d prefer the short version…

Abstract:

This study quantifies a year’s worth of mathematics learning in South Africa (0.3 standard deviations) and uses this measure to develop empirically-calibrated learning trajectories. Two main findings are, (1) only the top 16% of South African Grade 3 children are performing at an appropriate Grade 3 level. (2) The learning gap between the poorest 60% of students and the wealthiest 20% of students is approximately three Grade-levels in Grade 3, growing to four Grade-levels by Grade 9. The paper concludes by arguing that the later in life we attempt to repair early learning deficits in mathematics, the costlier the remediation becomes.

Excerpts:

Few would argue that the state of mathematics education in South Africa is something other than dire. This belief is widespread among academic researchers and those in civil society, and is also strongly supported by a host of local and international assessments of mathematical achievement extending back to at least 1995 (Howie & Hughes, 1998; Reddy, 2006; Fleisch, 2008; Spaull, 2013; Taylor et al., 2013). Many of these studies, and particularly those that focus on mathematics, have identified that students acquire learning deficits early on in their schooling careers and that these backlogs are the root cause of underperformance in later years. They argue that any attempts to raise students’ mathematical proficiency must first address these deficits if they are to be successful (Taylor et al., 2003). The present study adds further evidence to this body of work by using nationally representative data to provide some indication of the true size and scope of these learning deficits.

…

In South Africa, research in this area has generally focussed on in-depth localized studies of student workbooks and classroom observation (Ensor et al., 2009). For some examples, Carnoy et al. (2012) observe mathematics learning in Grade 6 classrooms from 60 schools in one South African province (North West) and compare these classrooms to 60 schools in neighbouring Botswana. On a smaller scale, Venkat & Naidoo (2012) focus on 10 primary schools in Gauteng and analyse coherence for conceptual learning in a Grade 2 numeracy lesson. Similarly Schollar (2008) conducted interviews and classroom observations as well as analysed a large sample of learner scripts to determine the development (or lack thereof) of mathematical concepts through the Grades.

Where the present research differs from these earlier studies is that it focuses on quantifying national learning deficits in general, rather than in specific learning areas. While the latter are essential for understanding what the problems are and how to fix them, analyses at the national level are also needed if we are to understand the extent and distribution of the problem, both of which are imperative for policy-making purposes. This is only possible by analysing multiple nationally-representative surveys of student achievement, which is the focus of the present study. The two core research questions that animate this study are as follows:

  • How large are learning deficits in South Africa and how are they distributed in the student population?
  • Do learning deficits grow, shrink or remain unchanged as students progress to higher Grades?

To answer these questions we analyse four nationally representative datasets of mathematics achievement, namely: (1) the Systemic Evaluation 2007 (Grade 3), (2) the National School Effectiveness Study 2007/8/9 (Grade 3, 4 and 5), (3) the Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ) 2007 (Grade 6), (4) and the Trends in International Mathematics and Science Study (TIMSS) 2011 (Grade 9).

…

The extant research on mathematics learning in South Africa strongly supports this conclusion with numerous researchers highlighting the inadequate acquisition of basic skills and the consequent negative effects on further learning. Taylor & Vinjevold (1999) summarise the findings from 54 studies[1] commissioned by the President’s Education Initiative and conclude that:

“At all levels investigated by [The President’s Education Initiative], the conceptual knowledge of students is well below that expected at the respective Grades. Furthermore, because students are infrequently required to engage with tasks at any but the most elementary cognitive level, the development of higher order skills is stunted” (Taylor & Vinjevold, 1999, p. 231).

This lack of engagement with higher order content is the prime focus of Reeves and Muller’s (2005) analysis of Opportunity-to-Learn (OTL) and mathematics achievement in South Africa, where OTL is the curriculum actually made available to learners in the classroom. Taylor et al. (2003, p. 129) in their book Getting Schools Working summarise succinctly the debilitating effects of cumulative learning deficits:

“At the end of the Foundation Phase [Grades 1-3], learners have only a rudimentary grasp of the principles of reading and writing … it is very hard for learners to make up this cumulative deficit in later years … particularly in those subjects that … [have] vertical demarcation requirements (especially mathematics and science), the sequence, pacing, progression and coverage requirements of the high school curriculum make it virtually impossible for learners who have been disadvantaged by their early schooling to ‘catch-up’ later sufficiently to do themselves justice at the high school exit level.’

And lastly, Schollar (2008) summarises the findings of the Primary Mathematics Research Project which looked at over 7000 learners from 154 schools in South Africa and concludes as follows:

“Phase I concluded that the fundamental cause of poor learner performance across our education system was a failure to extend the ability of learners from counting to true calculating in their primary schooling. All more complex mathematics depends, in the first instance, on an instinctive understanding of place value within the base-10 number system, combined with an ability to readily perform basic calculations and see numeric relationships … Learners are routinely promoted from one Grade to the next without having mastered the content and foundational competences of preceding Grades, resulting in a large cognitive backlog that progressively inhibits the acquisition of more complex competencies. The consequence is that every class has become, in effect, a ‘multi-Grade’ class in which there is a very large range of learner abilities and this makes it very difficult, or even impossible, to consistently teach to the required assessment standards for any particular Grade. Mathematics, however, is an hierarchical subject in which the development of increasingly complex cognitive abilities at each succeeding level is dependent on the progressive and cumulative mastery of its conceptual frameworks, starting with the absolutely fundamental basics of place value (the base-10 number system) and the four operations (calculation)” (Schollar, 2008, p. 1).

…

To provide an alternative measure of performance, we provide two examples of no-language items in NSES and show when students answer the question correctly – i.e. in Grade 3, Grade 4, Grade 5 or not by the end of Grade 5. Given that one needs to follow the same students from Grade 3 to 5 we limit the sample here to the panel sample of NSES students (8383 students). Figure 3 below shows a simple question testing two and three digit addition with no carrying. This is within the Grade 3 curriculum which states that students should be able to “perform calculations using the appropriate symbols to solve problems involving addition of whole numbers with at least three digits.” Although this is a Grade 3 level item and contains no language content, only 20% of Quintile 1-4 students could answer this correctly in Grade 3, with the proportion in Quintile 5 being twice as high (42%) but still low. While there is evidently some learning taking place in Grade 4 and 5, more than 40% of Quintile 1-4 children still could not answer this Grade 3 level problem at the end of Grade 5. In Quintile 5 this figure was only 22%.

Figure 3

Screen Shot 2014-12-11 at 11.29.06 AM

Figure 4 below shows a similar situation where the vast majority of Grade 3 children cannot answer this Grade 3 level problem. While some children learn the skill in Grade 4 or 5, the majority of children still cannot answer this problem at the end of Grade 5, despite it being set at the Grade 3 level.

Figure 4

Screen Shot 2014-12-11 at 11.30.35 AM

 Moving from learning deficits to learning trajectories

While the previous sections have identified the proportion of students that are not operating at a Grade 3 level, they do not provide much guidance in terms of learning trajectories into later Grades. The figures above show that some students are only learning part of the Grade 3 curriculum in either Grade 4 or Grade 5 and that many never seem to acquire these skills. However one cannot say to what extent they are also acquiring Grade 4 level skills in Grade 4 and Grade 5 level skills in Grade 5, although this is unlikely. This is because the NSES test was set at a Grade 3 level with only a small number of questions set at the Grade 4 level. One could use SACMEQ (Grade 6) and TIMSS (Grade 9) as measures of mathematical proficiency at higher levels, but these tests are not calibrated to be comparable to each other, or to earlier tests like the NSES. This is problematic since learning trajectories require data points distributed across the full range of educational phases which are comparable to each other both in terms of the content tested and the difficulty level of the tests. One alternative method to partially overcome the lack of inter-survey comparability is to measure the size of learning deficits in each data set using intra-survey benchmarks.

…

Applying the above method we calculate the difference in average achievement between Quintiles 1 (poorest 20% of students) and quintile 5 (wealthiest 20% of students) for the different surveys and then convert these into a common standard-deviation metric. The difference between quintiles 1 and 5 is 28 percentage points in NSES Grade 3, 130 SACMEQ points in Grade 6, and 122 TIMSS points in Grade 9. These different metrics are not directly comparable and there is no simple way of equating the scores. Consequently we convert the differences into within-survey standard deviations and then, using the 0.3 standard deviation benchmark as one year of learning, one can say that this difference was equal to 4 Grade-levels in Grade 3[1] (NSES), 4.4 Grade-levels in Grade 6 (SACMEQ) and 4.7 Grade-levels in Grade 9 (TIMSS).

Lewin (2007) provides a useful conceptual model for the trajectory needed to reach a particular goal – in this case matric (Grade 12). He refers to an ‘on-track-line’ and an ‘off-track-line’ where the off-track-line is any line below the on-track-line. In the present example, the on-track-line is calibrated to be equal to the average performance of Quintile 5 students.

To illustrate the above in a graph, we set the average Quintile 5 achievement to be equal to the Grade-appropriate benchmark such that the learning trajectory of these students are on the “on-track” trajectory and will reach matric (Grade 12) performing at roughly a Grade 12 level. We then calculate the difference between this ‘benchmark performance’ and the average performance of Quintiles 1, 2, 3 and 4 and then convert this difference into Grade-level equivalents using 0.3 standard deviations as equal to one Grade-level of learning. In doing so, we essentially create a learning trajectory spanning from Grade 3 (NSES) to Grade 9 (TIMSS) with linear projections for those Grades where we do not have data (Grade 7, 8, 10, 11 and 12). The exact figures for all calculations are provided in the online appendix. Figure 6 below shows the likely learning trajectories of the average student in each quintile of student socioeconomic status.

Figure 6 shows that the average student in Quintile 1, 2 and 3 is functioning at approximately three Grade-levels lower than the Quintile 5 benchmark in Grades 3, 4, 5 and 6. Observing average performance by quintile in Grade 9 shows that the difference between Quintile 1, 2 and 3 students and Quintile 5 students (the benchmark) has now grown to more than four Grade-levels. If it is assumed that Quintile 5 students in Grade 9 are functioning at roughly a Grade 9 level, then Quintile 1 and 2 students are functioning at roughly a Grade 4.5 level in Grade 9. The trajectory lines, one for Quintile 5 and one for the average of Quintiles 1-4, show that in Grade 3 there already exist large differences in performance (approximately three Grade-levels) and that by the time children enter Grade 9 this gap in performance has grown to about four Grade-levels. The linear trend in performance between these two groups suggests that if the same number of students in Quintiles 1-4 in Grade 9 continued in schooling until Grade 12 (i.e. no drop out between these two periods) they would be functioning at approximately 4.9 Grade levels lower than their Quintile 5 counterparts in Grade 12 (1.5 standard deviations lower).

Figure 6

Screen Shot 2014-12-11 at 10.44.44 AM

…

Returning to Lewin’s (2007) notion of an “on-track” progress line, perhaps the most important conclusion arising from this conceptual framework is that any performance below the “on-track” line creates an increasing gradient of expectation as the pupil moves into higher grades. This expectation is what is required by the curriculum to reach the goal (passing the grade 12 exam, for example) relative to where the student is at the present. As students’ learning deficits grow, the gradient of what needs to be achieved to reach the goal then progressively steepens to the point where it enters what Lewin (2007, p. 7) refers to as a ‘Zone of Improbable Progress.’ For example, the improvement that is required to bring the average Grade 9 Quintile 1 student in South Africa up to the required benchmark by Grade 12 is unrealistic given that they are performing at roughly a Grade 5 level in Grade 9. By contrast, the gradient of achievement required to bring the average Quintile 1 Grade 3 pupil up to the required benchmark by matric is slightly more manageable. The clear conclusion arising from this analysis is that intervening early to correct and prevent learning deficits is the only sustainable approach to raising average achievement in under-performing schools.

What we would add to this conclusion is that the root cause of these weak educational outcomes is that children are acquiring debilitating learning deficits early on in their schooling careers and that these remain with them as they progress through school. Because they do not master elementary numeracy and literacy skills in the foundation and intermediate phases, they are precluded from further learning and engaging fully with the Grade-appropriate curriculum, in spite of being enrolled in school. Lewin (2007, p. 10) refers to these children as ‘silently excluded’ since they are enrolled and attending school but learning little. Importantly, these children are precluded from further learning, not because of any inherent deficiency in their abilities or aptitudes, but rather because of the systematic and widespread failure of the South African education system to offer these students sustained and meaningful learning opportunities. Indeed, many children from poorer backgrounds have both the ability and the desire to succeed, and when provided with meaningful learning and remediation opportunities, do in fact succeed (see Spaull et al, 2012 for an example).

…

The clear policy recommendation which proceeds from these findings confirms what is becoming increasingly accepted, that any intervention to improve learning in South Africa needs to intervene as early as possible. Given South Africa’s egregiously high levels of inequality, it should come as no surprise that poor children in South Africa find themselves at a nexus of disadvantage, experiencing a lack of social, emotional and cognitive stimulation in early childhood. These children then enter a primary school system that is unable to equip them with the skills needed to succeed in life, let alone to remediate the large learning deficits they have already accumulated to date.

…

When faced with limited resources and a choice of where to intervene in the schooling system, the counsel from both the local and international literatures is unequivocal; the earlier the better. The need to focus on the primary Grades, and especially the pre-primary years, is not only driven by the fact that underperformance is so widespread in these phases, but also because remediation is most possible and most cost-effective when children are still young (Heckman, 2000). Due to the cumulative negative effects of learning deficits – particularly for vertically-integrated subjects like mathematics – it is not usually possible to fully remediate pupils if the intervention is too late (i.e. in high school), as too many South African interventions are. Nobel Laureate Professor James Heckman summarises the above succinctly when he explains that:

“Policies that seek to remedy deficits incurred in early years are much more costly than early investments wisely made, and do not restore lost capacities even when large costs are incurred. The later in life we attempt to repair early deficits, the costlier the remediation becomes” (Heckman, 2000, p. 5).

Full paper available here.

Tonight I am angry

disability 2

It’s currently 9:28pm and I’m sitting on my hotel bed in Pretoria oscillating between anger and tears. Today was the first day of a UNICEF conference on Early Childhood Development and I was presenting on insurmountable learning deficits (PPT here). It’s strange that I feel so emotional because it was an extremely successful day of presentations from people like Marie-Louise Samuels, Linda Biersteker, Linda Richter and Mark Tomlinson – all giants in the field of ECD. And yet, here I am clenching my jaw, crying, listening to the aircon and wondering how we got here. Fuck. I keep thinking of one line from Neruda’s poem:

“Sometimes I wake up early and even my soul is wet”

I find it really difficult to hear about the lived reality of people with disabilities. After the conference closed today I had a long conversation with Jean Elphick, an inspiring woman who works with disabled children in a township in Gauteng. She told me that most disabled children (51%) on their program stay at home each day and only a small proportion go to an appropriate school (watch this 4-minute video from Afrika Tikkun). They have been trying to get these kids into schools for three years, but with very limited success. The part that really hit me was when she told me that 15 of the disabled children she works with had been abused and some had been raped. Fifteen. Boys and girls. The children that stay at home unsupervised are especially vulnerable since they have no sexuality education and limited people outside the home to disclose to or confide in. Hardly any of the sexual abuse/rape cases of these children are ever heard in court and when they are rarely result in convictions. She told me about rape cases where there was even DNA evidence and yet the case had been shelved.

She told me about one disabled girl who went to the police station to open a case of rape against her assailant. The policeman told her that the bleeding from the rape was just her period and the doctor doing the J88 exam refused to record the anal rape. She had been vaginally and anally raped by this man. This child’s assailant was never convicted of rape. And this is not an isolated incident. Last year the M&G covered the issue and reported on one home for disabled children “Ikhaya Loxolo”, I include an excerpt from the article:

“In 2011, Ikhaya Loxolo had 10 residents, nine of them female. They were between 10 and 22 years old. “All nine had been raped previously – some repeatedly,” says Gunther. “The worst thing is that it happens so often that it’s normal to the community. It’s what happens to a mentally disabled girl.” One of the girls was raped when she fetched water from a spring near her home. She was 10 at the time. Another, a 14-year-old, was raped when she went home for the Christmas holidays. “There was no one to look after her and a drunk guy came into the house,” says Gunther. “He locked the door and raped her.”…”Rape is like a plague here. A lot of women and girls get raped but it’s especially the mentally disabled girls. These men know very well that these girls are mentally handicapped. That is why they target them, because they’re easy prey: they can’t fight back and mostly they can’t identify their attackers.” When a mentally disabled girl is raped in the district, says Gunther, her parents are “sad” about it, “but not shocked, because it happens all the time.”

Screen Shot 2014-12-10 at 8.11.56 AM

Screen Shot 2014-12-10 at 8.12.08 AM

Screen Shot 2014-12-10 at 8.12.54 AM

(The above are excerpts from Jean Elphick’s Keynote address at “The South African Professional Society on the Abuse of Children” – PDF available here)

Last night I heard another tragic story about a disabled illiterate person. My good friend (and hero) Veronica McKay was telling me about some of the people that take adult basic education classes (ABET) called Kha Ri Gude. The program aims to provide basic numeracy and literacy skills to illiterate and innumerate adults and is available across the country. As part of the drive to recruit participants they go door to door in the community and ask if anyone wants to join the classes. She told me that often the disabled men and women said that they were locked away in a room and only brought out and dressed up when the disability grant was collected each month. This is one of the reasons why some said they enjoyed the adult literacy classes – for some it was their only social interaction during the week.

At the beginning of the KRG program they ask participants to list what they want to gain, or be able to do, by the end of the program. This helps them tailor the classes to the needs of those participants. Some people say they want to be able to read the Bible, others that they want to open a bank account and others that they want to use an ATM. One of the illiterate participants (who was also disabled) reported that after the program she went to the ATM herself and withdrew the money from her disability grant, where previously her grandchildren did this for her. She reported that now she was receiving much more money. Basically she realised that her grandchildren were keeping some money for themselves and she didn’t know it because she was innumerate and illiterate. Veronica told me that after the course many of the participants said they felt much more empowered and confident to go outside. One of the blind Kha Ri Gude trainers, who now trains others who are blind and illiterate (using Braille) reflected on the KRG program and said: “Now people take me as a role model and they believe that if you give someone a chance then they can do what they want to do. Many people in Kha Ri Gude say to me, ‘Thank you, you revived my life.’

On a different occasion Veronica told me about one previously illiterate woman who, when asked why she did the course said, “Because now I know when I have enough money to go to the shop to buy things. Also if the shopkeeper gives me the correct change. Before I just had to hold out my hand with the money and the shopkeeper would take the money and give me back the change. But I couldn’t tell if it was the right change. I think he wasn’t always giving me the right change. Now I can tell.”

In keeping with the national tone of reflecting on the one year anniversary of  Madiba’s death, I include a quote from the Unifier:

“We have tried to give special emphasis to the rights of people living with disability. It is so easy to think of equality demands with reference primarily to race, colour, religion and gender, and to forget, or to relegate to secondary importance, the vast discrimination against disabled persons” – Nelson Mandela (Message to the Conference for the Disabled, 4 April 2004).

 //

NEEDU Grade 5 Reading Report 2013 (excellent!!)

ele

Today the Minister of Basic Education Mrs Angie Motshekga released the results of the Annual National Assessments for 2014 (report here, speech here). This is not my post on the ANA results – I will write an article for next week’s M&G for that. (However, if it were my blog on the ANAs I would talk about how Grade 9’s abysmal results are rooted in Foundation Phase and the early grade ANAs are not accurate reflections of learning. O, ya, and also that the ANASs are absolutely, categorically and unequivocally NOT comparable year-on-year). But as I said, this is not my blog on ANAs.

Instead it is my blog on the Minister’s excellent choice to focus on reading. In her speech today she devoted considerable attention to the importance of reading and highlighted the numerous initiatives that the DBE has undertaken to improve the state of reading in South Africa (see page 20 and 21).

The Minister also makes extensive reference to the 2013 NEEDU Grade 5 Reading Report (draft). Together with Lillie Pretorius’ excellent article, this NEEDU report was the best thing I’ve read on reading all year. To give an overview of the report let me quote from the introduction:

“This report begins with a brief discussion of literacy and the complexity of reading and reading instruction. It gives a short explanation of the difference between decoding and comprehension and the importance of oral reading fluency for understanding and interpreting what is being read. The report outlines the importance of reading norms, and in particular reading norms for a country like South Africa with the large majority of its early readers reading in a second language. Finally, before the NEEDU Grade 5 reading data is presented, the recent and current national strategies and interventions to improve learner reading proficiency are tracked, suggesting that the crises in reading in South Africa is not new, is not unknown, yet persists” (p 4).

I have argued elsewhere that I believe we need to adopt a national education goal in South Africa; “Every child must read and write fluently by the end of grade 3.” Anyone who is seriously interested in education in South Africa and how to improve the state we’re in should read this draft of the upcoming NEEDU report. It is truly excellent.

Matric markers STILL not tested – my 2014 rant

matric markers pic

Every year for the past four years the department of basic education has tried — unsuccessfully — to implement competency tests for matric markers. Each year the teacher unions derail these well-intentioned plans, with the South African Democratic Teachers’ Union (Sadtu) raising the biggest ruckus.

The department’s logic is flawless: the integrity of the marking and moderation procedures of the National Senior Certificate exam depends crucially on the ability of markers to assess student responses accurately. Furthermore, without directly testing the content knowledge and marking competency of teachers one cannot be sure that the quality of matric markers is such that matric pupils receive the marks they deserve.

Importantly, the tests the department proposes would be conducted in a confidential, dignified and equitable manner that would not undermine the professionalism of applicants.

Sadtu counters that all teachers are equally capable of marking the matric exams and thus there is no need for minimum competency tests for prospective markers. This flies in the face of everything we know about teachers’ content knowledge and the pedagogical skills of large parts of the South African education system.

In a 1999 book, Getting Learning Right, Penny Vinjevold and Nick Taylor summarised the results of 54 studies commissioned by the Joint Education Trust, and wrote: “The most definite point of convergence across the President’s Education Initiative studies is the conclusion that teachers’ poor conceptual knowledge of the subjects they are teaching is a fundamental constraint on the quality of teaching and learning activities, and consequently on the quality of learning outcomes.” By implication this includes their ability to mark complex material accurately.

More recently, a 2011 report [p13] by the Southern and Eastern African Consortium for Monitoring Educational Quality found that only 32% of grade six mathematics teachers in South Africa had desirable levels of mathematics content knowledge, compared with 90% in Kenya and 76% in Zimbabwe.

Similar findings
I could go on and mention the numerous provincial studies that have been conducted in the North West, the Eastern Cape, KwaZulu-Natal and elsewhere that all find the same thing — extremely low levels of teacher content knowledge in the weakest parts of the schooling system — which, crucially, make up the majority of South Africa’s schools.

Given this situation, one wonders how Sadtu can argue that all matric teachers are equally competent to mark the matric exams or that they should not be tested. The union stance is that a system of teacher testing will disadvantage teachers from poor schools who cannot compete with those from wealthier schools. Although it is certainly true that the department has failed to provide meaningful learning opportunities to teachers in these underperforming schools, jeopardising the marks of matric pupils to make this stand is misguided, unethical and potentially even illegal.

These are important but separate issues and should be dealt with in different forums. But it is worth noting that the Western Cape has been testing prospective matric markers in the province since 2011, the only province in the country to do so.

The logic of the unions on this matter is perplexing. On numerous occasions they have rightly argued that teachers in poorer schools have not had meaningful learning opportunities and, therefore, that teachers are unequally prepared to teach, and by implication also unequally prepared to mark. Yet now they are arguing that all matric teachers are equally capable of marking the matric exams? So which is it? You can’t have it both ways. They either are or are not equally competent to mark matric exams. If it is the former, one cannot ensure children will receive the marks they are due; and if it is the latter, then one simply cannot argue that teachers should not be assessed prior to being appointed as markers.

On this question, a colleague of mine asked the following question: “How does the department employ people to teach matric when they are not considered competent to mark?” The uncomfortable answer is that, unfortunately, many matric teachers are neither competent to mark nor to teach — and this is because of no fault of their own. The blame instead falls squarely at the feet of the department, which has not provided them with quality professional development opportunities.

If one looks at the specifics of appointing matric markers, the union objections become even more bizarre. Although all matric teachers are legally allowed to apply to be matric markers, who is appointed and the criteria used for making these appointments are solely at the department’s discretion. Provided that these criteria are aligned with the position and are not discriminatory on such grounds as race, gender and sexual orientation, the department can select whomever it decides is most capable of doing the job.

Selection criteria
Currently the selection criteria relate to qualifications, teaching experience and language proficiency, but — bizarrely — not content knowledge. Given the nature of the work — assessing student responses for grading purposes — it seems only logical that applicants should be able to demonstrate this competency prior to being appointed for possessing it.

Because of the importance of the matric exam’s results for the life chances of individual pupils both in terms of further education opportunities and labour-market prospects, the department should put its foot down and take a stand for the 700 000 or so part-time and full-time students who are writing matric this year: it should insist that the 30 000-odd matric markers be tested prior to appointment.

Pupils, parents and school governing bodies have every reason to be concerned when there is no formal testing process to ensure that the teachers who will mark their all-important matric exams have the competence to do so in a consistent, fair and unbiased manner. Whether or not competency tests for matric markers are implemented has nothing to do with the unions and everything to do with the fairness of the marking and moderation procedures.

In sum, should prospective matric markers be tested prior to appointment? Yes. Is this a union issue? No. Will this be the last we hear of it? Unfortunately not.

The most tragic part of the above article is that I wrote it in November 2013 (published in the M&G here) and yet I can republish it here with one amendment; changing the sentence in the first line “for the past three years” to “for the past four years.” Matric markers are STILL not assessed before they are appointed, despite practically everyone agreeing that they should be tested. The second and third largest teacher unions (NAPTOSA and SAOU) both do not oppose teacher testing) Most notably the Ministerial Task Team report on the NSC (2014) who concluded that “Only the Western Cape selected its markers in 2013 based upon competency tests and was possibly disadvantaged by the strictness of the marking in its final overall results. A multifaceted, urgent and substantial intervention is called for to deal with the significant problems with the marking and the impact of this on the validity and reliability of the results” (Page 150 of the report). Why is it that the Minister can’t do the right thing on an issue that is UNAMBIGUOUSLY clear, rather than caving to SADTU?! This situation is utterly utterly disgraceful.

DFID 2014 rigorous literature reviews on education

lit review pic

DFID has recently funded a series of rigorous literature reviews on a number of important topics. I’ve included the descriptions below (from here originally) as well as links to the evidence brief and full literature reviews:

Early childhood development and cognitive development in developing countries (2014)
This review aimed to: (i) review existing evidence on the review topic to inform programme design and policy making undertaken by the DfID, other agencies and researchers; and (ii) identify critical evidence gaps to guide the development of future research programmes

The impact of tertiary education on development (2014)
After a long period in which the international development community has placed emphasis on primary education, there is now renewed interest in tertiary education (TE). However, the extent and nature of the impact of TE on development remains unclear. This rigorous review seeks to address this question in the context of low and lower middle income countries (LLMICs).

Interventions to enhance girls’ education and gender equality (2014)
The central research question that this review sets out to investigate concerns the kind of interventions that research evidence suggests can lead to an expansion and improvement in girls’ education. It also considered evidence on the relationship between an expansion and improvement in girls’ education and a deepening of gender equality.

The role and impact of private schools in developing countries (2014)
The research question driving the review is: Can private schools improve education for children in developing countries? The conceptual framework set out a number of hypotheses and assumptions that underpin the polarised debate about the potential and real contribution of private schools. These are interrogated through a rigorous and objective review of the evidence and findings are mapped on to an evidenced theory of change.

Literacy, foundation learning and assessment in developing countries (2014)

Developing countries face distinct challenges in providing access to quality education. Educational provision also varies markedly in terms of teacher training, teaching and learning resources, school attendance, and motivation of parents, teachers and children for schooling. Against this backdrop, we consider the available evidence on foundation learning and literacy in order to identify key components for intervention that are appropriate to specific cultural and linguistic contexts.

The political economy of education systems in developing countries (2014)

Teachers and schools do not exist in isolation of the larger world around them. Frequently, many of their actions – and the school outcomes that they are accountable for – are influenced by incentives and constraints operating outside the schooling system. Each of these factors influences different aspects of education reform, whether policy design, financing, implementation or evaluation. Given the importance of these power relations in influencing student outcomes, there is surprisingly little literature to guide us in making related policy decisions. One reason is that examining these issues in the case of education may not be amenable to a particular disciplinary lens and is better served through an inter-disciplinary approach. A key contribution of this review is to pull together the essential literature from various disciplinary and interdisciplinary traditions and to provide a conceptual framework in which to situate the analysis of political economy issues in education research. Another contribution is to carefully review the existing literature and identify research gaps in it. The review organises the literature along 5 key themes.

Pedagogy, curriculum, teaching practices and teacher education in developing countries (2013)

This rigorous literature review, focused on pedagogy, curriculum, teaching practices and teacher education in developing countries. It aimed to: (1) review existing evidence on the review topic to inform programme design and policy making undertaken by the DFID, other agencies and researchers, and (2) identify critical evidence gaps to guide the development of future research programmes.

The political economy of education systems in conflict-affected contexts (2014)

This report is a rigorous literature review on the political economy of education systems in conflict-affected contexts and is aimed at education advisers and agencies, development practitioners, and Ministry of Education policy makers working in conflict-affected contexts. It is also aimed at the broader education and conflict community of research and practice linked to the Inter-Agency Network of Education in Emergencies (INEE). The report seeks to provide theoretically informed and policy-relevant insights on the global, national and local governance of education systems in conflict-affected contexts.

Some articles/links I liked…

abstract4

  • The Sutton Trust has recently published a study (2014) “What makes great teaching? Review of the underpinning research” which looks like a great overview (thanks Joe Muller)
  • “Poverty traps and social exclusion among children in South Africa” – 2014 ReSEP report for the SAHRC (summary report here, full report here)
  • “The balancing act between the constitutional right to strike and the constitutional right to education” (Deacon, 2014, SAJE)
  • The SARCHi Chair in Teacher Education at CPUT (Prof Yusuf Sayed) has put out a call for post-doctoral students/fellowships (Deadline 10 Dec)
  • Alistair Sparks weighs in on the current situation in South Africa: “There is only one way to rectify the deplorable state we are in, and it surely cannot be long before the stalwarts of the ANC come to recognise and act upon it. Zuma should be asked to step down, even if that requires granting him a blanket amnesty and allowing him to go and enjoy Nkandla. The country and the ANC itself can no longer afford him. His interim successor should then form a government of national unity drawn from all sectors of society, to get the country back on track ahead of the 2019 elections.“

School visits in the rural Eastern Cape (Nov 2014) – some reflections (Post 1 of 2)

vistaLast week I was fortunate enough to visit 9 primary schools in one of the remotest parts of our country – the rural Eastern Cape. Some of the schools had no toilets, others had no electricity and many were simply falling apart. The trip was planned by the Legal Resources Centre who are visiting 200 schools in the rural Eastern Cape as part of an ongoing court-case around the eradication of mud-schools in the country (see here for an overview of the litigation). (Given that this is an ongoing court-case and to respect the anonymity of teachers and principals I won’t mention the names of the schools in my discussion below). My interest in tagging along was to find out more information about the learning outcomes in these schools, and the views and concerns of teachers and principals. I am increasingly of the opinion that large-scale quantitative research, if not complemented by on-the-ground experience, misses much of the picture and so I wanted to try and understand where things breakdown and why. We often have wonderful policies but disastrous implementation and abysmal outcomes – why is that? Where is the break in the chain and what causes it?

In each school I spoke to the principal and a teacher and looked for evidence of work in the students’ exercise books and workbooks and spoke to some of the students. I would say that a lot of what we found is reiterated in the literature (often countless times) and I’ll include links to that literature as I go through some of what I found…

(1) The learning environments in almost all of these nine schools was truly shocking. The picture below shows the classroom environment in one of the schools – the grade 1’s are on the left of the picture and the grade 2’s are on the right. (All the schools we visited were multi-grade schools where one teacher teaches two or three grades, usually in one classroom – the principal is always a teacher as well. This is seen as the only feasible option when learner numbers in an area are very low). This school does not have enough buildings or classrooms and so this classroom also doubles up as the  kitchen and sometimes also as the staff room. The person in the green on the left of the picture was a volunteer who helps make the food for the children. The National School Nutrition Programme (NSNP) provides a hot meal to over 9 million South African students every day – this is one of the major successes of the Department of Basic Education and one for which we should all be proud (see also Kha Rhi Gude – ditto, thanks Veronica McKay!). The problem with the setup at this school, as the principal told us, is that the children stop concentrating as soon as they can smell the food being made – (usually prepared from 9am to be ready before 10am), but this school has no other option – there are no other rooms to use.

multigrade and school feeding

One school we visited overcame this problem by building their own “kitchen” using mud and sticks…

mud kitchen

Some of the schools we visited had no toilets whatsoever (“The children use the bush”). This is despite the fact that often the municipality has installed toilets for nearby residents (sometime 150m away) but won’t build any for the school because that is not their mandate! Talk about a lack of government integration/communication. In some schools it is not the lack of space or toilets that was the problem but the inadequate roofing. In this school the tin roof on the one side of the classroom is full of holes. I asked the teacher what she does when it rains and she said, “Oh, then we all sit on this side of the classroom.” She teaches both Grade 1 and Grade 2 in this classroom.

classroom

In my discussion with this teacher, I asked her when they start teaching English at the school. According to the curriculum document (CAPS) teachers are required to teach English for  2/3 hours per week in grades 1 and 2 and 3/4 hours in grade 3. The teacher replied and said that she teaches English FAL from Grade 1. However when I was walking around the classroom and came across the timetable for grades 1 and 2 I saw that English did not feature in the timetable. This is problematic. Unless children slowly start increasing their vocabulary and competence in English in grades 1-3, the transition to English as medium-of-instruction in grade 4 is extremely difficult.

Screen Shot 2014-11-27 at 11.23.19 AM

In another school we visited they had built on a tin-classroom extension onto the brick school in order to accommodate the learners. The tragic story behind this school is that it was built by the community because the next closest primary school is across the river in the valley and too many children were drowning trying to cross it to get to school. Unbelievably tragic.

tin classroom 2

tin classroom

(2) There is extremely little learning taking place in these schools.

In every school that I went to (except for one) there was very little evidence of learning or work. Paging through a student’s workbook, you will typically find that of the 80 pages in the Term 1/2 workbook, only 15-20 pages will have anything written on them (this despite the fact that we are now in term 4). The pages with the work on them are spread throughout the book so that you will find one or two pages of work and then 10 pages that are blank and then another page of work etc. It’s also not as if these students are working extensively in exercise books – in the exercise books that I looked at you will find one exercise every two weeks (sometimes once a month) and it will be extremely basic. In grade 1 it would be things like drawing a picture and writing a word. However, even when students are working in the workbook, it’s clear that they don’t actually know what they are doing. To provide some examples:

First is first…

first is first

workbook is are

The student below is clearly just writing whatever has been underlined, irrespective of what it means.

reading

In most of the schools I asked the Foundation Phase teacher if she thought that her Grade 3 students would be able to read this short story in English:

fun in the sun

Most of the teachers said that they thought their Grade 3 students would be able to read it. I asked if she could pick one student to read the story with me outside. To get the children relaxed I starting with an extremely elementary text, “The kids read books” (exercise from Gr 1 FAL book) which almost all the children could read. Then they would try and read the “Fun in the Sun” story (exercise from Gr1 HL workbook). Most of the selected children from the 9 schools (likely to be the better performing kids in the class) could read the story aloud in English – sometimes very slowly. However only 1 of the 9 students could answer the question “What colour is Sam’s cap?” and the same student was the only one who could answer the question “What colour is the mat?” Clearly these children are not ‘reading for meaning’ – i.e. they are illiterate. And this is in November 2014. These students will switch to English as medium of instruction in February 2015 when they enter grade 4 (see here for a nationally representative discussion and here for an excellent qualitative study on reading)

(3) Union meetings and departmental meetings are only ever held during school hours. One of the things that I was interested to find out was how often teachers and principals attend departmental meetings/training and union meetings and when these meetings are held. In this case all of the teachers I spoke to were part of SADTU (for a breakdown of SADTU membership by province see here). I asked the principals and teachers the following question “On which day of the week are departmental meetings usually held?” “On which day of the week are union meetings held?” Without any exceptions, all principals and all teachers said that the meetings happened during weekdays (various weekdays Mon-Fri) and during school hours  (usually 9am-1/2pm). There were approximately 4 union meetings a year and 3-4 departmental meetings or training days a year. One principal told me “In 2013 there were 5 workshops. This year we have had three workshops about how to mark the ANAs.” Yet another principal: “We have shut the school 4 or 5 times this year because there was training for the principal and one teacher on the same day and we are only 3 teachers.” This was something that I expected to hear but still found it very frustrating that everyone thought that it was totally OK to have meetings during school time. I think the one that infuriated me most was when someone told us that SADTU had organized a prayer meeting during school time and invited teachers to attend.

I include below the “Rules” poster that was put up on the wall in one of the schools we visited:

Screen Shot 2014-11-27 at 12.18.32 PM

If you battled to read it it says “(1) Learner must wear school uniform. (2) Educators must attend, clinics, workshops, seminars, and any governmental meetings. (3) All educators must be responsible for their duties delegated to them. (4) Educators must sign leave forms on their absence from school. (5) No educators are allowed to be out of classes in a teaching time, chatting, and discussing their problems.” I reserve the right to comment on this at a later date – there is so much to be said, mainly about priorities.

When I asked one teacher why they did not have the meetings on Saturdays the teacher replied “Because we are not paid to work on Saturdays. Why would we go if we are not paid?” Now I didn’t say this at the time, but that’s actually wrong. As part of teacher’s employment package, they are paid for 80 hours of ongoing professional development  (see page ix of this report). When I asked teachers what they did with the students when they went on training, they said that the other teachers looked after them or they told them not to come to school that day. In one of the schools that we visited the Grade R teacher was at a meeting and so the Grade R class was left unattended (the other two teachers were teaching their own classes). There were no books in the classroom and the students were just keeping themselves busy talking to each other and walking around:

grade r unattended

The SACMEQ study of 2007 asked principals a variety of questions about what they did with students when teachers were absent. There were 392 schools included in the study drawn as a nationally representative sample of primary schools in the country (see report here). I include the breakdown of the answers to those questions by province:

Screen Shot 2014-11-27 at 12.07.57 PM

Screen Shot 2014-11-27 at 12.08.29 PM

More than 40% of principals in the Eastern Cape said they they send students home “sometimes” and leave students unattended “sometimes” when a teacher is absent (on any given day about 10-12 % of teachers are absent from school – see 2010 HSRC report here). A number of people working in the field have told me that if you ever want to do teacher training or have meetings it must happen during school hours or else the teachers will not come. Something is seriously rotten in the state of Denmark. However not all principals are as complacent and compliant as these principals. One remarkable principal I spoke to had the following to say about teacher absenteeism: “Each and every week there is a memorial service. We are dying like flies. But you cannot have a teacher away every single week. I cannot.”

(4) Intense union involvement in the appointment of educators, principals and district officials. One of the common threads that came through in many of my discussions was the involvement of the unions – particularly SADTU – in the appointment and promotion of teachers, principals and district officials. This did not come as much of a surprise – there is currently a Ministerial Task Team looking into the sale of teaching posts. The thing I was interested in is how this actually happens. One of the people I spoke to was particularly insightful on this…let me include some excerpts from our conversation:

Nic: “You mentioned that the unions are involved in appointments and promotions, can you tell me how that works?”

X: “When you are selecting a Head of Department (HOD) for the school there are 2 parents from the SGB and 1 teacher, the principal is there but cannot vote. In the rural Eastern Cape many of the parents are not well educated. They know nothing about laws so it is just the principal and the teachers.  SADTU can very easily influence the parents through the teacher. If SADTU does not get the person that they want they will say there was an irregularity in the interview process. I once encouraged the parents to appoint a good mathematics teacher for my school and they did, but they were not SADTU’s choice so they had the teacher removed. They re-advertised the post but without subject specification because there was no SADTU member who had maths or science. I am now stuck with someone who is babysitting mathematics and my results are terrible. My ANAs are very low in mathematics. And you cannot challenge it.” [“Why can’t you challenge SADTU?”] They will go for you. They will accuse you of sexual misconduct and there must be an enquiry. They will accuse you of financial mismanagement. They will go for small things to catch you. You know you need 3 quotations if you buy something and you must write it down so that if you only have two or forgot to write it down, they will catch you. Most principals will make a small mistake. But these are honest mistakes. But they will catch you.”  “The Department is listening and and the union is managing. SADTU does not want to listen, they want to lead and they want to manage.”

Me: “Can you tell me about the appointment of district officials, curriculum advisors etc. do you know anything about how that works?” Response: “Yes, those are also appointed by SADTU. Everyone is looking for posts and if you want a post somewhere, SADTU can make it happen.”

Another principal had recently appointed a new teacher and I asked about the process and he/she said “We get given a list from the district office and pick someone from the list.” [“How were the people on the list selected?”] “We are in a rural area, we don’t know how those people got on the list.”

Unless we can figure out a way to eradicate these illegal practices and prosecute those involved, the Eastern Cape will remain completely dysfunctional. When I hear these stories piling up one on top of the other – always the same tune just in a different key – my blood starts to boil. People playing politics while Rome is burning. No one could possibly come up with any moral or ethical defence for the kinds of corrupt and collusive practices that permeates the entire education bureaucracy in the Eastern Cape (and many other provinces I’m sure). And the most tragic thing of all is that the burden of this cancer falls most heavily on the poorest of the poor, whose children will never receive a basic quality of education and never be able to develop their talents and personalities. They are condemned to unemployment, indignity and hereditary poverty…and so the cycle continues…

//

I intend to write a follow-up post where I discuss the infuriating issue of Nkandla-size mud-school replacements, as well as some of my other observations about the recent trip…

Further reading:

“Every child must read” – my M&G article

reading

(The article below appeared in the Mail & Guardian on the 21st of November 2014 – available on the M&G website here).

If you’re reading this sentence it means that somewhere, somehow, you learnt to read. Bravo! You acquired the magical skill of translating scribbles into language and making meaning from the print symbols on the pages and screens that permeate our lives.

It really is quite remarkable that a few scrawls on a page can make us weep with joy or seethe with rage as we engage with the heroes, villains and ideas of bygone or future eras. Imagine what your life would be like if you could not read. Imagine what school would have been like if you couldn’t read.

And yet, unfortunately, this is not an imaginary experience for thousands of South African children. It is their daily, lived experience. Constantly struggling to understand the words on a page, let alone deciphering the deeper meaning behind these funny dots and dashes. And this is not their fault.

The human brain is hard-wired to acquire language and almost all children can learn to read in just a few years if provided with the right teaching, resources and encouragement. However, many South African children do not attend schools where these necessary conditions are present. A number of South African studies have revealed that children who cannot read and write properly by grade four end up playing catch-up for the rest of their school days. These children never quite grasp what is expected from them, even as they are told they are failing and must try harder.

Let me explain some of the recent research findings on this very important topic.
In 2011 South Africa participated in an international study called PrePirls (pre-progress in international reading literacy study), which is aimed at assessing the reading ability of grade four children. The study examined a nationally representative sample of 341 primary schools drawn from across the country.

The reason for choosing to assess grade four is not arbitrary, but rooted in an understanding of when and how children learn to read.  The first three years of schooling are regarded as the “learning to read” phase, when children acquire the ability to decode text and convert print symbols into language.  In grade four they enter the “reading to learn” phase as they start acquiring new information through the skill of reading.

Children who cannot read properly by grade four are severely disadvantaged, because they cannot read fluently or read for meaning, and therefore don’t benefit much from higher grades. This places them in perpetual catch-up mode until they begin to approach matric and drop out of school in grades 10 and 11, as 50% of South African students do. Unfortunately the results of PrePirls are truly sobering.

If one looks at the reading achievement of these schools and splits the 341 schools into the better performing half (169 schools) and the worse performing half (172 schools) of the sample, the results speak for themselves. In the top half of schools, 10% of students were completely illiterate. That is to say that they could not locate and retrieve an explicitly stated detail in a short and simple text. These children cannot read at all.

In the bottom half of schools, an unbelievable 51% of students were completely illiterate! After four years of formal, full-time schooling, every second child in these 172 schools was completely illiterate. These 172 schools are statistically representative of half of South African primary schools. (These tests were done in the language that they had been learning in during grades one to three — an African language for most children, before switching to English in grade four).

These children who don’t learn how to read properly are then promoted to the next grade, but never manage to get their heads above water for the rest of their school days.

What to do?

Firstly we have to get the basics right in the Foundation Phase (grade one to three). We need a national reading campaign where all stakeholders (parents, teachers, principals, government officials, the minister, the president) all rally behind this goal: “Every child must read and write by the end of grade three.”

This is the very same goal that Brazil used as the core goal for primary schooling — with much success. One prominent South African researcher, Elizabeth Pretorius, has identified four necessary criteria to ensure all children learn to read:

(1) Teachers need to understand when and how children acquire reading and comprehension skills, as well as understand how to teach reading;

(2) Children need easy access to interesting books in their own language and in English;

(3) Children need to be constantly motivated to read, with reading seen as a pleasurable activity by students and teachers, and

(4) Children need to be given plenty of opportunities to read in and outside of the classroom.

Sadly there is currently no systematic evidence about which of the many interventions currently being implemented in South Africa actually work, and if they do work, which is best. It is of fundamental importance that a national reading strategy be based on scientific evidence regarding what most improves the acquisition of reading in South African schools.

If we do not get reading right in grades one to three, any intervention later in the system will only have a small impact on learning, and consequently the life chances of the poor.  The later in life we attempt to repair early learning deficits, the costlier the remediation becomes. We simply must ensure that every child timeously acquires that magical skill of translating scribbles into language. Our education system depends on it.
//

For an excellent (and much more detailed) article on reading see Elizabeth Pretorius’ original article here. 

Links I liked…

soitgoes

  • Excellent 2013 lecture by Michael Fullan on “Schools in need of re-education“
  • Her Majesty Susan Sontag talks to us mere mortals about “Modern Literacy“
  • The HSRC are looking for a post-doc student in their “Education and Skills Development” portfolio (deadline 17 Nov) – for more details see here.
  • What a tattoo looks like while it’s happening (semi-cringey) – IFL Science
  • Great collection of modern architecture photos
  •  “Writing and style guide for university papers and assignments” – from the University of Ottawa
  • If you want to get mad read this GroundUp article about Marikana “Lonmin’s Broken Promises“
  • Shocking new report by Oxfam (“Even it up“) on global inequality and what needs to be done.
  • Quote of the week: “It is worth noting that American students have never received high scores on international tests. On the first such test, a test of mathematics in 1964, senior year students in the US scored last of twelve nations, and eighth-grade students scored next to last. But in the following fifty years, the US outperformed the other eleven nations by every measure, whether economic productivity, military might, technological innovation, or democratic institutions. This raises the question of whether the scores of fifteen-year-old students on international tests predict anything of importance or whether they reflect that our students lack motivation to do their best when taking a test that doesn’t count toward their grade or graduation.” – Diane Ravitch in the NYRB “The myth of Chinese Super Schools” (I’ve received great feedback from friends and colleagues – many challenging Ravitch’s simplistic generalizations – but I still think the article is quite thought-provoking).

The best SA research article I’ve read this year

dsc_1831

This article by Elizabeth Pretorius (UNISA) is easily one of the best academic articles I’ve read this year! I would strongly recommend that anyone interested in (1) reading/literacy, (2) the education crisis in SA, or (3) how to get out of the mess we are currently in, should read this article!

Pretorius, E. 2014. Supporting transition or playing catch-up in Grade 4? Implications for standards in education and training. Perspectives in Education. 32 (1), pp 51-76.

ABSTRACT

This paper describes an intervention programme that was originally intended to support transition to English as language of learning and teaching (LoLT) in Grade 4 in a township school, using a pre- and post-test design. Because the pre-tests revealed very poor literacy levels in both Zulu home language and English, the intervention programme was modified in an attempt to fast-track the learners to literacy levels more appropriate to their grade. This paper outlines the intervention, presents the pre- and post-test results of the English literacy assessments, reflects on the effects of the intervention, and briefly considers some of the reasons for the initial poor literacy performance. Finally, a model for literacy development in high-poverty contexts is proposed to minimise the need to play catch-up in the Intermediate Phase.

JPAL Executive Education Course (on RCT evaluations)

CEI-Conversations-with-J-PAL
After submitting my PhD a few weeks ago I am slowly getting back into the swing of things. That includes blogging and the Q&A series. But for now I thought I would pass on this advert for JPAL’s latest Executive Education Course. JPAL is a great organization and I would highly recommend this course to those professionals who wants to get a better understanding of how randomised control trails work.
//
J-PAL Africa Executive Education Course
Location: School of Economics, University of Cape Town
Course Dates: 19-23 January 2015
Application Deadline: 4 November 2014
 
We are pleased to invite you to apply to the upcoming J-PAL Executive Education course in Evaluating Social Programmes, which will provide you with a thorough understanding of randomised evaluations. Our Executive Education courses are valued by people from a variety of backgrounds: managers and researchers from international development organizations, governments, as well as trained economists looking to retool. If you have colleagues or friends who may be interested in applying to this course, we encourage you to pass this invitation on to them as well.
 
The course is a 5-day, full-time course and will run from the 19 – 23 January 2015 at the University of Cape Town in South Africa. The purpose of the course is to provide participants with a thorough understanding of why and how to use randomisation in an impact evaluation and to provide pragmatic step-by-step training for conducting their own randomised evaluation. J-PAL affiliates with extensive experience in using randomised impact evaluations to test the effectiveness of social programmes, in Africa and globally, will teach the course.
 
You may view more information about the course content and fees here. You can also access the course applications through that website, or by following this link here. The deadline for applications is 4 November 2014. We receive far more applications to the course than we can accommodate, and we encourage applicants to apply early. We will notify participants of whether their application has been accepted by November 7th, after which payment of the course fee (see course fees here) will need to be made by 2 December, before a place on the course can be confirmed.  Once acceptance is confirmed, participants are responsible for making their own travel and accommodation arrangements.
 
We hope to receive your application for the course soon. Do let us know if you have questions about the course. 
 
Warm regards,
Laura Poswell
J-PAL Africa Executive Director

Big Brother is watching (Links I liked)

obama-big-brother-is-watching-you_1637245851

  • Earth meet Big Brother, Big Brother – Earth. WIRED’s latest article on Edward Snowden is profoundly shocking. The NSA has the potential to track “everyone” in a city using the MAC address from cellphones, it “accidentally” sunk the Internet in Syria while trying to spy on Syrian civilians, and frequently passes unredacted data on to Israel about Palestinian American citizens. WTF.
  • “The non-monetary benefits of learning may be difficult to measure, but they shape and determine what we recognize to be the quality of our life” – from “Why academic achievement matters (via Harry Patrinos).
  • After watching the trailer to Boyhood I’ve fallen in love with this song by Family of the Year.
  • Henry Miller on friendship – worth reading (thanks Clint Clark)
  • Next year’s IEA conference is happening in Cape Town (22-23 June 2015). For those who are doing work on IEA data (PIRLS/TIMSS) the deadline for submission is 1 December 2014.
  • The Free State Department of Basic Education is clearly trying to game the ANAs – see this article. We really need to be thinking about the potential unintended consequences of the ANAs. I think the ANAs are a positive development in the SA education system, but we need to be paying closer attention to the potential unintended consequences of the ANAs and how we can minimize these unintended consequences. For one thing, the ANAs are not ready to be used as an accurate indicator of student or school performance across the grades or over time. 
  • In case you were wondering what your skin looks like with and without sunscreen, see here. *will never not wear sunscreen again, or use double negatives*
  • “The impact of national and international assessment programmes on education policy, particularly policies regarding resource allocation and teaching and learning practices in developing countries” – Systematic review of the evidence by ACER 2014

There’s STILL madness in WEF rankings

wef

I don’t usually repost articles that I’ve written in the past but given that the World Economic Forum has recently released it’s 2014/15 Global Competitiveness report I thought it makes sense. The article below first appeared in the M&G on the 13th of June. If you’re part of the media and want to quote me on something RE the 2014/15 rankings you can use any of the following:

  • “These results are completely and utterly preposterous. Of course South Africa’s education system is in crisis and performing worse than other middle-income (and some low-income) countries) but it definitely isn’t the worst in the world.”
  • “The methods used to calculate these education rankings are subjective, unscientific, unreliable and lack any form of technical credibility or cross-national comparability.”
  • “The mistakes in the WEF’s methodology are so egregious that one needs only look at the list of countries and their respective rankings to appreciate how ridiculous they really are – failed states rank above modernising middle-income countries. How on earth do Japan and Cote d’Ivoire have the same ranking for the quality of their maths and science instruction? This is not science, this is an unscientific opinion survey”
  • “The WEF has seriously undermined its own technical credibility by reporting these ridiculous education rankings. Until it rectifies its methodology, no one should take the rankings seriously.”

//

In the past two weeks the South African media has had a field day lamenting the state of maths and science education in the country. This is because the World Economic Forum (WEF) recently ranked South Africa 148th (out of 148 countries) on the quality of its maths and science education.

Let me cut to the chase and say, unequivocally, that the methods used to calculate these education rankings are subjective, unscientific, unreliable and lack any form of technical credibility or cross-national comparability. I am not disputing that South Africa’s schooling system is currently in crisis (it is), or that South Africa performs extremely weakly relative to other low- and middle-income countries (it does). What I am disputing is that these “rankings” should be taken seriously by anyone or used as evidence of deterioration (they shouldn’t).

The mistakes in the WEF’s methodology are so egregious that one needs only look at the list of countries and their respective rankings to appreciate how ridiculous they really are. How is it possible that the quality of maths and science education in failed states such as Chad (ranked 127th on the WEF list), Liberia (125th) and Haiti (120th) is better than modernising middle-income countries such as Brazil (136th) and Mexico (131st)? How do countries such as Madagascar (82nd) and Zambia (76th) outrank countries such as Israel (78th), Spain (88th) and Turkey (101st)?

Preposterous
Although these preposterous rankings sound like an April Fool’s joke gone wrong, they are reported without qualm on page 287 of the WEF Information Technology Report 2014. Even a cursory analysis of the faulty ranking methodology the WEF employed shows how it is possible to arrive at these outlandish “rankings.” The WEF asked between 30 and 100 business executives in each country to answer questions (relating only to their own country), using a scale of one to seven to record their perceptions, with one representing the worst possible situation and seven the best possible situation.

The question relating to maths and science education was phrased as follows: “In your country, how would you assess the quality of maths and science education in schools?” with “one” being “extremely poor – among the worst in the world”, and “seven” being “excellent – among the best in the world”.

In South Africa, 47 business executives were surveyed for these rankings. On the question relating to maths and science, the average score among these 47 executives was 1.9, indicating that the vast majority of these South African business executives believed that the quality of maths and science education in the country was “among the worst in the world.” Yet this is really just a measure of the perceptions of these 47 businessmen, as the department of basic education has correctly pointed out.

By contrast, when the 55 Malawian and 85 Zambian business executives were surveyed, they were more optimistic about the maths and science education provided to students in their countries, yielding average scores of 3.2 and four respectively.

Outperform
This explains why Malawi ranks 113th and Zambia ranks 76th whereas South Africa ranks 148th. Yet we know from objective cross-national standardised testing in the region that Zambia and Malawi are two of the few countries that South Africa actually does outperform.

Clearly the ratings given by these business executives are subjective and dependent on their particular mental reference points, which obviously differ by country. These 47 South African executives were not asked to rank South Africa relative to other specific countries – such as Madagascar, Malawi or Mali – only relative to “the world”.

Although the perceptions of business executives are important in their own right, it is ludicrous to use these within-country perceptions to rank “the quality of maths and science education” between countries; particularly when we have objectively verifiable, cross-nationally comparable scientific evidence for maths and science performance for at least 113 countries.

Looking at South Africa specifically, we participate in two major cross-national standardised testing systems that aim to compare the mathematics and science performance of South African students with that of students in other countries. The Trends in International Mathematics and Science Study (Timss) tests grade eight students from middle- and high-income countries, and the Southern Africa Consortium for Monitoring Educational Quality (Sacmeq) study tests grade six students from 15 countries in sub-Saharan Africa.

Worse than South Africa
Of the countries participating in Sacmeq, South Africa came 8th in maths, behind much poorer countries such as Kenya (2nd), Swaziland (5th) and Tanzania (3rd), but ahead of Mozambique (10th), Namibia (13th), Zambia (14th) and Malawi (15th). Although this situation is no cause for celebration, it does show that these countries – which outrank South Africa in the WEF rankings – are in fact doing worse than South Africa in reality.

If we look beyond Africa to the Timss rankings, South Africa performs abysmally. Of the 42 countries that participated from around the world (including 21 developing countries), South Africa came joint last with Honduras in 2011. This should shock us to the core. But it does not mean that we have the worst education system in the world. Rather, we have the worst education system of those 42 countries that take part in these assessments.

There is a big difference. Only 21 developing countries took part in these assessments, but there are around 115 developing countries in the WEF tables. The fact that Mali, Madagascar, Liberia and Haiti (for example) do not take part in these assessments means that business executives in these countries have very little reliable information on the quality of education in their countries.

In South Africa the basic education department has wisely chosen to take part in these assessments so that we have reliable information on the performance of our education system, however low that performance might be.

Continuing participation
This is one thing that the department should be commended for –that is, for continuing to participate in these assessments, which provide valuable information, despite being lambasted by their findings.

Perhaps the best example of how flawed the WEF methodology is is illustrated by comparing Indonesia and Japan on the WEF rankings and on the well-respected Organisation for Economic Co-operation and Development’s Programme for International Student Assessment (Pisa) rankings, which also tests math and science, as does Timss.

In the WEF rankings, executives in Indonesia and Japan both gave an average score of 4.7 for the quality of maths and science education in their respective countries. This placed Japan 34th and Indonesia 35th of the 148 countries. Yet, of the 65 countries participating in the 2012 round of the Pisa maths and science testing, Japan came 7th (out of 65) and Indonesia came 64th. Go figure.

Although there are some early signs of improvement in the South African education system, we know that things remain dire. South African students perform worse than all middle-income countries that participate in assessments, and even worse than some low-income African countries.

But to claim that South Africa has the worst quality of maths and science education in the world, and to use executives’ perceptions over scientific evidence to do so, is irrational and irresponsible.

The WEF has seriously undermined its own technical credibility by reporting these ridiculous education rankings. Until it rectifies its methodology, no one should take the rankings seriously.