Nic Spaull

The relatively new open-access SA Journal of Childhood Education has recently put out a call for abstracts (see below) for their special issue on “Priorities and policy-making in South African Education” (Guest editors: Nick Taylor and Thabo Mabogoane). Given the policy relevance of this special issue, researchers at ReSEP (including myself) will be submitting a number of abstracts for work we are currently doing and intend to do. If you’re doing work in this field I’d encourage you to do the same, it’s likely to be a great issue!

//

Special issue: Call for Papers “Priorities and Policy-making in South African Education”

 Guest editors: Nick Taylor and Thabo Mabogoa

Despite considerable expenditures and efforts to improve performance and reduce inequality, there is limited evidence of substantial improvements in educational outcomes, or the equalisation thereof. Periodic reviews of the evidence have shown a number of recurring themes that are especially characteristic of schooling in South Africa. These include unequal access to socially, emotionally and cognitively stimulating environments (both in the home and at school), insufficient resources, low levels of curriculum coverage, low levels of teacher content knowledge, inadequate support and training opportunities for in-service teachers, challenges associated with learning and teaching in a second language, low levels of accountability and high dropout in upper secondary school (among many others).

While education officials are often aware of these challenges, most policy-makers find it difficult to synthesise this evidence, which is necessary for prioritisation and resource allocation. Making sense from research is particularly challenging  when it is presented in isolation from other problem areas and only speaks to other research within its ‘silo’.  It is now widely acknowledged that if government policies are to have the largest possible impact, they need to be based on rigorous evidence and peer-reviewed research. Furthermore the National Development Plan (the government’s guiding framework) has emphasised the need for the “process of prioritisation and sequencing” if the plan is to be implemented. Such a process of prioritisation and sequencing requires rich, inter-connected evidence on education in South Africa.

Consequently, this call for papers focuses on education research in South Africa that speaks directly to policy-making and prioritisation. Papers that synthesise existing evidence across research areas in education are especially welcome.

Instructions for authors: www.sajce.co.za

Journal administrator: childhooded@uj.ac.za

Online submissions and author registration: www.sajce.co.za

Deadline for abstract submission: 31 March 2015
Deadline for full papers (of accepted abstracts): 30 June 2015
Intended publication date: November 2015

The SAJCE is accredited by the Department of Higher Education and Training and  has applied, through the Academy of Science of South Africa  (ASSAF) for listing  on the open journals platform, ScIELO 

A Working Paper that I co-wrote with Janeli Kotze was released today on the Stellenbosch Economic Working Paper site (available here). The paper has also been accepted for publication in the International Journal of Educational Development and should be out next year. I will include some excerpts from the paper for those who’d prefer the short version…

Abstract:

This study quantifies a year’s worth of mathematics learning in South Africa (0.3 standard deviations) and uses this measure to develop empirically-calibrated learning trajectories. Two main findings are, (1) only the top 16% of South African Grade 3 children are performing at an appropriate Grade 3 level. (2) The learning gap between the poorest 60% of students and the wealthiest 20% of students is approximately three Grade-levels in Grade 3, growing to four Grade-levels by Grade 9. The paper concludes by arguing that the later in life we attempt to repair early learning deficits in mathematics, the costlier the remediation becomes.

Excerpts:

Few would argue that the state of mathematics education in South Africa is something other than dire. This belief is widespread among academic researchers and those in civil society, and is also strongly supported by a host of local and international assessments of mathematical achievement extending back to at least 1995 (Howie & Hughes, 1998; Reddy, 2006; Fleisch, 2008; Spaull, 2013; Taylor et al., 2013). Many of these studies, and particularly those that focus on mathematics, have identified that students acquire learning deficits early on in their schooling careers and that these backlogs are the root cause of underperformance in later years. They argue that any attempts to raise students’ mathematical proficiency must first address these deficits if they are to be successful (Taylor et al., 2003). The present study adds further evidence to this body of work by using nationally representative data to provide some indication of the true size and scope of these learning deficits.

…

In South Africa, research in this area has generally focussed on in-depth localized studies of student workbooks and classroom observation (Ensor et al., 2009). For some examples, Carnoy et al. (2012) observe mathematics learning in Grade 6 classrooms from 60 schools in one South African province (North West) and compare these classrooms to 60 schools in neighbouring Botswana. On a smaller scale, Venkat & Naidoo (2012) focus on 10 primary schools in Gauteng and analyse coherence for conceptual learning in a Grade 2 numeracy lesson. Similarly Schollar (2008) conducted interviews and classroom observations as well as analysed a large sample of learner scripts to determine the development (or lack thereof) of mathematical concepts through the Grades.

Where the present research differs from these earlier studies is that it focuses on quantifying national learning deficits in general, rather than in specific learning areas. While the latter are essential for understanding what the problems are and how to fix them, analyses at the national level are also needed if we are to understand the extent and distribution of the problem, both of which are imperative for policy-making purposes. This is only possible by analysing multiple nationally-representative surveys of student achievement, which is the focus of the present study. The two core research questions that animate this study are as follows:

• How large are learning deficits in South Africa and how are they distributed in the student population?
• Do learning deficits grow, shrink or remain unchanged as students progress to higher Grades?

To answer these questions we analyse four nationally representative datasets of mathematics achievement, namely: (1) the Systemic Evaluation 2007 (Grade 3), (2) the National School Effectiveness Study 2007/8/9 (Grade 3, 4 and 5), (3) the Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ) 2007 (Grade 6), (4) and the Trends in International Mathematics and Science Study (TIMSS) 2011 (Grade 9).

…

The extant research on mathematics learning in South Africa strongly supports this conclusion with numerous researchers highlighting the inadequate acquisition of basic skills and the consequent negative effects on further learning. Taylor & Vinjevold (1999) summarise the findings from 54 studies[1] commissioned by the President’s Education Initiative and conclude that:

“At all levels investigated by [The President’s Education Initiative], the conceptual knowledge of students is well below that expected at the respective Grades. Furthermore, because students are infrequently required to engage with tasks at any but the most elementary cognitive level, the development of higher order skills is stunted” (Taylor & Vinjevold, 1999, p. 231).

This lack of engagement with higher order content is the prime focus of Reeves and Muller’s (2005) analysis of Opportunity-to-Learn (OTL) and mathematics achievement in South Africa, where OTL is the curriculum actually made available to learners in the classroom. Taylor et al. (2003, p. 129) in their book Getting Schools Working summarise succinctly the debilitating effects of cumulative learning deficits:

“At the end of the Foundation Phase [Grades 1-3], learners have only a rudimentary grasp of the principles of reading and writing … it is very hard for learners to make up this cumulative deficit in later years … particularly in those subjects that … [have] vertical demarcation requirements (especially mathematics and science), the sequence, pacing, progression and coverage requirements of the high school curriculum make it virtually impossible for learners who have been disadvantaged by their early schooling to ‘catch-up’ later sufficiently to do themselves justice at the high school exit level.’

And lastly, Schollar (2008) summarises the findings of the Primary Mathematics Research Project which looked at over 7000 learners from 154 schools in South Africa and concludes as follows:

“Phase I concluded that the fundamental cause of poor learner performance across our education system was a failure to extend the ability of learners from counting to true calculating in their primary schooling. All more complex mathematics depends, in the first instance, on an instinctive understanding of place value within the base-10 number system, combined with an ability to readily perform basic calculations and see numeric relationships … Learners are routinely promoted from one Grade to the next without having mastered the content and foundational competences of preceding Grades, resulting in a large cognitive backlog that progressively inhibits the acquisition of more complex competencies. The consequence is that every class has become, in effect, a ‘multi-Grade’ class in which there is a very large range of learner abilities and this makes it very difficult, or even impossible, to consistently teach to the required assessment standards for any particular Grade. Mathematics, however, is an hierarchical subject in which the development of increasingly complex cognitive abilities at each succeeding level is dependent on the progressive and cumulative mastery of its conceptual frameworks, starting with the absolutely fundamental basics of place value (the base-10 number system) and the four operations (calculation)” (Schollar, 2008, p. 1).

…

To provide an alternative measure of performance, we provide two examples of no-language items in NSES and show when students answer the question correctly – i.e. in Grade 3, Grade 4, Grade 5 or not by the end of Grade 5. Given that one needs to follow the same students from Grade 3 to 5 we limit the sample here to the panel sample of NSES students (8383 students). Figure 3 below shows a simple question testing two and three digit addition with no carrying. This is within the Grade 3 curriculum which states that students should be able to “perform calculations using the appropriate symbols to solve problems involving addition of whole numbers with at least three digits.” Although this is a Grade 3 level item and contains no language content, only 20% of Quintile 1-4 students could answer this correctly in Grade 3, with the proportion in Quintile 5 being twice as high (42%) but still low. While there is evidently some learning taking place in Grade 4 and 5, more than 40% of Quintile 1-4 children still could not answer this Grade 3 level problem at the end of Grade 5. In Quintile 5 this figure was only 22%.

Figure 3

Figure 4 below shows a similar situation where the vast majority of Grade 3 children cannot answer this Grade 3 level problem. While some children learn the skill in Grade 4 or 5, the majority of children still cannot answer this problem at the end of Grade 5, despite it being set at the Grade 3 level.

Figure 4

 Moving from learning deficits to learning trajectories

While the previous sections have identified the proportion of students that are not operating at a Grade 3 level, they do not provide much guidance in terms of learning trajectories into later Grades. The figures above show that some students are only learning part of the Grade 3 curriculum in either Grade 4 or Grade 5 and that many never seem to acquire these skills. However one cannot say to what extent they are also acquiring Grade 4 level skills in Grade 4 and Grade 5 level skills in Grade 5, although this is unlikely. This is because the NSES test was set at a Grade 3 level with only a small number of questions set at the Grade 4 level. One could use SACMEQ (Grade 6) and TIMSS (Grade 9) as measures of mathematical proficiency at higher levels, but these tests are not calibrated to be comparable to each other, or to earlier tests like the NSES. This is problematic since learning trajectories require data points distributed across the full range of educational phases which are comparable to each other both in terms of the content tested and the difficulty level of the tests. One alternative method to partially overcome the lack of inter-survey comparability is to measure the size of learning deficits in each data set using intra-survey benchmarks.

…

Applying the above method we calculate the difference in average achievement between Quintiles 1 (poorest 20% of students) and quintile 5 (wealthiest 20% of students) for the different surveys and then convert these into a common standard-deviation metric. The difference between quintiles 1 and 5 is 28 percentage points in NSES Grade 3, 130 SACMEQ points in Grade 6, and 122 TIMSS points in Grade 9. These different metrics are not directly comparable and there is no simple way of equating the scores. Consequently we convert the differences into within-survey standard deviations and then, using the 0.3 standard deviation benchmark as one year of learning, one can say that this difference was equal to 4 Grade-levels in Grade 3[1] (NSES), 4.4 Grade-levels in Grade 6 (SACMEQ) and 4.7 Grade-levels in Grade 9 (TIMSS).

Lewin (2007) provides a useful conceptual model for the trajectory needed to reach a particular goal – in this case matric (Grade 12). He refers to an ‘on-track-line’ and an ‘off-track-line’ where the off-track-line is any line below the on-track-line. In the present example, the on-track-line is calibrated to be equal to the average performance of Quintile 5 students.

To illustrate the above in a graph, we set the average Quintile 5 achievement to be equal to the Grade-appropriate benchmark such that the learning trajectory of these students are on the “on-track” trajectory and will reach matric (Grade 12) performing at roughly a Grade 12 level. We then calculate the difference between this ‘benchmark performance’ and the average performance of Quintiles 1, 2, 3 and 4 and then convert this difference into Grade-level equivalents using 0.3 standard deviations as equal to one Grade-level of learning. In doing so, we essentially create a learning trajectory spanning from Grade 3 (NSES) to Grade 9 (TIMSS) with linear projections for those Grades where we do not have data (Grade 7, 8, 10, 11 and 12). The exact figures for all calculations are provided in the online appendix. Figure 6 below shows the likely learning trajectories of the average student in each quintile of student socioeconomic status.

Figure 6 shows that the average student in Quintile 1, 2 and 3 is functioning at approximately three Grade-levels lower than the Quintile 5 benchmark in Grades 3, 4, 5 and 6. Observing average performance by quintile in Grade 9 shows that the difference between Quintile 1, 2 and 3 students and Quintile 5 students (the benchmark) has now grown to more than four Grade-levels. If it is assumed that Quintile 5 students in Grade 9 are functioning at roughly a Grade 9 level, then Quintile 1 and 2 students are functioning at roughly a Grade 4.5 level in Grade 9. The trajectory lines, one for Quintile 5 and one for the average of Quintiles 1-4, show that in Grade 3 there already exist large differences in performance (approximately three Grade-levels) and that by the time children enter Grade 9 this gap in performance has grown to about four Grade-levels. The linear trend in performance between these two groups suggests that if the same number of students in Quintiles 1-4 in Grade 9 continued in schooling until Grade 12 (i.e. no drop out between these two periods) they would be functioning at approximately 4.9 Grade levels lower than their Quintile 5 counterparts in Grade 12 (1.5 standard deviations lower).

Figure 6

…

Returning to Lewin’s (2007) notion of an “on-track” progress line, perhaps the most important conclusion arising from this conceptual framework is that any performance below the “on-track” line creates an increasing gradient of expectation as the pupil moves into higher grades. This expectation is what is required by the curriculum to reach the goal (passing the grade 12 exam, for example) relative to where the student is at the present. As students’ learning deficits grow, the gradient of what needs to be achieved to reach the goal then progressively steepens to the point where it enters what Lewin (2007, p. 7) refers to as a ‘Zone of Improbable Progress.’ For example, the improvement that is required to bring the average Grade 9 Quintile 1 student in South Africa up to the required benchmark by Grade 12 is unrealistic given that they are performing at roughly a Grade 5 level in Grade 9. By contrast, the gradient of achievement required to bring the average Quintile 1 Grade 3 pupil up to the required benchmark by matric is slightly more manageable. The clear conclusion arising from this analysis is that intervening early to correct and prevent learning deficits is the only sustainable approach to raising average achievement in under-performing schools.

What we would add to this conclusion is that the root cause of these weak educational outcomes is that children are acquiring debilitating learning deficits early on in their schooling careers and that these remain with them as they progress through school. Because they do not master elementary numeracy and literacy skills in the foundation and intermediate phases, they are precluded from further learning and engaging fully with the Grade-appropriate curriculum, in spite of being enrolled in school. Lewin (2007, p. 10) refers to these children as ‘silently excluded’ since they are enrolled and attending school but learning little. Importantly, these children are precluded from further learning, not because of any inherent deficiency in their abilities or aptitudes, but rather because of the systematic and widespread failure of the South African education system to offer these students sustained and meaningful learning opportunities. Indeed, many children from poorer backgrounds have both the ability and the desire to succeed, and when provided with meaningful learning and remediation opportunities, do in fact succeed (see Spaull et al, 2012 for an example).

…

The clear policy recommendation which proceeds from these findings confirms what is becoming increasingly accepted, that any intervention to improve learning in South Africa needs to intervene as early as possible. Given South Africa’s egregiously high levels of inequality, it should come as no surprise that poor children in South Africa find themselves at a nexus of disadvantage, experiencing a lack of social, emotional and cognitive stimulation in early childhood. These children then enter a primary school system that is unable to equip them with the skills needed to succeed in life, let alone to remediate the large learning deficits they have already accumulated to date.

…

When faced with limited resources and a choice of where to intervene in the schooling system, the counsel from both the local and international literatures is unequivocal; the earlier the better. The need to focus on the primary Grades, and especially the pre-primary years, is not only driven by the fact that underperformance is so widespread in these phases, but also because remediation is most possible and most cost-effective when children are still young (Heckman, 2000). Due to the cumulative negative effects of learning deficits – particularly for vertically-integrated subjects like mathematics – it is not usually possible to fully remediate pupils if the intervention is too late (i.e. in high school), as too many South African interventions are. Nobel Laureate Professor James Heckman summarises the above succinctly when he explains that:

“Policies that seek to remedy deficits incurred in early years are much more costly than early investments wisely made, and do not restore lost capacities even when large costs are incurred. The later in life we attempt to repair early deficits, the costlier the remediation becomes” (Heckman, 2000, p. 5).

Full paper available here.

Last week I was fortunate enough to visit 9 primary schools in one of the remotest parts of our country – the rural Eastern Cape. Some of the schools had no toilets, others had no electricity and many were simply falling apart. The trip was planned by the Legal Resources Centre who are visiting 200 schools in the rural Eastern Cape as part of an ongoing court-case around the eradication of mud-schools in the country (see here for an overview of the litigation). (Given that this is an ongoing court-case and to respect the anonymity of teachers and principals I won’t mention the names of the schools in my discussion below). My interest in tagging along was to find out more information about the learning outcomes in these schools, and the views and concerns of teachers and principals. I am increasingly of the opinion that large-scale quantitative research, if not complemented by on-the-ground experience, misses much of the picture and so I wanted to try and understand where things breakdown and why. We often have wonderful policies but disastrous implementation and abysmal outcomes – why is that? Where is the break in the chain and what causes it?

In each school I spoke to the principal and a teacher and looked for evidence of work in the students’ exercise books and workbooks and spoke to some of the students. I would say that a lot of what we found is reiterated in the literature (often countless times) and I’ll include links to that literature as I go through some of what I found…

(1) The learning environments in almost all of these nine schools was truly shocking. The picture below shows the classroom environment in one of the schools – the grade 1’s are on the left of the picture and the grade 2’s are on the right. (All the schools we visited were multi-grade schools where one teacher teaches two or three grades, usually in one classroom – the principal is always a teacher as well. This is seen as the only feasible option when learner numbers in an area are very low). This school does not have enough buildings or classrooms and so this classroom also doubles up as the  kitchen and sometimes also as the staff room. The person in the green on the left of the picture was a volunteer who helps make the food for the children. The National School Nutrition Programme (NSNP) provides a hot meal to over 9 million South African students every day – this is one of the major successes of the Department of Basic Education and one for which we should all be proud (see also Kha Rhi Gude – ditto, thanks Veronica McKay!). The problem with the setup at this school, as the principal told us, is that the children stop concentrating as soon as they can smell the food being made – (usually prepared from 9am to be ready before 10am), but this school has no other option – there are no other rooms to use.

One school we visited overcame this problem by building their own “kitchen” using mud and sticks…

Some of the schools we visited had no toilets whatsoever (“The children use the bush”). This is despite the fact that often the municipality has installed toilets for nearby residents (sometime 150m away) but won’t build any for the school because that is not their mandate! Talk about a lack of government integration/communication. In some schools it is not the lack of space or toilets that was the problem but the inadequate roofing. In this school the tin roof on the one side of the classroom is full of holes. I asked the teacher what she does when it rains and she said, “Oh, then we all sit on this side of the classroom.” She teaches both Grade 1 and Grade 2 in this classroom.

In my discussion with this teacher, I asked her when they start teaching English at the school. According to the curriculum document (CAPS) teachers are required to teach English for  2/3 hours per week in grades 1 and 2 and 3/4 hours in grade 3. The teacher replied and said that she teaches English FAL from Grade 1. However when I was walking around the classroom and came across the timetable for grades 1 and 2 I saw that English did not feature in the timetable. This is problematic. Unless children slowly start increasing their vocabulary and competence in English in grades 1-3, the transition to English as medium-of-instruction in grade 4 is extremely difficult.

In another school we visited they had built on a tin-classroom extension onto the brick school in order to accommodate the learners. The tragic story behind this school is that it was built by the community because the next closest primary school is across the river in the valley and too many children were drowning trying to cross it to get to school. Unbelievably tragic.

(2) There is extremely little learning taking place in these schools.

In every school that I went to (except for one) there was very little evidence of learning or work. Paging through a student’s workbook, you will typically find that of the 80 pages in the Term 1/2 workbook, only 15-20 pages will have anything written on them (this despite the fact that we are now in term 4). The pages with the work on them are spread throughout the book so that you will find one or two pages of work and then 10 pages that are blank and then another page of work etc. It’s also not as if these students are working extensively in exercise books – in the exercise books that I looked at you will find one exercise every two weeks (sometimes once a month) and it will be extremely basic. In grade 1 it would be things like drawing a picture and writing a word. However, even when students are working in the workbook, it’s clear that they don’t actually know what they are doing. To provide some examples:

First is first…

The student below is clearly just writing whatever has been underlined, irrespective of what it means.

In most of the schools I asked the Foundation Phase teacher if she thought that her Grade 3 students would be able to read this short story in English:

Most of the teachers said that they thought their Grade 3 students would be able to read it. I asked if she could pick one student to read the story with me outside. To get the children relaxed I starting with an extremely elementary text, “The kids read books” (exercise from Gr 1 FAL book) which almost all the children could read. Then they would try and read the “Fun in the Sun” story (exercise from Gr1 HL workbook). Most of the selected children from the 9 schools (likely to be the better performing kids in the class) could read the story aloud in English – sometimes very slowly. However only 1 of the 9 students could answer the question “What colour is Sam’s cap?” and the same student was the only one who could answer the question “What colour is the mat?” Clearly these children are not ‘reading for meaning’ – i.e. they are illiterate. And this is in November 2014. These students will switch to English as medium of instruction in February 2015 when they enter grade 4 (see here for a nationally representative discussion and here for an excellent qualitative study on reading)

(3) Union meetings and departmental meetings are only ever held during school hours. One of the things that I was interested to find out was how often teachers and principals attend departmental meetings/training and union meetings and when these meetings are held. In this case all of the teachers I spoke to were part of SADTU (for a breakdown of SADTU membership by province see here). I asked the principals and teachers the following question “On which day of the week are departmental meetings usually held?” “On which day of the week are union meetings held?” Without any exceptions, all principals and all teachers said that the meetings happened during weekdays (various weekdays Mon-Fri) and during school hours  (usually 9am-1/2pm). There were approximately 4 union meetings a year and 3-4 departmental meetings or training days a year. One principal told me “In 2013 there were 5 workshops. This year we have had three workshops about how to mark the ANAs.” Yet another principal: “We have shut the school 4 or 5 times this year because there was training for the principal and one teacher on the same day and we are only 3 teachers.” This was something that I expected to hear but still found it very frustrating that everyone thought that it was totally OK to have meetings during school time. I think the one that infuriated me most was when someone told us that SADTU had organized a prayer meeting during school time and invited teachers to attend.

I include below the “Rules” poster that was put up on the wall in one of the schools we visited:

If you battled to read it it says “(1) Learner must wear school uniform. (2) Educators must attend, clinics, workshops, seminars, and any governmental meetings. (3) All educators must be responsible for their duties delegated to them. (4) Educators must sign leave forms on their absence from school. (5) No educators are allowed to be out of classes in a teaching time, chatting, and discussing their problems.” I reserve the right to comment on this at a later date – there is so much to be said, mainly about priorities.

When I asked one teacher why they did not have the meetings on Saturdays the teacher replied “Because we are not paid to work on Saturdays. Why would we go if we are not paid?” Now I didn’t say this at the time, but that’s actually wrong. As part of teacher’s employment package, they are paid for 80 hours of ongoing professional development  (see page ix of this report). When I asked teachers what they did with the students when they went on training, they said that the other teachers looked after them or they told them not to come to school that day. In one of the schools that we visited the Grade R teacher was at a meeting and so the Grade R class was left unattended (the other two teachers were teaching their own classes). There were no books in the classroom and the students were just keeping themselves busy talking to each other and walking around:

The SACMEQ study of 2007 asked principals a variety of questions about what they did with students when teachers were absent. There were 392 schools included in the study drawn as a nationally representative sample of primary schools in the country (see report here). I include the breakdown of the answers to those questions by province:

More than 40% of principals in the Eastern Cape said they they send students home “sometimes” and leave students unattended “sometimes” when a teacher is absent (on any given day about 10-12 % of teachers are absent from school – see 2010 HSRC report here). A number of people working in the field have told me that if you ever want to do teacher training or have meetings it must happen during school hours or else the teachers will not come. Something is seriously rotten in the state of Denmark. However not all principals are as complacent and compliant as these principals. One remarkable principal I spoke to had the following to say about teacher absenteeism: “Each and every week there is a memorial service. We are dying like flies. But you cannot have a teacher away every single week. I cannot.”

(4) Intense union involvement in the appointment of educators, principals and district officials. One of the common threads that came through in many of my discussions was the involvement of the unions – particularly SADTU – in the appointment and promotion of teachers, principals and district officials. This did not come as much of a surprise – there is currently a Ministerial Task Team looking into the sale of teaching posts. The thing I was interested in is how this actually happens. One of the people I spoke to was particularly insightful on this…let me include some excerpts from our conversation:

Nic: “You mentioned that the unions are involved in appointments and promotions, can you tell me how that works?”

X: “When you are selecting a Head of Department (HOD) for the school there are 2 parents from the SGB and 1 teacher, the principal is there but cannot vote. In the rural Eastern Cape many of the parents are not well educated. They know nothing about laws so it is just the principal and the teachers.  SADTU can very easily influence the parents through the teacher. If SADTU does not get the person that they want they will say there was an irregularity in the interview process. I once encouraged the parents to appoint a good mathematics teacher for my school and they did, but they were not SADTU’s choice so they had the teacher removed. They re-advertised the post but without subject specification because there was no SADTU member who had maths or science. I am now stuck with someone who is babysitting mathematics and my results are terrible. My ANAs are very low in mathematics. And you cannot challenge it.” [“Why can’t you challenge SADTU?”] They will go for you. They will accuse you of sexual misconduct and there must be an enquiry. They will accuse you of financial mismanagement. They will go for small things to catch you. You know you need 3 quotations if you buy something and you must write it down so that if you only have two or forgot to write it down, they will catch you. Most principals will make a small mistake. But these are honest mistakes. But they will catch you.”  “The Department is listening and and the union is managing. SADTU does not want to listen, they want to lead and they want to manage.”

Me: “Can you tell me about the appointment of district officials, curriculum advisors etc. do you know anything about how that works?” Response: “Yes, those are also appointed by SADTU. Everyone is looking for posts and if you want a post somewhere, SADTU can make it happen.”

Another principal had recently appointed a new teacher and I asked about the process and he/she said “We get given a list from the district office and pick someone from the list.” [“How were the people on the list selected?”] “We are in a rural area, we don’t know how those people got on the list.”

Unless we can figure out a way to eradicate these illegal practices and prosecute those involved, the Eastern Cape will remain completely dysfunctional. When I hear these stories piling up one on top of the other – always the same tune just in a different key – my blood starts to boil. People playing politics while Rome is burning. No one could possibly come up with any moral or ethical defence for the kinds of corrupt and collusive practices that permeates the entire education bureaucracy in the Eastern Cape (and many other provinces I’m sure). And the most tragic thing of all is that the burden of this cancer falls most heavily on the poorest of the poor, whose children will never receive a basic quality of education and never be able to develop their talents and personalities. They are condemned to unemployment, indignity and hereditary poverty…and so the cycle continues…

//

I intend to write a follow-up post where I discuss the infuriating issue of Nkandla-size mud-school replacements, as well as some of my other observations about the recent trip…

Further reading:

• Centre for Child Law report (2014) Mud to Bricks: A review of school infrastructure spending and delivery.
• Formative Evaluation of workbooks and textbooks – 2013 report by the Australian Council for Educational Research (ACER)
• I am still waiting for the NEEDU 2013 report to be released since the focus of that report was on rural schools. I’m not sure why the Minister has not released the report yet because I know it has been completed. Anyone know when it will be released?

(The article below appeared in the Mail & Guardian on the 21st of November 2014 – available on the M&G website here).

If you’re reading this sentence it means that somewhere, somehow, you learnt to read. Bravo! You acquired the magical skill of translating scribbles into language and making meaning from the print symbols on the pages and screens that permeate our lives.

It really is quite remarkable that a few scrawls on a page can make us weep with joy or seethe with rage as we engage with the heroes, villains and ideas of bygone or future eras. Imagine what your life would be like if you could not read. Imagine what school would have been like if you couldn’t read.

And yet, unfortunately, this is not an imaginary experience for thousands of South African children. It is their daily, lived experience. Constantly struggling to understand the words on a page, let alone deciphering the deeper meaning behind these funny dots and dashes. And this is not their fault.

The human brain is hard-wired to acquire language and almost all children can learn to read in just a few years if provided with the right teaching, resources and encouragement. However, many South African children do not attend schools where these necessary conditions are present. A number of South African studies have revealed that children who cannot read and write properly by grade four end up playing catch-up for the rest of their school days. These children never quite grasp what is expected from them, even as they are told they are failing and must try harder.

Let me explain some of the recent research findings on this very important topic.
In 2011 South Africa participated in an international study called PrePirls (pre-progress in international reading literacy study), which is aimed at assessing the reading ability of grade four children. The study examined a nationally representative sample of 341 primary schools drawn from across the country.

The reason for choosing to assess grade four is not arbitrary, but rooted in an understanding of when and how children learn to read.  The first three years of schooling are regarded as the “learning to read” phase, when children acquire the ability to decode text and convert print symbols into language.  In grade four they enter the “reading to learn” phase as they start acquiring new information through the skill of reading.

Children who cannot read properly by grade four are severely disadvantaged, because they cannot read fluently or read for meaning, and therefore don’t benefit much from higher grades. This places them in perpetual catch-up mode until they begin to approach matric and drop out of school in grades 10 and 11, as 50% of South African students do. Unfortunately the results of PrePirls are truly sobering.

If one looks at the reading achievement of these schools and splits the 341 schools into the better performing half (169 schools) and the worse performing half (172 schools) of the sample, the results speak for themselves. In the top half of schools, 10% of students were completely illiterate. That is to say that they could not locate and retrieve an explicitly stated detail in a short and simple text. These children cannot read at all.

In the bottom half of schools, an unbelievable 51% of students were completely illiterate! After four years of formal, full-time schooling, every second child in these 172 schools was completely illiterate. These 172 schools are statistically representative of half of South African primary schools. (These tests were done in the language that they had been learning in during grades one to three — an African language for most children, before switching to English in grade four).

These children who don’t learn how to read properly are then promoted to the next grade, but never manage to get their heads above water for the rest of their school days.

What to do?

Firstly we have to get the basics right in the Foundation Phase (grade one to three). We need a national reading campaign where all stakeholders (parents, teachers, principals, government officials, the minister, the president) all rally behind this goal: “Every child must read and write by the end of grade three.”

This is the very same goal that Brazil used as the core goal for primary schooling — with much success. One prominent South African researcher, Elizabeth Pretorius, has identified four necessary criteria to ensure all children learn to read:

(1) Teachers need to understand when and how children acquire reading and comprehension skills, as well as understand how to teach reading;

(2) Children need easy access to interesting books in their own language and in English;

(3) Children need to be constantly motivated to read, with reading seen as a pleasurable activity by students and teachers, and

(4) Children need to be given plenty of opportunities to read in and outside of the classroom.

Sadly there is currently no systematic evidence about which of the many interventions currently being implemented in South Africa actually work, and if they do work, which is best. It is of fundamental importance that a national reading strategy be based on scientific evidence regarding what most improves the acquisition of reading in South African schools.

If we do not get reading right in grades one to three, any intervention later in the system will only have a small impact on learning, and consequently the life chances of the poor.  The later in life we attempt to repair early learning deficits, the costlier the remediation becomes. We simply must ensure that every child timeously acquires that magical skill of translating scribbles into language. Our education system depends on it.
//

For an excellent (and much more detailed) article on reading see Elizabeth Pretorius’ original article here. 

This article by Elizabeth Pretorius (UNISA) is easily one of the best academic articles I’ve read this year! I would strongly recommend that anyone interested in (1) reading/literacy, (2) the education crisis in SA, or (3) how to get out of the mess we are currently in, should read this article!

Pretorius, E. 2014. Supporting transition or playing catch-up in Grade 4? Implications for standards in education and training. Perspectives in Education. 32 (1), pp 51-76.

ABSTRACT

This paper describes an intervention programme that was originally intended to support transition to English as language of learning and teaching (LoLT) in Grade 4 in a township school, using a pre- and post-test design. Because the pre-tests revealed very poor literacy levels in both Zulu home language and English, the intervention programme was modified in an attempt to fast-track the learners to literacy levels more appropriate to their grade. This paper outlines the intervention, presents the pre- and post-test results of the English literacy assessments, reflects on the effects of the intervention, and briefly considers some of the reasons for the initial poor literacy performance. Finally, a model for literacy development in high-poverty contexts is proposed to minimise the need to play catch-up in the Intermediate Phase.

• Earth meet Big Brother, Big Brother – Earth. WIRED’s latest article on Edward Snowden is profoundly shocking. The NSA has the potential to track “everyone” in a city using the MAC address from cellphones, it “accidentally” sunk the Internet in Syria while trying to spy on Syrian civilians, and frequently passes unredacted data on to Israel about Palestinian American citizens. WTF.
• “The non-monetary benefits of learning may be difficult to measure, but they shape and determine what we recognize to be the quality of our life” – from “Why academic achievement matters (via Harry Patrinos).
• After watching the trailer to Boyhood I’ve fallen in love with this song by Family of the Year.
• Henry Miller on friendship – worth reading (thanks Clint Clark)
• Next year’s IEA conference is happening in Cape Town (22-23 June 2015). For those who are doing work on IEA data (PIRLS/TIMSS) the deadline for submission is 1 December 2014.
• The Free State Department of Basic Education is clearly trying to game the ANAs – see this article. We really need to be thinking about the potential unintended consequences of the ANAs. I think the ANAs are a positive development in the SA education system, but we need to be paying closer attention to the potential unintended consequences of the ANAs and how we can minimize these unintended consequences. For one thing, the ANAs are not ready to be used as an accurate indicator of student or school performance across the grades or over time. 
• In case you were wondering what your skin looks like with and without sunscreen, see here. *will never not wear sunscreen again, or use double negatives*
• “The impact of national and international assessment programmes on education policy, particularly policies regarding resource allocation and teaching and learning practices in developing countries” – Systematic review of the evidence by ACER 2014

I don’t usually repost articles that I’ve written in the past but given that the World Economic Forum has recently released it’s 2014/15 Global Competitiveness report I thought it makes sense. The article below first appeared in the M&G on the 13th of June. If you’re part of the media and want to quote me on something RE the 2014/15 rankings you can use any of the following:

• “These results are completely and utterly preposterous. Of course South Africa’s education system is in crisis and performing worse than other middle-income (and some low-income) countries) but it definitely isn’t the worst in the world.”
• “The methods used to calculate these education rankings are subjective, unscientific, unreliable and lack any form of technical credibility or cross-national comparability.”
• “The mistakes in the WEF’s methodology are so egregious that one needs only look at the list of countries and their respective rankings to appreciate how ridiculous they really are – failed states rank above modernising middle-income countries. How on earth do Japan and Cote d’Ivoire have the same ranking for the quality of their maths and science instruction? This is not science, this is an unscientific opinion survey”
• “The WEF has seriously undermined its own technical credibility by reporting these ridiculous education rankings. Until it rectifies its methodology, no one should take the rankings seriously.”

//

In the past two weeks the South African media has had a field day lamenting the state of maths and science education in the country. This is because the World Economic Forum (WEF) recently ranked South Africa 148th (out of 148 countries) on the quality of its maths and science education.

Let me cut to the chase and say, unequivocally, that the methods used to calculate these education rankings are subjective, unscientific, unreliable and lack any form of technical credibility or cross-national comparability. I am not disputing that South Africa’s schooling system is currently in crisis (it is), or that South Africa performs extremely weakly relative to other low- and middle-income countries (it does). What I am disputing is that these “rankings” should be taken seriously by anyone or used as evidence of deterioration (they shouldn’t).

The mistakes in the WEF’s methodology are so egregious that one needs only look at the list of countries and their respective rankings to appreciate how ridiculous they really are. How is it possible that the quality of maths and science education in failed states such as Chad (ranked 127th on the WEF list), Liberia (125th) and Haiti (120th) is better than modernising middle-income countries such as Brazil (136th) and Mexico (131st)? How do countries such as Madagascar (82nd) and Zambia (76th) outrank countries such as Israel (78th), Spain (88th) and Turkey (101st)?

Preposterous
Although these preposterous rankings sound like an April Fool’s joke gone wrong, they are reported without qualm on page 287 of the WEF Information Technology Report 2014. Even a cursory analysis of the faulty ranking methodology the WEF employed shows how it is possible to arrive at these outlandish “rankings.” The WEF asked between 30 and 100 business executives in each country to answer questions (relating only to their own country), using a scale of one to seven to record their perceptions, with one representing the worst possible situation and seven the best possible situation.

The question relating to maths and science education was phrased as follows: “In your country, how would you assess the quality of maths and science education in schools?” with “one” being “extremely poor – among the worst in the world”, and “seven” being “excellent – among the best in the world”.

In South Africa, 47 business executives were surveyed for these rankings. On the question relating to maths and science, the average score among these 47 executives was 1.9, indicating that the vast majority of these South African business executives believed that the quality of maths and science education in the country was “among the worst in the world.” Yet this is really just a measure of the perceptions of these 47 businessmen, as the department of basic education has correctly pointed out.

By contrast, when the 55 Malawian and 85 Zambian business executives were surveyed, they were more optimistic about the maths and science education provided to students in their countries, yielding average scores of 3.2 and four respectively.

Outperform
This explains why Malawi ranks 113th and Zambia ranks 76th whereas South Africa ranks 148th. Yet we know from objective cross-national standardised testing in the region that Zambia and Malawi are two of the few countries that South Africa actually does outperform.

Clearly the ratings given by these business executives are subjective and dependent on their particular mental reference points, which obviously differ by country. These 47 South African executives were not asked to rank South Africa relative to other specific countries – such as Madagascar, Malawi or Mali – only relative to “the world”.

Although the perceptions of business executives are important in their own right, it is ludicrous to use these within-country perceptions to rank “the quality of maths and science education” between countries; particularly when we have objectively verifiable, cross-nationally comparable scientific evidence for maths and science performance for at least 113 countries.

Looking at South Africa specifically, we participate in two major cross-national standardised testing systems that aim to compare the mathematics and science performance of South African students with that of students in other countries. The Trends in International Mathematics and Science Study (Timss) tests grade eight students from middle- and high-income countries, and the Southern Africa Consortium for Monitoring Educational Quality (Sacmeq) study tests grade six students from 15 countries in sub-Saharan Africa.

Worse than South Africa
Of the countries participating in Sacmeq, South Africa came 8th in maths, behind much poorer countries such as Kenya (2nd), Swaziland (5th) and Tanzania (3rd), but ahead of Mozambique (10th), Namibia (13th), Zambia (14th) and Malawi (15th). Although this situation is no cause for celebration, it does show that these countries – which outrank South Africa in the WEF rankings – are in fact doing worse than South Africa in reality.

If we look beyond Africa to the Timss rankings, South Africa performs abysmally. Of the 42 countries that participated from around the world (including 21 developing countries), South Africa came joint last with Honduras in 2011. This should shock us to the core. But it does not mean that we have the worst education system in the world. Rather, we have the worst education system of those 42 countries that take part in these assessments.

There is a big difference. Only 21 developing countries took part in these assessments, but there are around 115 developing countries in the WEF tables. The fact that Mali, Madagascar, Liberia and Haiti (for example) do not take part in these assessments means that business executives in these countries have very little reliable information on the quality of education in their countries.

In South Africa the basic education department has wisely chosen to take part in these assessments so that we have reliable information on the performance of our education system, however low that performance might be.

Continuing participation
This is one thing that the department should be commended for –that is, for continuing to participate in these assessments, which provide valuable information, despite being lambasted by their findings.

Perhaps the best example of how flawed the WEF methodology is is illustrated by comparing Indonesia and Japan on the WEF rankings and on the well-respected Organisation for Economic Co-operation and Development’s Programme for International Student Assessment (Pisa) rankings, which also tests math and science, as does Timss.

In the WEF rankings, executives in Indonesia and Japan both gave an average score of 4.7 for the quality of maths and science education in their respective countries. This placed Japan 34th and Indonesia 35th of the 148 countries. Yet, of the 65 countries participating in the 2012 round of the Pisa maths and science testing, Japan came 7th (out of 65) and Indonesia came 64th. Go figure.

Although there are some early signs of improvement in the South African education system, we know that things remain dire. South African students perform worse than all middle-income countries that participate in assessments, and even worse than some low-income African countries.

But to claim that South Africa has the worst quality of maths and science education in the world, and to use executives’ perceptions over scientific evidence to do so, is irrational and irresponsible.

The WEF has seriously undermined its own technical credibility by reporting these ridiculous education rankings. Until it rectifies its methodology, no one should take the rankings seriously.

“We need more than a stab in the dark” – Hamsa Venkat & Nic Spaull

[This article first appeared in the Mail & Guardian on the 8th of August 2014.]

Almost everything that is associated with mathematics in South Africa is either contentious or depressing or both. One could talk about the flawed World Economic Forum rankings, the confusion around the pass mark in matric, or the fact that only 3% of Grade 9 students reached the “High” or “Advanced” mathematics benchmark in the 2011 round of international student testing in Timss. However it is not our intention to bang the now familiar drum of low and unequal performance – the refrain that best characterises our schooling system. Of course we need to know how bad things really are, but we also need to know why they are so bad, and perhaps more importantly how we get ourselves out of this quagmire.

In grappling with these issues we believe that the national discourse around schooling needs to turn towards our most critical resource: teachers. No education system can move beyond the quality of its teachers. At its most basic level this is essentially what schooling is; the student and the teacher in the presence of content. Harvard’s Professor Richard Elmore has argued again and again that there are really only three ways to improve student learning at scale: (1) raise the level of content that students are taught, (2) increase the knowledge and skills that teachers bring to the teaching of that content, or (3) increase the level of students’ active learning of the content. In the South African context the evidence points towards huge deficits in the latter two areas: teacher content knowledge and pedagogical skill as well as low levels of curriculum coverage and cognitive demand.

Without ambiguity or the possibility of misinterpretation, all studies of mathematics teachers in South Africa have shown that teachers do not have the content knowledge of mathematics needed to impart to students even a rudimentary understanding of the subject. Unfortunately, almost all of these studies have been small-scale localised initiatives aimed at testing teachers in only a few schools or at most in one district. One recent exception was the 2013 analysis by Nick Taylor and Stephen Taylor of the SACMEQ 3 (2007) data – the most recent nationally representative data on teacher content knowledge. At the end of their paper they concluded that, “The subject knowledge base of the majority of South African grade 6 mathematics teachers is simply inadequate to provide learners with a principled understanding of the discipline.” In a paper we published this week we extended Taylor and Taylor’s work and analysed the nationally representative SACMEQ data from a curricular perspective. We wanted to know what grade 6 mathematics teachers know relative to the curriculum that their students are expected to master (CAPS).

This is important to determine what level in-service and pre-service teacher training should focus on. Preliminary results from a Joint Education Trust study show that pre-service training courses offered by five South African institutions had large differences in the amount and the nature of mathematics on offer. Furthermore, in-service education is commonly piecemeal, and frequently related to ‘managing’ the curriculum and assessment rather than with promoting understanding and communication of mathematics.

The findings from our analysis were sobering. Based on the 401 Grade 6 teacher responses in the SACMEQ 3 sample, we found that 79% of South African grade 6 mathematics teachers have a content knowledge level below the grade 6/7 level band even though they are currently teaching grade 6 mathematics. It is also worth noting that our definition of grade-level-mastery was a relatively low benchmark – teachers only needed to score 60% of the items in a grade band correct to be classified as competent in that band. Breaking this grade band analysis down further, the following patterns of results were seen:

• 17% of the teachers had content knowledge below a grade 4 or 5 level
• 62% of the teachers had a grade 4 or 5 level of content knowledge
• 5% of the teachers had a grade 6 or 7 level of content knowledge
• 16% of the teachers had at least a grade 8 or 9 level of content knowledge

Our analysis also confirmed particular weaknesses on problems relating to ratio and proportion, and multiplicative reasoning more generally – the kind of thinking that underlies many tasks involving fractions and decimal working as well.

While sobering, this analysis is useful for policy purposes and useful for thinking pragmatically about primary mathematics teacher education and development. The results suggest the need to begin work at the level of concepts at lower levels (Grades 4 and 5) in order to build more solid foundations of key ideas, rather than starting with higher-level mathematics.

We would argue that many of the problems we see in South African schools often have their roots in low levels of teacher content knowledge. When teachers lack confidence in the subject they are teaching this leads to two consequences. Either they do not cover those parts of the curriculum with which they are uncomfortable or they restrict classroom interactions to low-level problems that limit students’ opportunity to learn. Gaps in content knowledge also lead to highly disconnected mathematics teaching. This works against helping students to see connections between mathematical ideas, connections that are important for flexible and efficient problem-solving.

There are some signs of mobilization in the education field. The Association of Mathematics Educators of South Africa established a mathematics teacher education group in 2013 and has begun gathering information on pre-service course offerings. The Joint Education Trust study nearing completion is doing the same for the Intermediate Phase level. The Department of Basic Education has started preliminary work on developing tests which can be used to identify which teachers have critically low levels of content knowledge. All these initiatives are commendable and show promise, but the key obstacle to progress remains a lack of evaluation of in-service teacher training programs.

We know that content knowledge is not the whole story: good mathematics teaching requires a host of practical and interactional skills, but deep and connected content knowledge is a critical base. In researching our paper, we were unable to find evidence of any intervention that has been shown to raise mathematics teacher content knowledge at any scale in South Africa. Not a single one. Programs need to be piloted and evaluated before they are scaled up and only scaled up if they actually work. They should also be evaluated at different scales. Models that work for 10 schools may not work for 100 schools. What works in Gauteng may not work in the Eastern Cape. In the absence of rigorous evaluation we are shooting in the dark on a wing and a prayer. Our teachers deserve better.

There are moves towards more open discussions about problems related to teachers’ mathematical knowledge and greater consensus around the need for longer term interventions and evaluation of our development models and efforts. We believe that our findings and those of others, contentious as they might be, are important to face and acknowledge if we are to develop intervention models and content that build from the ground as it currently stands towards the improved mathematical outcomes that we all so desperately want to see.

//

Professor Hamsa Venkatakrishnan holds the position of SA Numeracy Chair at Wits University. Nic Spaull is an education researcher in the Economics Department at Stellenbosch University. Their joint research paper can be found at: http://www.ekon.sun.ac.za/wpapers/2014/wp132014/wp-13-2014.pdf