Category Archives: Uncategorized

An Introduction to Quantitative Data Analysis for Researchers in Education (8-12 June)

numbers

An Introduction to Quantitative Data Analysis for Researchers in Education

Stellenbosch University  |  8 – 12 June 2015

Overview

The aim of the “Introduction to Quantitative Data Analysis for Researchers in Education” course is to provide educational researchers with basic quantitative skills needed to interact with large-scale datasets in education. Although South Africa participates in a number of cross-national studies of educational achievement, few South African researchers possess the requisite quantitative skills needed to use these datasets (or others) in meaningful ways. This is problematic given that it is now widely acknowledged, both locally and internationally, that there is a serious need for education research that combines both qualitative and quantitative insights. It is for this reason that the course is primarily aimed at faculty and graduate students in education departments at South African universities. Other education researchers (from NGOs, think-tanks, policy forums) are also welcome to apply[1].

The course is intended to provide participants with a conceptual understanding of basic statistical procedures for quantitatively exploring and understanding data using a range of real-world data sets. The course introduces participants to measures of central tendency (mean, median, mode), non-central tendency (percentiles, quartiles, deciles etc.), and distribution (variance, standard deviation, inter-quartile range). Participants will learn how to calculate these statistics in STATA and also how to create graphs and tables that describe education data. The aim is always to equip participants with the skills needed to create their own graphs, tables and statistics in STATA and to interpret them accurately for their own purposes. All lectures and practicals will use South African education data, including SACMEQ, PIRLS, TIMSS, ANA and matric, as well as some administrative data (EMIS).

The course will be from 9am to 4:30pm each day from 8-12 June 2015 and will be held at Stellenbosch University. Each day there will be two interactive lectures and one extended practical session in STATA. No prior knowledge of STATA is necessary and participants need not have their own STATA license. The course will be held at a computer lab at Stellenbosch University and all computers will have STATA installed on them. The lecturers for the course will be Nic Spaull and Servaas van der Berg assisted by other ReSEP researchers.

Aims

The aim of the course is to take participants that have a basic understanding about statistics (means, frequencies, percentages etc.) and to help them to answer the following questions:

  • What is a standard deviation and how does one use and interpret it?
  • What is a standard error and how does it relate to a confidence interval?
  • What is a median? a mode? a percentile?
  • How does one decide how large a sample needs to be?
  • What is the difference between correlation and causation?
  • When can we say that one thing causes something else?
  • How do you create and interpret graphs in STATA? Including histograms, line charts, stacked bar charts, scatterplots, boxplots and kernel densities?

By the end of the course participants should be familiar enough with STATA and the basic concepts of descriptive statistics that they can generate and interpret graphs and tables in STATA by themselves. They should also have a basic understanding of the major educational datasets in South Africa (PIRLS, TIMSS, SACMEQ) and be able to answer their own questions using these datasets.

Costs

All costs for the course (including the course-fee, course-materials, transport to Stellenbosch and accommodation in Stellenbosch) will be covered by the PSPPD programme.

Applications

To apply to attend the course, please send an email to carinebrunsdon@sun.ac.za with your CV and a brief motivation as to why you think the course will be beneficial to you and your research (maximum 1-page). Please make the subject line of your email “PSPPD Data Analysis course” and indicate if you will be requiring transport to, and accommodation in, Stellenbosch and which city you will be coming from. There are limited spaces available (approximately 30) and preference will be given to senior faculty and PhD students, although masters students are also encouraged to apply.

Accessibility

If there are any applicants with special learning needs please indicate this on your application and we will do our best to accommodate your needs. We encourage you to liaise with us and the Office for Students with Special Learning Needs (OSSN) at Stellenbosch University.

Deadline for applications: 30th April 2015.

[1] We will be running a separate, but similar, course for education policy makers later in the year. Any interested individuals should please contact us at carinebrunsdon@sun.ac.za.

//

The PDF version of this call for applications can be downloaded here.

Q&A with Carole Bloch

carole bloch

The aim of the Q&A series is to get an inside look into some of South Africa’s leading education academics, policy-makers and activists. This is the twenty-third interview in the series. Carole Bloch is Director of PRAESA.

(1) Why did you decide to go into education?

As a student, I taught guitar to children, then music appreciation to preschool children. I loved this experience and found that I connected really well with little children and was fascinated by their imaginations and the way they played and thought. After my BA at UCT, it was really luck that I got to do a PGCE in the UK… a long story, but I never looked back. I loved teaching, first teens with literacy learning problems, later preschoolers who played voraciously, with everything they could get their hands on. I experienced first-hand how to facilitate reading and writing as a personally meaningful, emergent process… I’ve been a teacher, and a learner in literacy education ever since.

(2) What does your average week look like?

I am a bit of an octopus these days – I have tentacles waving about in all directions with Nal’ibali. Keeping a national campaign moving along means having an overall vision at the same time as you are involved in details. There is ongoing networking and fund raising, overseeing and informing the literacy and literary information we put out across platforms, training and mentoring programmes and of course troubleshooting technical challenges, like newspaper supplements not arriving where they should on time and supporting colleagues, dealing with payments and staffing issues. Then there is always the daily email deluge! We communicate with great rapidity which means things can happen quickly, but I sometimes feel quite alarmed by the sheer volume of messages that come my way! The evenings and early mornings are often times to catch up with reading and trying clear my head to write.

(3) While I’m sure you’ve read many books and articles in your career, if you had to pick one or two that have been especially influential for you which one or two would they be and why?

It’s hard to choose just one or two: Illich Chukovsky’s 1960 classic From 2 to 5 on the extraordinary linguistic genius of young children and all of Vivian Paley’s books, especially The Boy who would be a Helicopter on the enormous literacy learning and general educative power of imaginative play and stories and Stephen Krashen’s Power of Reading from the 1990s which summarises the research on free voluntary reading (see http://www.sdkrashen.com/).

(4) Who do you think are the current two or three most influential/eminent thinkers in your field and why?

There are so many. Quite a few I don’t agree with, so I won’t mention them! For me, social anthropologist, Barbara Rogoff ‘s work on children participating as cultural apprentices in communities of (reading and writing) practice is really useful when thinking about what we need to understand about how reading culture development takes place. It fit’s amazingly well with the New Literacy Studies – Brian Street and David Barton et al’s conceptions of literacy as social practice – what I like especially about her is that she makes clear that people both join communities of practice and change them by their participation, such a critical insight for South African environments (eg Rogoff 1991, 2005).

Kenneth Goodman, though sadly much vilified I think is one of the greatest thinkers about the reading  process – it is in many ways really anticipating some of the recent insights from brain research – about how the senses work in general such as Chris Frith’s fascinating 2007 book Making up the Mind, on how the brain predicts (Goodman said that “reading is a psycholinguistic guessing game” (see eg Goodman, 1967). His work also helps us to understand that the process underlying reading is the same in any language (research with languages very different from English, like Chinese shows this), and so we don’t have to get bogged down worrying about that in literacy teaching with little children. There’s a really interesting article by him and others too which contests some of the neuroscientifc claims about how children read put forward by phonics proponents like Sally Shaywitz (see http://ericpaulson.wp.txstate.edu/files/2014/05/strauss_goodman_paulson_2009.pdf). I think there is so much differing ‘evidence’ that refers to teaching reading by experts in diverse fields, like linguistics, (who often tend to like to dissect languages and think when you learn to read you have to do this too) and also neuroscientists who do not necessarily understand how little children learn to read (eg Stanislas De Haene 2009). These can influence policy and have negative effects on the lives of young children – I mention some of this later.

5) What do you think is the most under-researched area in South African education?

Understanding the way babies and young children learn to speak, read and write in multilingual settings.

6)  What is the best academic advice you’ve been given?

I think two pieces of advice have stood out for me over the years – the one was from Neville Alexander who I worked closely with for two decades. In the days when very few people were working on ‘alternative’ approaches to young children’s literacy teaching, I sometimes would feel despondent when my ideas and approach seemed to fall on unresponsive ears. Neville used say “Don’t worry about what other people say or think – you know what you are doing, just get on with it”. I realised how significant a) support and b) conviction are to push on and keep going. Another person who helped me with a wise comment which I’ve never forgotten was Elsa Auerbach, an adult literacy specialist from Boston, who’s classic article also influenced me many years ago: http://linksprogram.gmu.edu/tutorcorner/NCLC495Readings/Auerbach-Sociocontemp_familyLit.pdf). When I asked Elsa how she would help teachers and teacher educators to deepen their literacy knowledge and understandings, given the huge disparities in education we were trying to address, she said to me very simply, that we all need the same kind of opportunities– in a nutshell to read, reflect and experience many demonstrations of good practice. This simple insight has guided me for many years as I’ve mentored others.

7) You are currently the director of PRAESA and involved with Nali’bali, for those that are unfamiliar with these organizations can you give an overview of their aims and approach and maybe some of your/their plans for the future?

PRAESA from its beginnings in 1992 was an NGO based at UCT involved in multilingual education, research, training and materials development – essentially to help transform children’s educational opportunities using the foundation of mother tongue based bilingualism. Our research and all development work has been embedded in the view that a home language or languages should be the bedrock for learning, used to deepen thinking and conceptual understanding (see www.praesa.org.za). Other languages can be learned and added to a child’s linguistic repertoire, rather than being replacements. The longer the home language is used, the more support the child is actually getting. As many of us are aware, this mother tongue based education has not been implemented, except for in experimental ways, I believe to the extreme detriment of the educational opportunities for the majority of children.

In 2012, we initiated the Nal’ibali reading-for-enjoyment-campaign. This grew out of the previous twenty years of literacy research and practical work in multilingual settings which used stories and home languages for language and literacy teaching and learning. Because the story form is universal to all of us and integral to the way our minds work, the obvious route to literacy learning is to inspire a love of reading among all children. So Nal’ibali aims to nurture storytelling and reading for personal satisfaction, particularly in children, but also in the adults who are their role models and nurturers. Nal’ibali involves an ongoing collaborative effort with many partners to help put into place the conditions that support the initial and ongoing literacy development of all children irrespective of class, linguistic or cultural backgrounds. We’re doing this through mentoring, workshops and collaboration with communities, supporting reading clubs, literacy organisations and trying to elicit the support of volunteers of all ages, integrated with a media campaign and the development of multilingual literacy resources and stories. Our vision is a literate society that uses writing and reading in meaningful ways and where children and adults enjoy stories and books (and of course non-fiction too) together as part of daily life. The mission – and clearly we all need to be involved for this to happen – individuals, NGO’s, universities, government and business – is to create the conditions across South Africa that inspires and sustains reading-for-enjoyment practices (www.nalibali.org and facebook nalibaliSA).

8) You have been heavily involved in research on early literacy in African settings, can you give us an overview of what we know about early literacy in African settings and also what we don’t know?

I can only talk about my views as to what I think we know and don’t know or somehow don’t acknowledge or value …

In a nutshell, we know that most children, irrespective of class, socio-cultural and linguistic background are capable of becoming competent, avid and creative readers and writers but that huge numbers of mainly African language speaking children don’t – and that the conditions that need to be in place for successful learning to take place are mainly in place for children of the elite only. We know that a combination of factors is involved and that these cut across home, community and school. But we don’t seem to widely appreciate the incredible importance of the ‘invisible’ literacy learning that takes place in the daily, informal community and home language ‘goings on’ of literate homes, and what it means when such learning, for whatever reasons, cannot take place.

We know that teachers ‘bring with them’ like children do, their literacy theories and practices into the classroom, and that a real stumbling block in the early years is how we still tend to train teachers to view their task as teaching skills as a priority over demonstrating and making possible the use of written language for personally meaningful reasons (This contributes to the learn to read/read to learn myth). We should know that this blocks many children off from highly effective learning strategies that could reveal them as exuberant emergent readers and writers that we expect from most young English speaking children. We don’t widely acknowledge, and maybe we don’t know, that the consequence is a cyclical one of adults tending to underestimate poor children’s capabilities in formal education situations, believing the children are struggling with ‘the basics’, when actually the struggle is that the basics of written language are being denied them!

We know that the push down from higher education is exacerbating this through the justification of curricula that package skills and knowledge in ways that override considerations about how to motivate young children ‘s enormous learning capacity. Global forces push down too – a current example is an assessment packages like EGRA, which grew from the USA ‘s DIBELS, that has caused so much heartache and stress for so many families (see The Truth About Dibels, Goodman 2006).

We know that low status and use of African languages for print functions (including the dearth of fiction and non fiction) means fewer adults are leisure readers. But we don’t widely value or address the fact that it seems extremely difficult to teach others to read when you don’t have your own repository of knowledge and stories arising from the texts you’ve read over time, to draw from – with the overwhelming effect of poor reading habits being that you tend not to have what it takes to reap the benefits from and pass on a passion for knowledge and story to others.

Given what we do know, we don’t know why government (with support from business) seems unable to invest with unflinching determination in the translation of desirable world texts, including ones from Africa, to support African publishers to produce a steady flow of the books we need and order these to stock community and school libraries …. to inspire reading and creative writing among adults and children in African languages and English and also to use in the training and mentoring of adults to grow to know and manage these collections. We also don’t know why there is an insistence on making teaching so very hard for teachers and learning difficult for the majority of children living in South Africa after grade 3 by forcing teaching in a language often not known well enough to use with dignity and depth.

9)   If you ended up sitting next to the Minister of Education on a plane and she asked you what you think are the three biggest challenges facing South African education today, what would you say?

The first is the fact that it is a tragedy that we haven’t implemented our Language in Education Policy of 1997, but that it is not too late and that this needs leadership from government and lots of information – in fact a campaign – to allow parents the opportunity to appreciate the issues involved in educating their children from a language perspective – how they would come to realise that they do not need to choose between English or African languages but that both are possible, and desirable.

The second is related and I’ve raised already – that government needs to act on the fact that until publishing in African languages is supported in a serious way, so that these languages are used in print for high status functions, including literature – and more of our adult population starts reading regularly for personal satisfaction and for pleasure, many children won’t become readers and writers in the fullest sense.

The third, if she was still listening, would be to discuss how to use literature to find practical ways to create a different ethos among us – one that promotes and encourages empathy and respect for each adult and child living in South Africa irrespective of background. We’d gather people together to generate a curriculum of shared stories for children of all ages, from South Africa, Africa and the rest of the world, which reflect the highs and lows of humanity – to support the growth of a new generation of people who reject stereotyping and prejudice, and value what we share in common, as well as our differences.

10)   If you weren’t in education what do you think you would be doing?

I’d be a professional gardener as I love growing things, or I’d be a cellist.

11) What is the most rewarding and most frustrating thing about your job?

The most rewarding thing is seeing partnerships grow that are allowing so many people across South Africa to get involved in quite relaxed ways to enjoy the substance of reading: Seeing how my colleagues inspire others is humbling. Watching how interest in books grow in people of all ages when they are motivated. Seeing sceptical and weary-looking adults put on their playful hats to communicate with children and share stories in animated ways  – this is stunning for me.

Frustrations are around how hard it can be to convince others that sometimes the most simple seeming solutions are the most profound sometimes… and of course the time it takes to get things done, mainly because we just don’t have the capacity, either human or financial – and knowing how much more SA industry and government could do to help us change the desperate situation we face in literacy education.

12)   Technology in education going forward – are you a fan or a sceptic and why?

Total fan and total sceptic! I’m a fan of making the most use of technology. In Nal’ibali, for example, we offer a growing repository of free multilingual stories and guidance etc. on web, mobi and cellphone for adults to use with children of all ages. I think that the freedom it allows to create multilingually is extraordinarily powerful and I love the potential and sometimes actual freedom to share material without the rigid constraints of traditional publishing. But I’m a sceptic about the wisdom of proclaiming paperless learning. I don’t believe we should attempt to create an either-or situation. Particularly, but by no means only – for babies and young children we still want print on paper and books. I think we need to support and nurture our publishing industry more now than ever before.

13) If you were given a R15million research grant (and complete discretion on how to spend it) what would you use it for?

I’d facilitate a major qualitative research process on various aspects of Nal’ibali: I’d like to track groups of children living in different settings from home to reading club to school over a period; document the indicators of the effects of reading for enjoyment on motivation, engagement and achievement, literacy and school learning, family dynamics etc. I know that our only glimmer of hope to persuade policy makers, linguists and many involved in education that what we are doing is essential is ‘scientific’ evidence!

 //

Some of the others on my “to-interview” list include Veronica McKay, Thabo Mabogoane, Maurita-Glynn Weissenberg, Shelley O’Carroll, Yael Shalem, Linda Richter and Volker Wedekind. If you have any other suggestions drop me a mail and I’ll see what I can do.

Previous participants (with links to their Q&A’s) include, Johan MullerUrsula HoadleyStephen TaylorServaas van der BergElizabeth HenningBrahm FleischMary Metcalfe, Martin Gustafsson, Eric AtmoreDoron IsaacsJoy OliverHamsa VenkatLinda Biersteker, Jonathan ClarkeMichael MyburghPercy Moleke , Wayne Hugo, Lilli PretoriusPaula EnsorCarol MacdonaldJill Adler and Andrew Einhorn.

Developing an “access-to-learning” statistic: Combining access & quality in Sub-Saharan Africa

kenya

I’m currently in Washington D.C. for the Comparative and International Education Society (CIES) conference of 2015. Although it was -17′ C when I landed the weather has actually been lovely and DC seems like a lovely city to live in.

On Monday I presented two papers that I co-authored with Stephen Taylor on creating a composite measure of access and quality for 11 countries in sub-Saharan Africa. The presentation was very well received by the other delegates and a number of them have since commented on the importance of this research given where we are at cusp of the next set of post-MDG goal-setting, the “Sustainable Development Goals.” Personally I believe this is the most important research I’ve done to date, and it’s also the research of which I am most proud.

For those who would like to read the two papers I have included links below, and the PowerPoint slides can be found here. Both Stephen and I are trying to disseminate this research so if you know of anyone who might be interested in it please do forward it along to them. As always, comments and questions welcome.

Spaull, N., & Taylor, S., (2015). Access to what? Creating a composite measure of educational quantity and educational quality for 11 African countries. Comparative Education Review. Vol. 58, No. 1. (WP here).

ABSTRACT

The aim of the current study is to create a composite statistic of educational quantity and educational quality by combining household data (Demographic and Health Survey) on grade completion and survey data (Southern and Eastern African Consortium for Monitoring Educational Quality) on cognitive outcomes for 11 African countries: Kenya, Lesotho, Malawi, Mozambique, Namibia, South Africa, Swaziland, Tanzania, Uganda, Zambia, and Zimbabwe. Doing so overcomes the limitations of earlier studies that fo- cused solely on either quantity or quality. We term the new statistic “access to literacy” and “access to numeracy” and report it by gender and wealth. This new measure combines both quantity and quality and consequently places educational outcomes at the center of the discourse.

Taylor, S., & Spaull, N., (2015) Measuring access to learning over a period of increased access to schooling: The case of Southern and Eastern Africa since 2000. International Journal of Educational Development . Vol. 41 (March) pp47-59 (WP here).

ABSTRACT

This paper examines the extent to which increased access to primary schooling in ten Southern and East African countries between 2000 and 2007 was also accompanied by increased access to actual learning. We develop a measure of access to learning that combines data on education access and learning achievement to measure the proportions of children in the population (including those enrolled and not enrolled) that reach particular thresholds of literacy and numeracy. In all countries there was greater access to learning in 2007 than in 2000. These improvements in access to learning especially benefited girls and children from poor households.

“Some Children Are More Equal Than Others” [moving documentary on SA Educ]

Screen Shot 2015-02-18 at 11.07.21 AM

Today Stefan Gottfried released his short documentary on education in South Africa, aptly named “Some Children Are More Equal Than Others.” I came into contact with Stefan last year when he was working with the Legal Resources Centre. It happened to be at the same time I was visiting mud-schools in the Eastern Cape late last year (blog post here). The documentary was produced on a shoestring budget but clearly conveys the tragedy and anguish of hundreds of thousands of parents in South Africa. The motif that runs through much of the documentary is that the low quality of education offered to the majority of South Africa’s children becomes a poverty trap and prevents any form of social mobility. It reminded me of something I wrote 2 years ago:

“While the low-level equilibrium that South Africa finds itself in has its roots in the apartheid regime of institutionalised inequality, this fact does not absolve the current administration from its responsibility to provide a quality education to every South African child. After 19 years of democratic rule most black children continue to receive an education which condemns them to the underclass of South African society, where poverty and unemployment are the norm, not the exception. This substandard education does not develop their capabilities or expand their economic opportunities, but instead denies them dignified employment and undermines their own sense of self- worth. In short, poor school performance in South Africa reinforces social inequality and leads to a situation where children inherit the social station of their parents, irrespective of their motivation or ability. Until such a time as the Department of Basic Education and the ruling administration are willing to seriously address the underlying issues in South African education, at whatever political or economic cost, the existing patterns of underperformance and inequality will remain unabated” (from here).

Although I see the tragic education stats on a daily basis, it really hits home when you see the pain and anguish of black parents who see and understand that education is the route out of poverty for their kids and are trying their hardest to get their children into “good” schools but failing at every turn. Watch the documentary and ask yourself “What can I do to change this tragic, dangerous and deeply unfair situation?”

Curro private school assigns kids to classes based on race. WTF?!

race

It would seem that as far as education is concerned 2015 is not off to a good start. First we had the MEC for education in KZN explaining that her “visions of an ideal education system” include 18th century pseudo-science, namely streaming 6 year old children based on the size and shape of their skulls (phrenology) and the style of their handwriting (graphology). She has subsequently apologized to “everyone who felt offended” but did not retract the statement. Now this week Eyewitness News reports that a private school in Pretoria – Curro Roodeplaat – is assigning children to classes based on race?! They report that a group of 30 parents have signed a petition demanding an explanation from the school and the company. The explanation given by the company (Curro is one of the few for-profit private school chains in the country) was equally dumb-founding.  Curro Holdings’ regional manager Andre Pollard explained their rationale as follows:

“It’s not because we would like to segregate the whites, it’s just because of friends. Children are able to make friends with children of their culture.”

This sounds like a social/friendship-based version of ‘separate development.’ Fifty-two years ago the Bantu Education Act was made into law, segregating children based on the colour of their skin. Still today we are dealing with the aftermath of that one piece of legislation. By conflating education and subjugation it transformed our schools from sites of learning to sites of activism and resistance. It destroyed the culture of teaching and learning and created a lingering fear of education as an instrument of political subjugation.

With a history such as ours how can a school possibly argue that separating children based on race or ‘culture’ is necessary?! The sheer nerve that their regional manager can justify this racial segregation saying that children find it easier to make friends with children of their own culture is astounding and shameful. This is not the first time I have heard about racial discrimination in Pretoria’s Curro schools, but it is the first time I have seen it reported in the media. Currently there are 43 Curro schools across the country raising revenue of over R400 million in 2014. The company recently invested R1,5 billion in expanding their facilities with an aim of reaching 80 schools by 2020.  Only 4% of South African students attend Independent (i.e. non-public) schools but the vast majority of those are not-for-profit schools with about 0,4% attending for-profit schools like Curro.

While we might be tempted to brush this off as an isolated instance at a single Curro school, we need to ask why this behaviour is seen as acceptable by their regional manager? Curro needs to unequivocally condemn these practices and ensure that they are not practiced at any of their other schools. Looking at the speeches and policies at the time of our transition one can clearly see that education was (and is) seen as one of the most promising channels of integration, nation-building and transformation. The actions of this school show utter contempt for the democratic project in South Africa. There are many amazing Independent schools in this country who offer high quality education in a socially inclusive and culturally sensitive way. It would seem that Curro Roodeplaat is not one of them.

Special issue of SA Journal of Early Childhood Education: Call for abstracts

call for abstracts

The relatively new open-access SA Journal of Childhood Education has recently put out a call for abstracts (see below) for their special issue on “Priorities and policy-making in South African Education” (Guest editors: Nick Taylor and Thabo Mabogoane). Given the policy relevance of this special issue, researchers at ReSEP (including myself) will be submitting a number of abstracts for work we are currently doing and intend to do. If you’re doing work in this field I’d encourage you to do the same, it’s likely to be a great issue!

//

Special issue: Call for Papers “Priorities and Policy-making in South African Education”

 Guest editors: Nick Taylor and Thabo Mabogoa

Despite considerable expenditures and efforts to improve performance and reduce inequality, there is limited evidence of substantial improvements in educational outcomes, or the equalisation thereof. Periodic reviews of the evidence have shown a number of recurring themes that are especially characteristic of schooling in South Africa. These include unequal access to socially, emotionally and cognitively stimulating environments (both in the home and at school), insufficient resources, low levels of curriculum coverage, low levels of teacher content knowledge, inadequate support and training opportunities for in-service teachers, challenges associated with learning and teaching in a second language, low levels of accountability and high dropout in upper secondary school (among many others).

While education officials are often aware of these challenges, most policy-makers find it difficult to synthesise this evidence, which is necessary for prioritisation and resource allocation. Making sense from research is particularly challenging  when it is presented in isolation from other problem areas and only speaks to other research within its ‘silo’.  It is now widely acknowledged that if government policies are to have the largest possible impact, they need to be based on rigorous evidence and peer-reviewed research. Furthermore the National Development Plan (the government’s guiding framework) has emphasised the need for the “process of prioritisation and sequencing” if the plan is to be implemented. Such a process of prioritisation and sequencing requires rich, inter-connected evidence on education in South Africa.

Consequently, this call for papers focuses on education research in South Africa that speaks directly to policy-making and prioritisation. Papers that synthesise existing evidence across research areas in education are especially welcome.

Instructions for authors: www.sajce.co.za

Journal administrator: childhooded@uj.ac.za

Online submissions and author registration: www.sajce.co.za

Deadline for abstract submission: 31 March 2015
Deadline for full papers (of accepted abstracts): 30 June 2015
Intended publication date: November 2015

The SAJCE is accredited by the Department of Higher Education and Training and  has applied, through the Academy of Science of South Africa  (ASSAF) for listing  on the open journals platform, ScIELO 

Starting Behind and Staying Behind: Insurmountable learning deficits in mathematics (new Working Paper)

Screen Shot 2014-12-11 at 10.44.44 AM

A Working Paper that I co-wrote with Janeli Kotze was released today on the Stellenbosch Economic Working Paper site (available here). The paper has also been accepted for publication in the International Journal of Educational Development and should be out next year. I will include some excerpts from the paper for those who’d prefer the short version…

Abstract:

This study quantifies a year’s worth of mathematics learning in South Africa (0.3 standard deviations) and uses this measure to develop empirically-calibrated learning trajectories. Two main findings are, (1) only the top 16% of South African Grade 3 children are performing at an appropriate Grade 3 level. (2) The learning gap between the poorest 60% of students and the wealthiest 20% of students is approximately three Grade-levels in Grade 3, growing to four Grade-levels by Grade 9. The paper concludes by arguing that the later in life we attempt to repair early learning deficits in mathematics, the costlier the remediation becomes.

Excerpts:

Few would argue that the state of mathematics education in South Africa is something other than dire. This belief is widespread among academic researchers and those in civil society, and is also strongly supported by a host of local and international assessments of mathematical achievement extending back to at least 1995 (Howie & Hughes, 1998; Reddy, 2006; Fleisch, 2008; Spaull, 2013; Taylor et al., 2013). Many of these studies, and particularly those that focus on mathematics, have identified that students acquire learning deficits early on in their schooling careers and that these backlogs are the root cause of underperformance in later years. They argue that any attempts to raise students’ mathematical proficiency must first address these deficits if they are to be successful (Taylor et al., 2003). The present study adds further evidence to this body of work by using nationally representative data to provide some indication of the true size and scope of these learning deficits.

In South Africa, research in this area has generally focussed on in-depth localized studies of student workbooks and classroom observation (Ensor et al., 2009). For some examples, Carnoy et al. (2012) observe mathematics learning in Grade 6 classrooms from 60 schools in one South African province (North West) and compare these classrooms to 60 schools in neighbouring Botswana. On a smaller scale, Venkat & Naidoo (2012) focus on 10 primary schools in Gauteng and analyse coherence for conceptual learning in a Grade 2 numeracy lesson. Similarly Schollar (2008) conducted interviews and classroom observations as well as analysed a large sample of learner scripts to determine the development (or lack thereof) of mathematical concepts through the Grades.

Where the present research differs from these earlier studies is that it focuses on quantifying national learning deficits in general, rather than in specific learning areas. While the latter are essential for understanding what the problems are and how to fix them, analyses at the national level are also needed if we are to understand the extent and distribution of the problem, both of which are imperative for policy-making purposes. This is only possible by analysing multiple nationally-representative surveys of student achievement, which is the focus of the present study. The two core research questions that animate this study are as follows:

  • How large are learning deficits in South Africa and how are they distributed in the student population?
  • Do learning deficits grow, shrink or remain unchanged as students progress to higher Grades?

To answer these questions we analyse four nationally representative datasets of mathematics achievement, namely: (1) the Systemic Evaluation 2007 (Grade 3), (2) the National School Effectiveness Study 2007/8/9 (Grade 3, 4 and 5), (3) the Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ) 2007 (Grade 6), (4) and the Trends in International Mathematics and Science Study (TIMSS) 2011 (Grade 9).

The extant research on mathematics learning in South Africa strongly supports this conclusion with numerous researchers highlighting the inadequate acquisition of basic skills and the consequent negative effects on further learning. Taylor & Vinjevold (1999) summarise the findings from 54 studies[1] commissioned by the President’s Education Initiative and conclude that:

“At all levels investigated by [The President’s Education Initiative], the conceptual knowledge of students is well below that expected at the respective Grades. Furthermore, because students are infrequently required to engage with tasks at any but the most elementary cognitive level, the development of higher order skills is stunted” (Taylor & Vinjevold, 1999, p. 231).

This lack of engagement with higher order content is the prime focus of Reeves and Muller’s (2005) analysis of Opportunity-to-Learn (OTL) and mathematics achievement in South Africa, where OTL is the curriculum actually made available to learners in the classroom. Taylor et al. (2003, p. 129) in their book Getting Schools Working summarise succinctly the debilitating effects of cumulative learning deficits:

“At the end of the Foundation Phase [Grades 1-3], learners have only a rudimentary grasp of the principles of reading and writing … it is very hard for learners to make up this cumulative deficit in later years … particularly in those subjects that … [have] vertical demarcation requirements (especially mathematics and science), the sequence, pacing, progression and coverage requirements of the high school curriculum make it virtually impossible for learners who have been disadvantaged by their early schooling to ‘catch-up’ later sufficiently to do themselves justice at the high school exit level.’

And lastly, Schollar (2008) summarises the findings of the Primary Mathematics Research Project which looked at over 7000 learners from 154 schools in South Africa and concludes as follows:

“Phase I concluded that the fundamental cause of poor learner performance across our education system was a failure to extend the ability of learners from counting to true calculating in their primary schooling. All more complex mathematics depends, in the first instance, on an instinctive understanding of place value within the base-10 number system, combined with an ability to readily perform basic calculations and see numeric relationships … Learners are routinely promoted from one Grade to the next without having mastered the content and foundational competences of preceding Grades, resulting in a large cognitive backlog that progressively inhibits the acquisition of more complex competencies. The consequence is that every class has become, in effect, a ‘multi-Grade’ class in which there is a very large range of learner abilities and this makes it very difficult, or even impossible, to consistently teach to the required assessment standards for any particular Grade. Mathematics, however, is an hierarchical subject in which the development of increasingly complex cognitive abilities at each succeeding level is dependent on the progressive and cumulative mastery of its conceptual frameworks, starting with the absolutely fundamental basics of place value (the base-10 number system) and the four operations (calculation)” (Schollar, 2008, p. 1).

To provide an alternative measure of performance, we provide two examples of no-language items in NSES and show when students answer the question correctly – i.e. in Grade 3, Grade 4, Grade 5 or not by the end of Grade 5. Given that one needs to follow the same students from Grade 3 to 5 we limit the sample here to the panel sample of NSES students (8383 students). Figure 3 below shows a simple question testing two and three digit addition with no carrying. This is within the Grade 3 curriculum which states that students should be able to “perform calculations using the appropriate symbols to solve problems involving addition of whole numbers with at least three digits.” Although this is a Grade 3 level item and contains no language content, only 20% of Quintile 1-4 students could answer this correctly in Grade 3, with the proportion in Quintile 5 being twice as high (42%) but still low. While there is evidently some learning taking place in Grade 4 and 5, more than 40% of Quintile 1-4 children still could not answer this Grade 3 level problem at the end of Grade 5. In Quintile 5 this figure was only 22%.

Figure 3

Screen Shot 2014-12-11 at 11.29.06 AM

Figure 4 below shows a similar situation where the vast majority of Grade 3 children cannot answer this Grade 3 level problem. While some children learn the skill in Grade 4 or 5, the majority of children still cannot answer this problem at the end of Grade 5, despite it being set at the Grade 3 level.

Figure 4

Screen Shot 2014-12-11 at 11.30.35 AM

 Moving from learning deficits to learning trajectories

While the previous sections have identified the proportion of students that are not operating at a Grade 3 level, they do not provide much guidance in terms of learning trajectories into later Grades. The figures above show that some students are only learning part of the Grade 3 curriculum in either Grade 4 or Grade 5 and that many never seem to acquire these skills. However one cannot say to what extent they are also acquiring Grade 4 level skills in Grade 4 and Grade 5 level skills in Grade 5, although this is unlikely. This is because the NSES test was set at a Grade 3 level with only a small number of questions set at the Grade 4 level. One could use SACMEQ (Grade 6) and TIMSS (Grade 9) as measures of mathematical proficiency at higher levels, but these tests are not calibrated to be comparable to each other, or to earlier tests like the NSES. This is problematic since learning trajectories require data points distributed across the full range of educational phases which are comparable to each other both in terms of the content tested and the difficulty level of the tests. One alternative method to partially overcome the lack of inter-survey comparability is to measure the size of learning deficits in each data set using intra-survey benchmarks.

Applying the above method we calculate the difference in average achievement between Quintiles 1 (poorest 20% of students) and quintile 5 (wealthiest 20% of students) for the different surveys and then convert these into a common standard-deviation metric. The difference between quintiles 1 and 5 is 28 percentage points in NSES Grade 3, 130 SACMEQ points in Grade 6, and 122 TIMSS points in Grade 9. These different metrics are not directly comparable and there is no simple way of equating the scores. Consequently we convert the differences into within-survey standard deviations and then, using the 0.3 standard deviation benchmark as one year of learning, one can say that this difference was equal to 4 Grade-levels in Grade 3[1] (NSES), 4.4 Grade-levels in Grade 6 (SACMEQ) and 4.7 Grade-levels in Grade 9 (TIMSS).

Lewin (2007) provides a useful conceptual model for the trajectory needed to reach a particular goal – in this case matric (Grade 12). He refers to an ‘on-track-line’ and an ‘off-track-line’ where the off-track-line is any line below the on-track-line. In the present example, the on-track-line is calibrated to be equal to the average performance of Quintile 5 students.

To illustrate the above in a graph, we set the average Quintile 5 achievement to be equal to the Grade-appropriate benchmark such that the learning trajectory of these students are on the “on-track” trajectory and will reach matric (Grade 12) performing at roughly a Grade 12 level. We then calculate the difference between this ‘benchmark performance’ and the average performance of Quintiles 1, 2, 3 and 4 and then convert this difference into Grade-level equivalents using 0.3 standard deviations as equal to one Grade-level of learning. In doing so, we essentially create a learning trajectory spanning from Grade 3 (NSES) to Grade 9 (TIMSS) with linear projections for those Grades where we do not have data (Grade 7, 8, 10, 11 and 12). The exact figures for all calculations are provided in the online appendix. Figure 6 below shows the likely learning trajectories of the average student in each quintile of student socioeconomic status.

Figure 6 shows that the average student in Quintile 1, 2 and 3 is functioning at approximately three Grade-levels lower than the Quintile 5 benchmark in Grades 3, 4, 5 and 6. Observing average performance by quintile in Grade 9 shows that the difference between Quintile 1, 2 and 3 students and Quintile 5 students (the benchmark) has now grown to more than four Grade-levels. If it is assumed that Quintile 5 students in Grade 9 are functioning at roughly a Grade 9 level, then Quintile 1 and 2 students are functioning at roughly a Grade 4.5 level in Grade 9. The trajectory lines, one for Quintile 5 and one for the average of Quintiles 1-4, show that in Grade 3 there already exist large differences in performance (approximately three Grade-levels) and that by the time children enter Grade 9 this gap in performance has grown to about four Grade-levels. The linear trend in performance between these two groups suggests that if the same number of students in Quintiles 1-4 in Grade 9 continued in schooling until Grade 12 (i.e. no drop out between these two periods) they would be functioning at approximately 4.9 Grade levels lower than their Quintile 5 counterparts in Grade 12 (1.5 standard deviations lower).

Figure 6

Screen Shot 2014-12-11 at 10.44.44 AM

Returning to Lewin’s (2007) notion of an “on-track” progress line, perhaps the most important conclusion arising from this conceptual framework is that any performance below the “on-track” line creates an increasing gradient of expectation as the pupil moves into higher grades. This expectation is what is required by the curriculum to reach the goal (passing the grade 12 exam, for example) relative to where the student is at the present. As students’ learning deficits grow, the gradient of what needs to be achieved to reach the goal then progressively steepens to the point where it enters what Lewin (2007, p. 7) refers to as a ‘Zone of Improbable Progress.’ For example, the improvement that is required to bring the average Grade 9 Quintile 1 student in South Africa up to the required benchmark by Grade 12 is unrealistic given that they are performing at roughly a Grade 5 level in Grade 9. By contrast, the gradient of achievement required to bring the average Quintile 1 Grade 3 pupil up to the required benchmark by matric is slightly more manageable. The clear conclusion arising from this analysis is that intervening early to correct and prevent learning deficits is the only sustainable approach to raising average achievement in under-performing schools.

What we would add to this conclusion is that the root cause of these weak educational outcomes is that children are acquiring debilitating learning deficits early on in their schooling careers and that these remain with them as they progress through school. Because they do not master elementary numeracy and literacy skills in the foundation and intermediate phases, they are precluded from further learning and engaging fully with the Grade-appropriate curriculum, in spite of being enrolled in school. Lewin (2007, p. 10) refers to these children as ‘silently excluded’ since they are enrolled and attending school but learning little. Importantly, these children are precluded from further learning, not because of any inherent deficiency in their abilities or aptitudes, but rather because of the systematic and widespread failure of the South African education system to offer these students sustained and meaningful learning opportunities. Indeed, many children from poorer backgrounds have both the ability and the desire to succeed, and when provided with meaningful learning and remediation opportunities, do in fact succeed (see Spaull et al, 2012 for an example).

The clear policy recommendation which proceeds from these findings confirms what is becoming increasingly accepted, that any intervention to improve learning in South Africa needs to intervene as early as possible. Given South Africa’s egregiously high levels of inequality, it should come as no surprise that poor children in South Africa find themselves at a nexus of disadvantage, experiencing a lack of social, emotional and cognitive stimulation in early childhood. These children then enter a primary school system that is unable to equip them with the skills needed to succeed in life, let alone to remediate the large learning deficits they have already accumulated to date.

When faced with limited resources and a choice of where to intervene in the schooling system, the counsel from both the local and international literatures is unequivocal; the earlier the better. The need to focus on the primary Grades, and especially the pre-primary years, is not only driven by the fact that underperformance is so widespread in these phases, but also because remediation is most possible and most cost-effective when children are still young (Heckman, 2000). Due to the cumulative negative effects of learning deficits – particularly for vertically-integrated subjects like mathematics – it is not usually possible to fully remediate pupils if the intervention is too late (i.e. in high school), as too many South African interventions are. Nobel Laureate Professor James Heckman summarises the above succinctly when he explains that:

“Policies that seek to remedy deficits incurred in early years are much more costly than early investments wisely made, and do not restore lost capacities even when large costs are incurred. The later in life we attempt to repair early deficits, the costlier the remediation becomes” (Heckman, 2000, p. 5).

Full paper available here.