EGRS is recruiting as well :)

57c129023dd337b563123a2040341bb6.jpg

The Director of Research in the DBE – Dr Stephen Taylor – is looking for a Project Associate and a Project Intern for their Early Grade Reading Study (deadline for applications: 2 June 2017). The work they are doing on early grade reading is some of the most interesting and important work on the topic not only in South Africa but across the continent. Figuring out how you improve early grade reading outcomes at scale and using rigorous scientific evidence and evaluation. Having seen the early results of EGRS 1 I can say that there is a lot of promise in the coaching model for improving teacher’s content knowledge about reading and their pedagogical practice.

While the salaries may not be amazing 🙂 the team you will be working with really is and you may think of this as a launch pad into a rapidly growing and exciting field. Stephen Taylor, Mpumi Mohohlwane, Carol Nuga Deliwe, Janeli Kotze and Brahm Fleisch are all involved in the team. If you know of anyone please forward the TOR’s in the links above and below to them – deadline 2 June 2017.

EGRS 1

 

egrs2

I’m also working in this early grade reading space but on developing a (video and coaching based) certificate to teach early grade reading for the Allan Gray Orbis Foundation Endowment. I think we will also be recruiting soon. Stay tuned 🙂

We’re hiring: Project Administrator

gantt-chart-pictofigo-hi-006Ahoy blog-followers and other readers on the intertrons! It’s been a while since I’ve updated my blog on what I’ve been doing since I got from the OECD in Paris. Things were a little up and down about where I wanted to live, where I was going to work and the longer-term plans for life. So some of that has been resolved and things are moving ahead swiftly. I’ve been appointed as a Senior Researcher at RESEP in the Economics Department at Stellenbosch University. At the same time I’ve been seconded to the Allan Gray Orbis Foundation Endowment for half of my time to develop a video and coaching-based course to teach Foundation Phase isiXhosa teachers how to teach reading.

This is a really exciting 2-year project that I will be heading up and involving a wide range of Foundation Phase and isiXhosa experts and film professionals. A slightly-outdated (soon to be updated) overview of what we want to achieve in the course can be found HERE. If you know any isiXhosa early-grade-reading experts please also email me with more information.

For this project we are looking for a super-organised Project Administrator to work with myself and the Project Manager as we develop the course over the next 2 years. The link to the job description and application portal is here: http://agof.erecruit.co.za/candidateapp/Jobs/Categories/Project_Office/ea53aba45375451695c82c473eeec9f8

Please forward this to any isiXhosa Home-Language potential applicants you may know. I am open to adjusting the scope of the position based on the skills and expertise of the successful applicant, but it would definitely involve running the administrative side of the project (book-keeping, logistics admin, liaising with service providers etc).

Nic

Who makes it into PISA in Turkey?

Below is a quick summary of what I was working on at the OECD in Paris last year. The full paper is now available online here:

2Of the OECD countries that participate in the Programme for International Student Assessment (PISA), Turkey has one of the lowest levels of performance and the highest rates of improvement in PISA scores. However, most analyses have traditionally ignored one vital question: what percentage of 15 year olds in Turkey are eligible for the PISA sample in each wave of PISA? A new OECD Working Paper focuses on this specific question and sheds new light on the performance of Turkey between 2003 and 2012. It shows that the percentage of students that were eligible for PISA in Turkey between 2003 and 2012 nearly doubled from 36% to 68% (using OECD indicators) or from 45% to 80% (using household survey data). This is summarised in Figure 1 below which provides information from the Turkish Demographic and Health Survey (DHS) of 2003, 2009 and 2012.

Figure 1: The educational status and PISA-eligibility of 15-16 year olds in Turkey in DHS 2003, DHS 2008 and DHS 2013

1

While PISA aims to assess a nationally representative sample of 15 year olds, because PISA is a school-based survey, in reality it excludes all students that are no longer in school (due to drop out or non-enrolment). PISA also only samples 15 year olds if they are in Grade 7 or higher. So PISA is really a nationally representative sample of 15 year old students that are still enrolled in school and are currently in Grade 7 or higher. This might sound like a trivial technicality, and in most wealthy OECD countries like Germany or Japan it is. But some OECD countries (like Turkey and Mexico) and many partner countries (like Vietnam and Indonesia) have high levels of student dropout and delay leading to low levels of PISA sample coverage. As an aside, the new PISA-for-Development initiative aims to also survey out-of-school 15 year olds in some of the developing countries participating in that programme.

Since the beginning of PISA, the OECD has reported the percentage of 15 year olds that are actually eligible for PISA, what is called “Coverage Index 3*.” This statistic is calculated using census and enrolment data in each country and is provided in the overall PISA Reports and Technical Reports for all participating countries. For example, in Turkey in PISA 2003, only 36% of 15 year olds were eligible for PISA. That is to say that PISA 2003 in Turkey is only representative of 36% of the country’s 15 year olds. By comparison, the figure in Germany in 2003 was 93%. Table 1 below provides the Coverage Index 3 rates for a selected group of PISA countries with low levels of sample coverage (Germany and Canada are included as reference countries). From this we can see that a number of partner countries have very low levels of sample coverage, including Costa Rica, Indonesia, Peru and Vietnam, but also that some OECD countries (such as Brazil, Mexico and Turkey) have low levels of sample coverage.

Table 1: The percentage of the total 15 year old population covered by the PISA sampling frame (Coverage Index 3) in selected countries

Screen Shot 2017-04-18 at 7.57.17 AM

Given these low levels of sample coverage in some countries, it is reasonable to ask: how would the results change if we included PISA-ineligible 15-year-olds in our calculations? This is the focus of a new working paper published this week, titled “Underestimating Progress and Inequality in Turkey (2003-2012): Using PISA and DHS to form a complete picture of access and quality”. The method and results are briefly summarised below.

The paper applies a new method developed by Spaull & Taylor (2015) which aims to combine statistics on the learning outcomes of 15 year olds that are still in school (using PISA) with data on the number and type of 15 year olds that are not in school (using household-survey data). By assuming that PISA-ineligible students would not have reached PISA Level 2 in Reading and Mathematics – a relatively conservative assumption – we can calculate the percentage of the total population of 15 year olds that reach Level 2 in PISA, rather than only the percentage of those that are still in school. Figure 2 and Figure 3 below provide these breakdowns for Turkey in PISA 2003 and PISA 2012 respectively, and also by gender and socioeconomic subgroups. (Note: ‘Poor40’ means the poorest 40% of 15-16 year olds, and ‘Poor40F’ means poorest 40% of 15-16 year olds that are also female).

Figure 2 Access to Literacy (Level 2) in Turkey 2003 (PISA 2003 and DHS 2003)

Screen Shot 2017-04-18 at 7.57.47 AM

Figure 3 Access to Literacy (Level 2) in Turkey 2012 (PISA 2012 and DHS 2013)

Screen Shot 2017-04-18 at 7.57.59 AM

Note the results above will be different from those found in PISA reports because these results include PISA ineligible 15-16 year olds in the calculations.

The 7 main findings from the above are as follows:

  1. There have been significant increases in PISA sample eligibility over time from around 45% of 15-16 year olds in 2003 to 80% in 2013.
  2. If we account for the growth in PISA eligible 15-16 year olds, the improvement in the percentage of 15-16 year olds acquiring Level 2 is between 2 times larger (for Mathematics) and 5 times larger (in Reading) than previously thought.
  3. Severe delays in grade progression in Turkey have been virtually eliminated and dropout has declined by 42% over this period.
  4. In 2003 the richest group were twice as likely to be eligible for the PISA sample than the poorest group.
  5. Although both boys and girls benefitted over the period, girls benefitted the most.
  6. The gap in access-to-literacy and access-to-numeracy between rich and poor has not changed and is larger than previously thought.
  7. 15-16 year olds in the East region of Turkey are less likely to be eligible for the PISA sample than 15-16 year olds in other regions.

Whether one chooses to use the Coverage Index 3 figures reported by the OECD itself, or those calculated from various DHS surveys, the conclusion is the same: there are large and changing proportions of Turkish students that do not make it into the PISA sampling frame and this has a substantial effect on the validity of inter-country and inter-temporal comparisons. This analysis shows that the gains in Turkey between 2003 and 2012 have actually been even more impressive than if one only looked at PISA data. This method could also usefully be applied to other middle-income and developing countries with high and changing percentages of PISA-eligible 15 and 16 year olds.

References

OECD. (2016). PISA for Development: Benefits for participating countries. PISA for Development Brief (Vol. 2). Paris.

Spaull, N., and Taylor, S., (2015). Access to what? Creating a composite measure of educational quantity and educational quality for 11 African countries. Comparative Education Review. Vol. 58, No. 1.

*There is an error in the Coverage Index 3 values provided in the PISA 2003 Report. The correct Coverage Index 3 values for 2003 can be found in the PISA Technical Report for 2003. See the Working Paper for a full discussion.

The Godfather Speaks

godfYesterday RESEP had our first internal education workshop of the year and it was a great success. One of the presentations was by Martin Gustafsson on various trends and shocks in the  education system. I found it to be especially important and interesting and asked if I could include the presentation in blog format, which Martin kindly agreed to (see below). For those that don’t know Martin he is easily one of the top 3 most knowledgeable people about the SA education system.

The 2 main points that Martin makes are:

  1. There is irrefutable evidence that there has been real progress in learning outcomes in SA over the last decade,
  2. That there is a very strange demographic trend in SA (confirmed by 2 independent data sources) that there was a large increase in births in 2004 and 2005 (in the order of 10%) which is extremely peculiar. Note this has significant implications for resource allocation, class sizes etc. This was discussed in the DBE 2016 Sector Review (see excerpt below), but hasn’t been given nearly enough attention.

123

Regarding #2, from discussions with Martin, who has been involved in reviewing the 2013 SACMEQ results, it seems clear that the 2007 to 2013 SACMEQ trend has indeed been positive for South Africa, and about as large as one could expect (and roughly in line with what has been seen in TIMSS). But these positive trends are smaller than the extremely large improvements, which I’ve argued are implausible, seen in earlier preliminary SACMEQ results presented to the Parliamentary Portfolio Committee on Basic Education in 2016. What the sector urgently needs is the finalising of the 2013 SACMEQ process and the release of official learner results for all SACMEQ countries.

The trends seen in #3 below are from this 2016 paper of Martin’s.

456

“At the lower primary level, what has clearly occurred is the entrance of a ‘wave’ of larger birth cohorts. In 2011, Grade 1 enrolments increased by 5.2% relative to 2010. In 2012, Grade 2 enrolments increased by 7.0%. By 2015, the increase had reached Grade 5, with enrolments in this grade increasing by 5.2% relative to 2014. As seen in Table 2 in the appendix, substantial increases were seen across all provinces, with the exception of Eastern Cape, which in fact saw a decline . The increases were strongest in Gauteng and Western Cape, almost certainly because these two provinces experienced a combination of two factors, both the demographic change in terms of larger birth cohorts, plus migration into these provinces. By 2015, grades 1 to 7 enrolments had reached 7,1 million in public ordinary schools, the highest figure since 2007 (when the enrolment total was also around 7.1 million). Moreover, total enrolment in just grades 1 to 5 was higher than it had been in any year since 2002. Projections indicate that the increases will continue to be felt in grades beyond Grade 5 in 2016 and beyond. ” Page 9 of this 2016 DBE Sector Report.

7

*Note in the graph s above (top left) the steady rise in births from 1982 to 1998 are simply because of better data-capturing rather than increased births. It stabilises from about 2000.

8910

For those that haven’t read the DBE 2016 Sector Review I would strongly recommend doing so!

Links on 21st C teaching…

21

I’ve been giving a few presentations on 21st Century Skills and thought it’d be helpful to provide some of the links I like on this topic. If you have any you’d like me to add please do include them in the comments.

Helpful links and reports:

wef

maker

Useful websites:

casel

Some videos I like:

Esther Wojcicki – Empowering Students

Screen Shot 2017-03-20 at 9.26.01 AM

Linda Darling-Hammond – New Learning for a Rapidly Changing World

Screen Shot 2017-03-20 at 9.26.47 AM

And what does the future look like? I like a few of the new WEF videos on The Fourth Industrial Revolution 

Screen Shot 2017-03-20 at 9.57.30 AM

and The New Vision for Education and

Screen Shot 2017-03-20 at 9.59.42 AM

“Matric really does start in Gr1” – my M&G/Teacher article on TIMSS 2015

screen-shot-2017-01-31-at-7-05-23-am

(The article below first appeared in The Teacher magazine in January, and in the Mail & Guardian on the 27th of January 2017)

The month of January is an interesting one in the South African calendar. It seems that this is the one time in the year when everyone cares about education, or more accurately, cares about matric. And in some senses there is a good reason for this; matric is the gateway to higher education. If we look at 25-35 year olds in the 2011 Census data we see that for those with less than a matric, 47% were unemployed (using the broad definition), compared to 33% among those whose highest qualification was a matric and only 8% for those with a bachelors degree. While matric used to be the big distinguisher between the haves and have-nots, increasingly the difference is between those with some kind of tertiary qualification and everyone else. And in 2016, as usual, mathematics proved to be a tough gateway subject for those who want to study further. Only one in three learners who wrote Mathematics in 2016 got 40% or more in the subject, with a recent study showing that only about 15,000 matrics achieved 70% or more in mathematics. That’s about 1,5% of students who start school in Grade 1.

As anyone familiar with maths will tell you, the subject is a hierarchical one that builds upon itself. You need to understand multiplication and division before you can understand fractions or rate and proportion. So where do the wheels come off? Is it just before matric, perhaps in Grade 9 or 10? Or even earlier in Grade 4 or 5? To shed some light on this question we can turn to some recent mathematics research published by the Human Sciences Research Council (HSRC) at the end of last year. Every four years South Africa participates in an international study of mathematics achievement called the Trends in International Mathematics and Science Study or TIMSS. We test a nationally-representative sample of Grade 5 and Grade 9 learners on a standardised international test, which is the same test that is written by thousands of other students in other countries and allows us to compare ourselves to these other countries. (Although almost all the countries test their Grade 4 and Grade 8 students on these tests). The most recent round of tests was done in 2015.

The results showed that only 34% of our Grade 9 learners could do basic mathematics, i.e. could reach the lowest international benchmark. That is to say that 66% of our learners could not do basic computations or match tables to bar graphs or read a simple line graph. They had not acquired a basic understanding about whole numbers, decimals, operations or basic graphs. However there was one piece of encouraging information that emerged from the study and that was that the number of students that could do basic mathematics increased from 24% in 2011 to 34% in 2015 in South Africa, one of the fastest improvements seen internationally.

But if 66% of our Grade 9 learners can’t do basic maths, when did these gaps emerge? In 2015, for the first time in our history, South Africa also participated in the primary-school level of TIMSS that usually tests Grade 4 learners – we tested our Grade 5 learners. It showed that 61% of Grade 5 learners could not reach the Low International Benchmark. Learners who do not reach the low international benchmark at the primary level cannot add and subtract three or four digit whole numbers like 218 and 191. They do not recognise familiar geometric shapes or parallel and perpendicular lines. They cannot read and complete simple bar graphs and tables. For example, according to our curriculum, multiplying a 3-digit number by a 1-digit number is meant to be covered in Term 1 of Grade 4 yet only 41% of our Grade 5 learners could calculate “512 x 3 =____”, which was one of the questions in 2015. So almost 60% of our Grade 5 learners are already significantly behind the curriculum in 2015.

The conclusion that the root of our problems is not in high school but rather much earlier, in the lower grades of primary school is not a new finding – there are many studies showing this from at least 1999. In a report published nearly 10 years ago, one South African education researcher (Dr Eric Scholar) analysed the mathematics achievement in 154 schools across the country. He concluded that the low levels of achievement we see in the higher grades are rooted in weak foundations in primary school. To quote his exact conclusion:

‘‘The fundamental cause of poor learner performance across our education system is a failure to extend the ability of learners from counting to true calculating in their primary schooling. All more complex mathematics depends, in the first instance, on an instinctive understanding of place value within the base-10 number system, combined with an ability to readily perform basic calculations and see numeric relationships … Learners are routinely promoted from one Grade to the next without having mastered the content and foundational competences of preceding Grades, resulting in a large cognitive backlog that progressively inhibits the acquisition of more complex competencies. The consequence is that every class has become, in effect, a ‘multi-Grade’ class in which there is a very large range of learner abilities and this makes it very difficult, or even impossible, to consistently teach to the required assessment standards for any particular Grade. Mathematics, however, is a hierarchical subject in which the development of increasingly complex cognitive abilities at each succeeding level is dependent on the progressive and cumulative mastery of its conceptual frameworks, starting with the absolutely fundamental basics of place value (the base-10 number system) and the four operations (calculation)’’

While as a country we continue to obsess about the matric pass rate, the research is really quite clear. The majority of our young people are acaquiring learning deficits early on in primary school and then carrying these with them as they move through school. As they are promoted into higher grades there is a decoupling between what learners know and can do and what the curriculum expects from them. We need to acknowledge that matric starts in Grade 1 (and even earlier), and that it really is possible to improve primary schooling if that is where we focus most of our time, energy and resources.

DBE needs you (2 vacancies)

we-need-you

My good friend and colleague Dr Stephen Taylor (who is the Director of Research in the National Department of Basic Education (DBE) in Pretoria) is looking for 2 people to join his team in the Research Monitoring and Evaluation unit. The full job advertisement is here and I’ve included screenshots of the relevant position below. I’ve done a Q&A with Stephen (and Martin Gustafsson who is also in DBE) to give you a flavour of the kind of people they are. I would whole-heartedly recommend Stephen as a boss and a person 🙂 they are creating a really impactful team (Janeli Kotze and Mpumi Mohohlwane have also recently been appointed there full-time). If you are looking for an excellent team, to have an impact in education and have good quantitative skills – apply! (Deadline 17 Feb).

screen-shot-2017-01-30-at-8-01-34-am

screen-shot-2017-01-30-at-8-01-43-am

PS I should totally get a finders fee for this kind of thing 🙂