Explaining the matric results in 7 GIFs

*11pm 5/01/18: New addendum added. Note that the throughput pass rates or comparison rates discussed here are not the ‘real’ pass rate. It is not 37% but about 55% (the former is a crude approximation and underestimates the true pass rate by about 15-18 percentage points primarily because the denominator has repeaters) See Martin Gustafsson’s comment at the end of the post. 

Every year the matric results come out and the whole country goes ballistic. Front pages of the newspapers, every radio, every TV station. And suddenly everyone has an opinion about what’s going on. All the people. so. many. opinions. “The Free State is the best province”, “We should ban the IEB and have one exam”, “The rising pass rate shows our interventions are working”. No. No. No.

  I was determined that this year I was going to stay out of the #MatricResults2017 morass and instead focus on our program to teach Foundation Phase teachers how to teach reading, because 78% of kids in SA can’t read. But no one seems to be reading the 97-page 2017 NSC Examination Report which has loads of information. Spend an hour or two analysing the data and it’s pretty easy to see what the trends are and what’s going on. So here are some of my take home points:

(1) The matric pass rate is one (very incomplete) measure of the education system and it’s a bad idea to focus on it in isolation.

The matric pass rate is calculated as the numbers passing divided by the numbers writing. It can go up if either the numerator goes up or the denominator goes down (or both).  Nick Taylor argued this in 2011 and I’ve discussed this in 2015 and 2016. The gist of it is that it’s possible to artificially raise the matric pass rate by letting fewer weaker students write the exam (in SA this is referred to as culling or gate-keeping). So to get around this we calculate a ‘throughput pass rate’ where we divide the total number of passes by the Gr10 enrolment two years earlier or by the Grade 2 enrolment 10 years earlier. This is a VERY crude measure and does not take into account grade repetition in Gr2 and Gr10, migration, mortality etc (see Martin Gustafsson’s comment at the end of this post for a fuller discussion). So what do the trends look like in the “Traditional” matric pass rate and the “Throughput comparison rate? Basically, while the traditional pass rate has been rising the Gr10 throughput comparison rate has been declining for three years now and the Gr2 throughput comparison rate has been declining for two years. Note the ‘real’ matric pass rate is closer to 55% not 75%, but also not as low as 37%. Basically the ‘real’ pass rate is 15-18 percentage points higher than these crude estimates once you take into account repeaters and some other factors (again, see Martin’s comment at the end).

(2) The provinces with the largest increases in matric pass rate have the largest declines in numbers writing matric

Basically what happens is that provinces (via principals and teachers) let fewer students reach and write matric (usually the weaker ones) which artificially lowers their denominator and raises their traditional pass rate. So the three provinces that have higher pass rates in 2017 compared to 2016 (EC, LP and KZN) also had the largest declines in the number of Gr12s writing matric between 2016 and 2017.

(3) The Free State is NOT the best performing province in the country. Some people think that. They are wrong. 

As mentioned in point (1) above, the traditional pass rate can be very misleading. This is especially the case with the Free State. While it has the highest traditional pass rate (86%), it also has the largest decline in Gr10 throughput comparison rate between 2016 and 2017 (from 43% to 36%). What does that mean? Basically that the change between 2016 and 2017 for the Free State seems to be driven by the fact that they are letting fewer of their (weaker) Gr10’s reach matric in the first place.

If one looks at the Grade 2 throughput comparison rate then Limpopo and the Northern Cape seem to doing quite a bit worse than they were in 2016.

(4) So if we use the throughput pass rate who does better, the Western Cape or Gauteng?

For those who follow education statistics it’s pretty clear that the WC and GP are always the top two provinces in the country. Sometimes it’s WC (PIRLS Literacy 2016, TIMSS-N 2015), sometimes it’s GP (TIMSS-Gr9 2015). So if the Free State suddenly pops up as number 1 you’ve got to look into the data a little. And what do we find? Surprise, surprise, if you use either the Gr2 throughput comparison rate or the Gr10 throughput comparison rate then the FS is fifth. And who’s first? Gauteng and the Western Cape, kind of both. You see if you take the Gr2 throughput comparison rate then it’s GP but if you take the Gr10 throughput comparison rate then it’s WC. You usually see GP ahead if you use Gr8 throughput as well. The reason is usually because the dropout that does occur in the WC usually happens before Gr10 so their Gr10 numbers are artificially low.

But you might then ask: why not just use the Grade 2 throughput rate instead because it’s more reflective of the whole education system? Well, it’s mainly because there is likely some inter-provincial migration between Grade 2 and Grade 12 and this is most likely from poorer provinces towards WC and GP. So the WC and GP throughput comparison rates are always slightly inflated, and the Gr2 throughput rate is the most inflated.

(5) There are other things to focus on instead of the matric pass rate

I know that for those writing and passing matric it’s a momentous and important occasion. It’s the culmination of our schooling cycle and (unfortunately for some) a rite of passage into middle class life (note: necessary but not sufficient for said passage). But as a country I feel like we need to take active steps to de-emphasise the matric results and instead focus on the ailing primary school system that seems to drop off the radar after a week of new results being released. 78% of our Grade 4 kids cannot read for meaning in any language. 66% of our Grade 9’s can’t do basic maths or basic science. Perhaps because 79% of our Grade 6 maths teachers can’t pass Grade 6 maths tests? We need to get back to basics and ensure that every child learns to read for meaning and understand the concepts of number and numeracy. If the wheels come off by Grade 4 then that’s where we need to be intervening. Prevention is better than cure. A stitch in time saves nine. A matric obsessed country misprioritizes its resources.

Transmission ends.

Addendum: Important comment from the ever-wise Martin Gustafsson (I agree with all of his points here, including that the ‘real’ matric pass rate is best estimated as 55%.

“Nic, whilst I absolutely agree with the thrust of this, I don’t agree with the 37 to 39% figure. The problem with dividing Grade 12 passes with Grade 2 enrolment 10 years ago is that there are many repeaters in Grade 2. We have relatively good data for recent years pointing to around 10% of Grade 2 learners being repeaters. We don’t know with certainty what the situation was in 2007, but the figure then was probably higher. Counting NON-repeaters only in your denominator would give you a better idea of who should proceed to Grade 12. But there are other issues: mortality, migration (e.g. across provinces if you want to look at provincial ratios), enrolment in TVET colleges, supplementary examination results, part-time Matriculants, IEB. Once you take all of this into account you get around 55%, not the 37 to 39% you refer to. The DBE has published the calculations (done by me) behind the 55%. It’s at http://www.education.gov.za/Portals/0/Documents/Reports/Education%20Sector%20review%202015%20-%202016.pdf.

PS: for those who want to review the numbers and do their own calculations the Excel file and references for all the stats are available here.