Monthly Archives: December 2013

Links I liked…

economist SA chart 20130713_gdc865

The last chapter in any successful genocide is the one in which the oppressor can remove their hands and say, ‘My God, what are these people doing to themselves? They’re killing each other. They’re killing themselves while we watch them die.’ This is how we came to own these United States. This is the legacy of manifest destiny.” 

ANA 2013 results don’t make sense

See below for the article I wrote for the M&G which appeared in the print edition  (13 Dec 2013) and is now available online here.

M&G 05 DEC 2013 19:20 NIC SPAULL

  • Annual literacy and numeracy tests for grades one, six and nine aren’t comparable over time or across grades, so talk of improvement is misleading.
Annual National Assessment results not comparable over time or across grades makes all talk about improvement or trends misleading. (Madelene Cronje, M&G)

I have a love-hate relationship with the yearly publication of the national assessment results.On the one hand I am very proud of the annual national assessments and glad that we have them. Testing primary school children using standardised assessments is imperative to target support where it is needed and also to hold the basic education department and schools accountable. We definitely shouldn’t scrap them, since without them we would be stabbing in the dark.

On the other hand, I get depressed when the results are released because, given the way they are currently implemented, we actually are stabbing in the dark. For the national assessments to fulfil the function for which they were created, the results need to be comparable across grades, over time and between geographical locations. Unfortunately, given the sorry state of affairs that is the 2013 national assessment, none of these criteria are currently met.

The highlights version of last week’s release goes something like this: “Performance in grades one to three is adequate. Results for most grades show a steady increase. Grade nine performance is an unmitigated disaster.”

The unfortunate part is that the only statement I actually believe is the last one.

The reason is that Basic Education Minister Angie Motshekga, presumably shocked to the core by the grade nine mathematics average of 14%, appointed a ministerial task team to assess whether the 2013 grade nine test was fair, valid and reliable. They concluded it was.

This result is in stark contrast with the much higher mathematics averages in grades one (60%), two (59%) and three (53%). If the minister had asked the task team to look into those tests as well, my suspicion is that it would have reached the opposite conclusion.

The reason I think this is that all the existing research in South Africa points to the fact that children are not acquiring the foundational numeracy and literacy skills in primary school and that this is the cause of underperformance in higher grades. The 2011 pre-Progress in International Reading Literacy Study, for example, found that 29% of South African grade four students could not “locate and retrieve an explicitly stated detail” — that is, they were completely illiterate.

Yet in the assessment results, the home language average was 49% and the first additional language was 39% — higher than one would expect given previous grade-appropriate tests.

Likewise, the average score on the grade three national assessment numeracy test was 53%, up from 41% in 2012. Putting aside for a moment the fact that such gargantuan improvements have never been seen around the world — in any country, ever — the level is also way out.

The National School Effectiveness Study, published in 2011, found that grade three students scored an average of 29% on a grade three-level test. If South Africa improved at the fastest rate ever seen globally (which is 0.08 standard deviations a year), the score in 2013 would be about 38% in grade three — not the 53% Motshekga reported. These results don’t make sense.

Another peculiarity is the huge drop in mathematics performance between the relatively high grade six average of 39% and the depressingly low grade nine average of 14% only three grades later. From an educational assessment perspective, this is surely because the grade six test was much easier than the grade nine one, which, the task team established, was set at the appropriate level.

There are other possible reasons why the results in grades one, two and three are so high. One is that for grades one and two teachers were allowed to invigilate their own students during the test — which could obviously be problematic.

Another is that the grade three marks in unmonitored schools were significantly higher than those in the verification sample. In order to assess the fidelity of the administration, scoring and collating processes of the annual national assessments, 2 164 of the 24 355 schools were monitored by an external body. The exams from these schools were then marked and captured by an independent body.

The national unverified grade three literacy average was 51%, which was considerably higher than that found by the verification body, namely 42%.

But the report completely ignores these differences and rather focuses on the higher unverified scores, claiming that the verified scores are “not significantly different from the mean scores of pupils from the whole population” — something that is clearly untrue, as is evident from the report itself.

In some instances, the discrepancies are considerable. In the grade three literacy test, for example, the unverified provincial average for the Eastern Cape was 47% and for the Western Cape it was 50%.

This is in stark contrast with the true average found in the verification sample, which was 35% in the Eastern Cape and 49% in the Western Cape. And yet for some bizarre reason, the department decided to stick with the unverified marks for all grades and both subjects. There were also considerable discrepancies in the grade six and grade nine home language results.

Lastly, one simply cannot compare national assessment results over time. To do this, the tests would have to be calibrated and linked using psychometric analysis — something the department did not do. Last week’s national assessment report is somewhat bipolar on this point.

The report cautions: “The comparability of the tests from one year to the other cannot be guaranteed, which implies that comparability of the results from one year to the other may not be accurate.”

But these cautions did not seem to deter the minister, who said when releasing the results: “I am confident that performance in the education system is on an upward trend and all our interventions and programmes are beginning to produce the desired outcomes” — a confidence I, unfortunately, do not share.

Elsewhere the report says: “There is currently a strong emphasis on ensuring that tests from different years are comparable to each other, so that trends over years can be reliably monitored. In this regard a process is already under way.”

Is this meant to be reassuring? The fact that the process of ensuring psychometric comparability over time is “under way” is a half-baked admission that it was not ready or used for 2013, making any annual comparisons impossible.

Unfortunately, there is no technical report to confirm or deny this. Even without a technical report, the erratic and colossal changes occurring year on year are simply impossible.

In 2012, only 24% of grade six students had acceptable achievement (more than 50%) in the first additional language test — but this shot up to 41% in 2013 (a 71% increase) only one year later. By contrast, for grade nine students — only three grades later — the proportion achieving at an acceptable level came down from 21% in 2012 to 17% in 2013. Anyone familiar with educational assessments would balk at such large and inconsistent changes.

Many questions still remain unanswered. Who was the technical committee that advised the department on the national assessments of 2013? Why were their names not included in the report? What “process” of psychometric comparability is “under way” and where is the department in that process? Who was the “service provider” that verified the assessment results and why was it not listed, as the Human Sciences Research Council was in 2011?

One could also speak about the dangers of giving erroneous feedback to teachers or allocating resources based on faulty data — both of which are the spectres we will have to live with in the future. In testing seven million children the department has bitten off more than it can chew and, in the process, undermined its own technical credibility.

If we could trade ambition for competence, we may have a test that was actually telling us something clear instead of the muddled mess that is the national assessment for 2013. This testing must and should go on, but for heaven’s sake do it properly.

Gender equality – facts and figures


“The blunt truth is that men still run the world. This means that when it comes to making the decisions that most affect us all, women’s voices are not heard equally. Of the 195 independent countries in the world, only 17 are led by women. Women hold just 20 percent of seats in parliaments globally. In the November 2012 election in the United States, women won more congressional seats than ever before, bringing them up to 18 percent. In the United Kingdom, 22 percent of seats in Parliament are held by women. In the European Parliament, one-third of the seats are held by women. None of these figures are close to 50 percent.

The percentage of women in leadership roles is even lower in the corporate world. A meagre 4 percent of the Fortune 500 CEOs are women. In the United States, women hold about 14 percent of executive officer positions and 17 percent of board seats, numbers that have barely budged over the past decade. The gap is even worse for women of color, who hold just 4 percent of top corporate jobs, 3 percent of board seats, and 5 percent of congressional seats. Throughout Europe, women hold 14 percent of board seats. In the United Kingdom, women hold about 7 percent of executive directorships and 15 percent of board seats among the FTSE 100 companies. These numbers drop to 5 percent of executive directorships and 9 percent of board seats among the FTSE 250.”

This is an excerpt from Sheryl Sandberg’s book  Lean In. I actually bought the book as a Christmas present for a friend – to try and counteract the sexist, religious sub-culture she’s been socialized into – but started reading it pre-wrapping and will definitely buy a copy for myself. Another gem: “Warren Buffett has stated generously that one of the reasons for his great success was that he was competing with only half of the population.”

The stats presented above show that in 2013 we are nowhere near gender-quality, even in the world’s most liberal and progressive democracies. A little closer to home, in the Stellenbosch University Economics Department we have three times as many male professors (13) as we do female professors (4). Of course we can find reasons why this is the case but explanations are not justifications and this is an indictment as far as I am concerned. It’s also in stark contrast to the South African Parliament where almost half of MP’s (45%) are female. We clearly need more and stronger regulations which legislate and incentivise gender equality targets in government, in academia, in business, and in society. Until there is equal representation in the halls of power we will all lose out – men and women, but especially children.

The eloquence of the fake signing man – Sarah Britten


I don’t repost other people’s articles often but this is too good not to – see Sarah Britten’s article on what the fake signing man says about SA politics below. Pritchett talks about this in his new book referring to it as isomorphic mimicry:

In our conversation, Lant unpacks the problems inherent in what he calls “isomorphic mimicry”: building institutions and processes in weak states that look like those found in functional states. “They pretend to do the reforms that look like the kind of reforms that successful [countries] do, but without their core underlying functionalities,” says Lant. “Instead, countries wind up with all the trappings of a capable system—institutions, agencies, and ministries—without its functionalities.” from here


The Eloquence of the Fake Signing Man – Sarah Britten

I won’t lie. A lot of this is bloody funny (read some of the best jokes about it here). The fake sign language interpreter is now a cultural phenomenon, featuring on major US comedy shows and catalysing a new meme.

And yet, at the heart of this, is a terrible sadness. I felt tremendous pity for Thamsanqa Jantjieas I watched him interviewed by Karyn Maughan on eNews. Here was a man cornered, desperate: a man who could see his world falling apart in front of his eyes.

A modern Walter Mitty, he was holding on for all he was worth to his sense of self. I am a man, he said. I am a provider. His wife brought out a blue dustbin bag filled with medication. She looked resigned.

Chutzpah, I had first thought. It turns out that all you need to get past the CIA and an entire phalanx of men straight out of The Matrix is magnificent incompetence. To fake it till you make it next to the leader of the hypocritically free world takes cojones. “I am proud to be South African” said Anton Taylor of Jozi shore.

But the story is so much more complicated. A mentally unstable man with a history of violent outbursts stood a metre away from the most powerful leaders in the world and passed himself off as an interpreter. This was Mr Bean goes to the UN — only it was very serious.

This is what he said. As it turned out, his gibberish spoke volumes.

In South Africa, the signing man told the world, you don’t actually have to know what you are doing in order to get a job. You don’t have to have any ability whatsoever, as long as it looks, to most, as though you can go through the motions — whether you are a teacher, a police officer, a bureaucrat, a government official or (as some have suggested) a state president.

There are those who see through you and complain, but they are ignored. Ours is not a culture of accountability. So one gig leads to the next. You’ve done it before so you get to do it again, because everyone in a position of power agrees that the emperor’s new threads are stylish. You stand there and tell us that the appearance of something becomes more important than the substance of it. Your obvious inability to do your job does not prevent you from getting ahead, until you reach the most prominent stage in the world, and then pretending suddenly isn’t enough. Too many people noticed — too many people who couldn’t just be dismissed because of their politics or race, which is how criticism is normally dealt with.

Thamsanqa discovered that eventually, somebody will see what you are doing, and call you out on it, and there will be nowhere to run. And you will be blamed, and the decision makers who allowed a smaller lie to metastasize into this awful mess will escape censure. Because in South Africa, nobody is ever held responsible — unless you’re low enough down the food chain and lack political connections. Then it’s all your fault.

In his desperate attempts to maintain a facade of functionality in front of the world, as he heard voices and saw angels, Thamsanqa Jantjie said more about the state of South Africa’s current rulers than all the analysts and spin doctors ever could.

He might not have been able to express a coherent word, but the fake signing man turned out to be remarkably eloquent.

Thank you Madiba for changing all the rules

thank you madiba

On the fifth of December 2013 Nelson Mandela died. He was the father of our country and the greatest man it has ever produced. While we all knew it was coming soon, the finality of death and the fact that his presence is no longer with us is truly saddening and has marked the beginning of a time of national mourning, remembrance and reflection. The picture above is a photo I took yesterday at Kirstenbosch Gardens at the Nelson Mandela sculpture which has become one of thousands of places around the country where people have laid cards and flowers in memory of our first democratic president.  This colorful card by Caleb  was such a gem: “Dear madiba I will miss you so much. thank you for changing all the rules, Love caleb.” It’s difficult to provide a better summary than this parsimonious account. Madiba really did change all the rules – the rules of apartheid, the rules of society and the rules of our country, yes, but also the rules we had allowed to govern ourselves – hatred, unforgiveness, discrimination and oppression. Madiba we will always love you. x

I still want to write a post about Madiba and what he means to me and what he represents to South Africa and the world but that will come later. Till then I really liked Maya Angelou’s tribute poem “His Day is Done” and Desmond Tutu’s tribute to Madiba. [Also this awesome flash mob tribute by the Soweto Gospel Choir in Woolies Parkview]


Links I liked…

  • Lant Pritchett on Education in Poor Countries” – an EconTalk podcast (64 min) I am definitely going to listen to. Pritchett is always good and Russ Roberts is an excellent interviewer.
  • Justin Sandefour summarises the findings from an important low-cost-private-school voucher intervention in Andra Pradesh (by Karthik Muralidharan and Venkatesh Sundararaman (o_O) whose full article is available here.
  • Great 2007 article by Deon Filmer: “Education inequalities around the world” – one of the brightest minds in the field. Really important reading.
  • M&G article summarising some of the (encouraging) findings arising from the National Income Dynamics Study (NIDS)
  • My M&G article: “ANA results are not comparable” explaining why the ANA 2013 results are not comparable to previous ANAs or across grades. Banging the same old drum in the hopes that someone is listening!
  • New DBE reports now available online, including one on internal efficiencymacro indicators and the school monitoring survey
  • Interesting and disruptive innovation in Kenya – Bridge International Academy offers a school-in-a-box solution where teachers are trained for 7 weeks and offer scripted lessons in a highly structured and specified way. Most educationists will hate it but I just think to myself – what kind of education would these kids be receiving if they weren’t at these pop-up schools? Sometimes the counter-factual is far worse than even a below-average solution.
  • How to speak and write post-modern” – I must say I chortled more than a few times.
  • Interesting Economist article on Creationists and Science: “After they hit 18, half of evangelical youngsters lose their faith; entering a public university is especially perilous. As a generation, millennials (those born between the early 1980s and 2000s), are unimpressed by organised anything, let alone organised religion…Recent research (notably cross-species comparisons of gene sequences rendered non-functional by mutations) has greatly strengthened the case that humans and chimpanzees share a common ancestor. A creationist speaker in Baltimore shrugged such discoveries off, declaring that “science changes, but the word of God never changes.”
  • Very interesting – Kagame on RwandaOur thinking is based on people,” the tall, lean president tells This is Africa, from his hotel room on a recent trip to New York. “Investing in our people, what our people are capable of. In national budgets, we focus on education, health, we look at technology, skills, innovation, creativity. We are always thinking about people, people, people.” Rwanda looks to South Korea and Singapore, as well as developed countries, for lessons on “the way they managed their problems throughout history. And the central role played by people is very clear,” the president says. “We try to take all of that to our own needs and we have not been disappointed. You never go wrong by investing in people.
  • Pritchett quoting an Indian policy-makerYou guys from the World Bank say you want to help the government of India with our development agenda but then all you want to talk about is poverty, poverty, poverty. Let me point out two things. First, India is a democracy and hence to be the government at all we have to have 51 percent of the votes and we don’t have that many poor voters. Second, once we are the government of India we are the government of all Indians, not just the poor ones, so our agenda has to reflect the aspirations of all Indians. So either you are really helping us with our development agenda or you are just pushing yours.” Wow. Just wow.
  • Two South African education conferences in 2014 that are worth attending – SABEC (31 March JHB) – and SAERA (13 Aug DBN).