Nic Spaull

 On the 12^th of September the Department of Basic Education (DBE) presented a briefing to the Basic Education Portfolio Committee on the SACMEQ IV (2013) results. This was the first time that these results were presented in a public forum, and the first time that the national average SACMEQ IV reading score (558) and the national average SACMEQ IV mathematics score (587) were presented. The presentation is publicly available here.

I have an intimate knowledge of the SACMEQ IV data given that I was appointed by the SACMEQ Secretariat to analyse the SACMEQ IV data for the international report (date of appointment: 19 April 2016, Purchase Order Number UB-P02016001112, University of Botswana). After analysing the data I raised a number of serious technical concerns arising from the data  that significantly affect the comparability and validity of the findings and advised the SACMEQ Secretariat to address these concerns before any publication of the results (letter dated 26 May 2016). I also emailed the SACMEQ Technical Advisory Committee (9 June 2016) outlining the technical issues. Only one member responded and indicated that the item-response-theory (IRT) analysis should be redone with two independent verification checks. Based on the subsequent response from the SACMEQ Secretariat indicating that this would not happen I explained that I could not in good conscience continue with the analysis and chose to resign on technical grounds (resignation letter dated 7 August 2016).

The principal grounds for my technical concerns and subsequent resignation was the non-comparability of the results between SACMEQ III and SACMEQ IV because of the different methodologies employed when calculating test scores between SACMEQ III and SACMEQ IV, and particularly the fact that weaker students had been excluded from the final results in the process. This does not seem to have been addressed since the results presented to Parliament were the same as those that I identified as problematic.

Unfortunately, the contract that I have signed with SACMEQ prevents me from publishing any results that are based on that data until the international report has been publicly released, at which time I will provide a full account of my technical concerns and reasons for the non-comparability. I have also subsequently deleted the data on SACMEQ’s request.

The Department of Basic Education is already aware of all of my concerns since I emailed a number of members of the Department’s research team drawing attention to these problems and cautioning against publishing any results until they could be rectified. It would seem that the Department has chosen to push ahead and report these problematic results to Parliament in spite of these numerous concerns.

Comments on the SACMEQ IV presentation to parliament:

• The gains in the SACMEQ reading and mathematics scores between 2007 and 2013 are so unbelievably large that they would make South Africa the fastest improving education system in the world, together with Country 2 and Country 7 (names of other countries were excluded in the presentation to parliament. These countries improved by more than 0.9 standard deviations or 0,13 standard deviations per year. The improvement from 495 to 587 is an 18,5% improvement in test scores (or 2,7% improvement per year). A 2012 study looking at how fast education systems can improve points to Brazil which is the fastest improving education system in any testing system. Yet the SACMEQ IV results presented to parliament would have us believe that South Africa (and Country 2 and Country 7) improved at least twice as fast as Brazil, the fastest improving country. This is extremely extremely unlikely. (It is also unlikely that the teacher test scores have dropped so drastically). We know from other data (such as TIMSS 2003 and 2011) that South Africa has improved in Grade 9 mathematics but this improvement was only half as large as that reported by SACMEQ IV. South Africa’s scores may well have improved between 2007 and 2013 but we cannot say if they have improved or decreased until the results are actually comparable.
• The fact that teacher test scores plummeted at the same time that student test scores soared should already make us very curious about the technical procedures that might lead to such a situation.

I think the best way forward is for the Department of Basic Education to issue a statement explaining whether they believe the SACMEQ III and IV results are comparable and why and whether these results were based on an earlier version of the data, one which has subsequently changed. And secondly for us to wait for the SACMEQ IV Technical report to be released by SACMEQ so that the procedures, methods and assumptions underlying the SACMEQ IV results can be scrutinised by researchers around the world. This is the reason that the TIMSS, PIRLS and PISA results are all only released at the same time as the full international report.

SACMEQ is an extremely important indicator of changes in reading and mathematics achievement over time. It’s reputation and technical credibility must be upheld for it to retain its position as the main cross-national testing system in the region. To do so, the methods and analysis must be above reproach and open to scrutiny by independent researchers.

So I have finally taken to crowd-sourcing in my research and I need your help!

Do you know of any ‘outlier’ or exceptional township or rural primary schools in either KZN, Limpopo or Gauteng? Schools that manage to succeed against the odds and achieve great learning outcomes.

In a new ESRC/DFID study we want to understand how these schools manage to get the results they do, and specifically to understand the school leadership and management characteristics in these schools. But first we need to identify these outlier schools. We’re trying a number of different approaches to identifying these schools and then triangulating the results.

So word or mouth or your first-hand experience with an exceptional township/rural school could really help us. If you have any suggestions please send me an email at NicholasSpaull[at]gmail.com with the name of the primary school and why you think it’s an outlier school and any contact info if you have it!

Looking forward to hearing from you!!

Nic

Fieldworkers required: Research on Socio-Economic Policy (ReSEP), based in the Economics Department at Stellenbosch University, has embarked on a research project focussing on exceptional township and rural primary schools in three provinces in South Africa. ReSEP plans to recruit 9 experienced fieldworkers to assist them with school-based research.

Job description: Fieldworkers

Periods of work: Early October 2016 with an option to renew for forthcoming fieldwork

Overall period of fieldwork: Early October 2016, February/March 2017, September/October 2017

Daily rate for fieldwork: R800 per day with R100 per day subsistence allowance

Daily rate for training: R450 per day

Description of the project: The main aim of the research is to better understand why some schools perform better than others specifically in township and rural areas. We will develop a new survey instrument that captures the actual practices and behaviours of teachers and principals in challenging contexts.

This will be done where each exceptional school is paired with a nearby ‘typical’ school, i.e. one with similar geographical and socioeconomic characteristics.

The study will be conducted in various phases from May 2016 to September 2018 in 60 schools across the Western Cape, KwaZulu-Natal and Limpopo provinces. This advert is based on fieldwork for the first phase, piloting the various research instruments in three schools per province, this may however be extended to the subsequent data collection phases (February and October 2017) based on fieldworker performance and availability.

Fieldworkers will be required to

• interview school principals and teachers using quantitative instruments;
• conduct reading and vocabulary tests with young children;
• conduct assessments of classrooms and learner workbooks;
• provide informative insights into the school environments they visit.

Minimum requirements: 1) A bachelor’s degree (although in exceptional cases 3 year diploma’s also considered) 2) Fluency in reading and writing in English as well as one of the three following languages: Sepedi, isiZulu or IsiXhosa

Preference will be given to individuals:

• with previous research experience particularly from academic disciplines such as linguistics, psychology, teaching and education research. Although other background disciplines will also be considered.
• with a valid driver’s license and regular driving experience.  with a research interest in the project.
• who live in the province in which they are surveying schools (i.e. Limpopo, Western Cape or KwaZulu-Natal).

Qualities:

Ideal candidates should possess the following qualities:

• Integrity and honesty
• Organised with the ability to plan effectively
• Confident and professional
• Good interpersonal skills
• Works easily in a team
• Enjoys working with children
• Takes initiative to solve problems as they arise
• The ability to objectively observe an environment without influencing how it functions.
• The ability to articulate clearly in language and writing your observations.

Required time commitments:

• 2016: Early October 2016 (6 days). 2 days compulsory training. 3-4 days pilot fieldwork.
• Working hours in field: 6.30am/7am start to be at a school by 8am depending on travel distances. 8 hour working day. Each fieldworker will be expected to verify the GPS location/address of the piloting schools prior to the field visit. Late arrivals will not be accepted as this will jeopardise the completion of data collection.
• Important note: Fieldwork will likely involve being away from home Monday to Friday (given that schools may be located very far from central hubs) and will potentially require driving to required areas on a Sunday to be able to start fieldwork promptly in schools on a Monday morning. All travel arrangement, logistics and accommodation will be arranged through ReSEP, however it is the responsibility of each fieldworker to verify the suitability of the arrangement.

Application:

12 shortlisted candidates will be required to attend training at a central location where 9 fieldworkers (3 per province) will be chosen from the most suitable candidates identified. All shortlisted candidates will be compensated the daily rate of R450 for the duration of the training.

If you are interested in applying for the position please send to mschreve@sun.ac.za by Mon. 8 August 2016

i) Your CV. Including two references.

ii) A short written piece (500 word limit) on what characteristics/features in your opinion distinguish a school as being better than others. The piece must be written in English and in one of the three following languages: Sepedi, isiZulu or IsiXhosa.

iii) A covering letter. In your covering letter please explain why you think you would be a good match for this position. In the subject line please include: “Application: ESRC fieldworker.”

The University reserves the right to investigate qualifications and conduct background checks on all candidates. Should no feedback be received from the University within four weeks of the closing date, kindly accept that your application did not succeed.

In her latest policy brief, my friend and colleague Dr Gabi Wills discusses the overprioritisation of matric/high-schools and the underprioritisation of Foundation Phase (Gr1-3) and primary schools. The full policy brief is here. She finds that high school principals are significantly more likely to have been visited by a dsitrict official than a primary school principal, leading to lower monitoring and support at this critical phase. The need to prioritise the Foundation Phase and reading specifically was the main conclusion from two major research projects that we have just finished (see here). Until we realise that we will win or lose the battle of improved educational outcomes for the poor in the primary school, and specifically in Grades 1-3, then we are doomed to go around this mountain one more time!

I include some excerpts from the policy brief below:

The Ministerial Task Team Report into the sale of posts – colloquially referred to as the Jobs-for-Cash report has finally be released by the DBE. Full report HERE. The report is far-reaching, frank and damning. We all owe a huge thanks to Professor Volmink and his team for their investigation and this much needed report. When corruption is so systemic, as it is in SA education, to write a report like this without fear or favour is the mark of a deep conviction that something has to change.

I have taken out a few excerpts for those who don’t have time to read the whole (285-page report:

• “All the Deputy Directors-General in the Department of Basic Education are SADTU members and attend meetings of that Union. (p93)
• “The Gauteng MEC said that “the major Union is in charge of education” in his Province” (p.81)
• HOD North West – “He said that his Department has so many cases of wrongdoing that if he asked the SAPS to follow them up, it would amount to closing down the Department. ” (p87)
• “Of the 81 cases that were investigated, 38 cases provided grounds for either reasonable suspicion requiring further investigation or point directly to wrongdoing amounting to criminal conduct. (Reasonable suspicion exists when there are objective grounds or empirical facts that form the basis for the suspicion as opposed to mere conjecture.) “ (p18)
• North West – “When the claims were made in the City Press in April 2014, the NWDOE appointed a forensic company to investigate the claim that SADTU had influenced the appointment of senior officials in the provincial department. The report, dated 15 August 2011, was made available to the MTT. Further investigation by the MTT indicates that in 9 cases investigated by the forensic investigative company, there was no evidence to support the allegations in 2 cases. However in 6 cases SADTU officials or representatives exerted improper influence; however is no evidence of money being involved. In one case, a senior district official acted improperly in support of the promotion of a friend. In none of the cases was action taken in respect of the findings.“ (p20)
• Gauteng – “Following claims made in City Press in May 2014, The GDE acted promptly and appointed a firm of attorneys to investigate the claims of improper influence at school level. The detailed report included information from both written and verbal statements and concluded, inter alia, that posts are being sold for cash, that the parties operate in networks, and that there is a climate of fear that keeps people from exposing these practices. The GDE then moved to take disciplinary action, but the majority of witnesses did not wish to give signed statements.This outcome is unacceptable as in this case there is evidence that a network exists that is selling posts for cash. Steps need to be taken to support the individuals who were willing to take the risk of speaking. This requires further investigation by a competent authority. It is not acceptable that a teacher who is courageous enough to report alleged corrupt acts was told must be told to go to her local police station. This cannot be acceptable. In several cases where teachers and other informants have acted with enormous courage to report gross abuse no action has been taken.” (p20)
• “RECOMMENDATION 4: That the Department of Basic Education regain control of administering the education system in all Provinces so that clear distinctions are established between the roles and functions of the DBE and the concerns of Teacher Unions.” (p21)
• “The deployment of officials to the Department from Unions weakens the Department because those so deployed often struggle to demonstrate that they are able to balance the complexity of competing loyalties and demands.” (p25)
• “RECOMMENDATION 10: That both school- and office-based educators cease to be office- bearers of political parties and that educators in management posts (including school principals) be prohibited from occupying leadership positions in Teacher unions.” (p25)
• “If ‘undue influence’ (a polite name for corruption) is a result of cadre deployment, then cadre deployment is likely to lead to corruption. It is the impression of the Task Team that such corruption is endemic to greater and lesser degrees in the entire education system and that as a first move to cleanse the system, cadre deployment should not be permitted.” (p25)
• “RECOMMENDATION 15: That the South African Council of Educators (SACE) be reconceptualised and freed from Union and political domination.” (p28)
• “All the Deputy Directors-General in the Department of Basic Education are SADTU members and attend meetings of that Union. That being the case, it is not improbable to say that schooling throughout South Africa is run by SADTU. The significance of cadre deployment in South Africa’s education system is discussed below in Chapter Four.” (p93)
• “Early in May, the North West’s Deputy HOD was held hostage overnight by SADTU officials over the unresolved matter of temporary teachers.” (p.82)

Some media reports are starting to emerge in Times Live and Citizen but it will be interesting once the journalists and commentators get their teeth into the detailed report.

[Guest post – Stephen Taylor]

The Statistician General (SG) recently responded to my letter to the Business Day. A full version of what I have written is available here. I would simply reiterate what I wrote initially. In brief:

Numerous newspapers and media outlets were running with headlines such as “Black youth less educated now than 20 years ago” (Business Day) and “Stats SA claims black youth are less skilled than their parents” (Daily Maverick). The articles were referencing a recent Stats SA report as well as comments made by the SG.

My position is simple: These claims are factually incorrect and Stats SA reports and data do not actually say this. Note that I did not “adjudge a mistake” in the actual Stats SA report. I do not know exactly what the SG said at the release, which may have lead the media to run with the headlines they did, but I speculated about what I thought might have been causing the confusion.

The SG’s response has confirmed that I was right about what had caused the confusion. He had in mind a very specific “progression ratio” – the ratio of degree graduates to matriculants. This ratio may have been decreasing for black and coloured youth, but this is simply because the increase in degree graduates has not kept pace with the increase in matriculants. Importantly, both the likelihood of achieving a matric and the likelihood of achieving a degree have been fast increasing, especially for black youth. This means it cannot possibly be true that black and coloured youth are worse off educationally than their parents, which is the message the media were propagating. None of the media reports said anything about this “progression ratio” which the SG has now referred to. Instead the media were reporting that lower proportions of the entire black and coloured population were attaining a degree. And that isn’t true.

According to Higher Education data, the number of black degree graduates per year has increased more than four-fold from 11 339 in 1994 all the way to 48 686 in 2014.[Note: New SU Working Paper published by Hendrik van Broekhuizen last week on this topic]. This is much faster than population growth. For white youth, this number has essentially remained flat at roughly 20 000 graduates per year. Of course there is still a lot of inequality – these numbers mean whites are still about 6 times more likely to attain a degree than black youth.

This “progression ratio” may have some analytic value, as the SG points out. For example, it may reflect a greater policy prioritisation of secondary education relative to higher education. But this indicator must be clearly explained and it also has its limitations. I cannot imagine anyone celebrating improved educational outcomes if the numbers of black and coloured youth attaining a degree had gone down, while the numbers attaining a matric had gone down even faster, leading to an increase in the degree-to-matric progression ratio!

I know progress hasn’t been as fast as we need. But it is far from the truth to suggest educational outcomes have regressed.

Hi education folk

We’ve recently been awarded a grant by the UK Economic and Social Research Council (ESRC) and DFID to do research on exceptional township and rural primary schools in three provinces – Western Cape, KwaZulu-Natal and Limpopo. I will be leading the project together with a great team of co-investigators and researchers: Ursula Hoadley (UCT), Jaamia Galant (UCT), Nick Taylor (JET), Servaas van der Berg (ReSEP), Gabi Wills (ReSEP) and Linda Zuze (HSRC).

Given that we will be using a matched-pair approach, the aim of the project is to to firstly identify at least 10 high-performing township/rural schools in each of the three provinces and then match those schools to ‘typical’ neighbouring schools (thus 60 schools in total). The project will run for 2.5 years (beginning June 2016) and will involve in-depth qualitative research, administering and analysing assessments, creating new school-leadership and management instruments and analysing the data emerging from the project.

At the moment we are looking for a part time (50%) project administrator to help us with the administration relating to the project. The full job-advert can be found here. I include a short extract:

The ideal candidate should be an efficient and organised person with some experience in managing the administrative side of academic projects. They should be friendly and able to communicate well, as well as plan effectively and take initiative to solve problems as they arise. They should also be able to liaise with (1) the funders – DFID/ESRC, (2) Stellenbosch University finance and administration personnel, and (3) the 5 co- investigators as well as other researchers and field-workers. The job will include following up with government officials, managing flights and accommodation, planning reporting cycles, managing time-sheets, budgets and funding claims, proof-reading and printing questionnaires etc.

They should also be friendly 🙂 Basically we’re looking for a go-to admin guru to help us with this project. If your skills and experience are such that you would also be able to help on the non-admin research side (doing research-assistant work or interviews/assessment) then we may involve you in more than an administrative role. However this would be on top of your administrative responsibilities, and in such a case the remuneration is somewhat negotiable. It is a part-time position and it’s based at Stellenbosch University (in the Economics Department) so remuneration would be in line with university salary scales (I believe it is about post-level 11, although it would be 50% of the amount stated here) and partially dependent on qualifications and/or experience.

If you’re interested in working with this great team on an exciting new project please do apply 🙂 (and if you know of anyone who may be interested please forward this on to them!)

DEADLINE FOR APPLICATIONS: 12 May 2016

I’m currently involved in a number of research projects that look specifically at reading in the Foundation Phase in South Africa. In my view this is the cause of (and solution to) so many of the problems we see in our education system.

I thought it would be a good idea to get some of the SA reading researchers  in the same room to talk about the work they’re currently doing and talk about the way forward. So we’ve organized a half-day workshop for the 28th of April (8:30am-2:30pm). It will be an informal seminar of researchers and should be relatively small (<40 people) to make sure we have time to interact.

Prof Elbie Henning at UJ has kindly offered to host the workshop at UJ (as an aside I am now a part-time research fellow at her SARChI Chair).  See the programme below. There are a number of exciting presentations, perhaps most exciting is early feedback from Stephen Taylor et al’s RCT in the North West!

The good news is that we have additional space for 5 or 6 additional participants who would like to attend. So if you’re involved in reading research, teaching reading teachers or policy-making in the Foundation Phase and would like to come on the 28th please send me an email (nicholasspaull[at]gmail.com) with the title “Reading in FP Workshop.” Please also include a paragraph about who you are and why you’d like to attend.