It is well known that learner performance in mathematics in South Africa is poor. However, less is known about what learners actually do and the extent to which this changes as they move through secondary school mathematics. In this study a cohort of 250 learners was tracked from Grade 9 to Grade 11 to investigate changes in their performance on a diagnostic algebra test drawn from the wellknown Concepts in Secondary Maths and Science (CSMS) tests. Although the CSMS tests were initially developed for Year 8 and Year 9 learners in the UK, a Rasch analysis on the Grade 11 results showed that the test performed adequately for older learners in SA. Error analysis revealed that learners make a wide variety of errors even on simple algebra items. Typical errors include conjoining, difficulties with negatives and brackets and a tendency to evaluate expressions rather than leaving them in the required open form. There is substantial evidence of curriculum impact in learners’ responses such as the inappropriate application of the addition law of exponents and the distributive law. Although such errors dissipate in the higher grades, this happens later than expected. While many learner responses do not appear to be sensible initially, interview data reveals that there is frequently an underlying logic related to mathematics that has been previously learned.
This article has its origins in the discourse of poor performance and failure in school mathematics in South Africa and other developing countries. That said, not all learners in South Africa are failing mathematics but the vast majority are not coping (Spaull,
In our project we seek to shift the discourse from ‘learners can’t do’ to ‘what learners can do is …’. This requires sensitive diagnostic instruments that enable us to measure change over time, even when learners’ performance is poor. However, there are no adequate local diagnostic instruments available for this purpose. Consequently, we drew on a selection of algebra items from the Concepts in Secondary Maths and Science (CSMS) tests (Hart & Johnson,
Our findings show that many learners in Grade 11 still have difficulty with aspects of basic algebra. This reinforces the comments made on the performance of Grade 12 learners on the National Senior Certificate Paper 1 each year. For example, in 2012 the examiners commented:
Many of the errors made in answering this paper have their origins in poor understanding of the basics and foundational competencies taught in the earlier grades. For example, algebraic manipulation, factorisation, solution of equations and inequalities. (Department of Basic Education,
In this article we draw on quantitative and qualitative data to illuminate the kinds of errors that learners make in relation to basic algebra. Thus we go beyond merely noting
Our analysis also shows evidence of curriculum impact, that is, how learners apply new procedures and laws they have been taught, but in inappropriate ways. Typically, this changes over time as learners become more familiar with the new procedures and when they should be applied but the cycle continues as they learn further procedures and then apply these inappropriately. This finding gave rise to more substantial concerns regarding the use of the CSMS items outside the UK and with older learners. Based on a simple Rasch analysis of the Grade 11 data we show that the items performed sufficiently well for the purposes of this study although the nature of the errors is clearly impacted by learners’ exposure to more mathematics as they move from Grade 9 to Grade 11.
Internationally, there has been much interest in learners’ errors in mathematics for the past 30 years. The early work drew strongly on constructivist perspectives (e.g. BenZeev,
In South Africa, there has been a resurgence in the focus on learner errors and error analysis in mathematics in recent years (Brodie,
We work from a sociocultural perspective but draw on some aspects of discursive approaches (Sfard,
Research on the transition from arithmetic to algebra has been ongoing since the 1970s (e.g. Collis,
Linked to the CSMS study, Küchemann (
Despite all that has been learned, algebra remains a substantial obstacle for many learners of secondary mathematics internationally as borne out in the performance of SA learners on the CSMS items 45 years after they were first developed.
The algebra items used in this study were first developed and administered to a nationally representative sample of English learners in the 1970s as part of the CSMS study. The design of the items was strongly informed by Piagetian ideas that were dominant at the time and were intentionally designed to be recognisably connected to the UK mathematics curriculum of the time, but required learners to: use methods ‘which were not obviously ‘rules’ (Hart & Johnson,
The CSMS algebra test focuses on generalised arithmetic, the use of symbols to denote numbers and letters as variables (Collis,
Over the period 2010 to 2014 our project worked in 10 schools in the Johannesburg area in South Africa. Six of the schools are in socalled townships where there are only basic resources for teaching and learning. The other four schools are lowfee schools located in suburban areas. Here too resources are limited although these schools are typically better equipped than those in the townships. In all schools, English is the language of instruction but almost no learners speak English as their main language.
We set out to track a cohort of learners from Grade 9 to Grade 11 (approximately 15 to 17 years old) across all project schools, regardless of whether they progressed to the next grade at the end of the year and regardless of whether they chose Mathematics or Mathematical Literacy
The limitations of this sample are that it became a small, opportunistic, nonrandom sample with a smaller proportion of weaker learners than the intended sample. Nevertheless, it provides a unique snapshot of learners’ performance in algebra across the transition from Grade 9 to Grade 11.
The test instruments did not consist only of CSMS items. We designed three tests, one for each grade, containing curriculumrelated algebra and functions items and a selection of CSMS algebra items which were common to all three tests. We focus here only on the CSMS items, which have been validated in the UK in both studies. In selecting the items, we chose only those clusters of items that had been ‘levelled’ in the CSMS hierarchy (see Hart,
In 2010 the CSMS items were piloted with Grade 8 and Grade 10 learners in schools similar to the project schools. The results of this pilot indicated that the items and the associated coding scheme were appropriate in the SA context. The three full tests were then piloted in project schools with different cohorts prior to administering to the tracked cohort.
The tests were administered in October each year (i.e. near the end of the school year). In 2011 and 2012 they were administered by project team members. In 2013 they were administered by team members in some schools and by teachers in other schools. We deliberately wanted to include teachers in the research process but, with hindsight, this was not wise because some teachers did not follow the directions given regarding who was to write the test, which ultimately impacted on the size of the tracked cohort.
The test codes were adapted from the CSMS codes. In the main, we used the most prevalent CSMS codes and then added additional codes to capture the wide range of responses that would otherwise have merely been coded as ‘other’. Coding was carried out by trained research assistants and moderated by senior members of the project team.
In 2012 and 2013, following the analysis of pilot data and the test scripts, several small studies were conducted by postgraduate students (Honours level) who were supervised by senior members of the project team. These studies involved an analysis of scripts and oneonone taskbased interviews where each postgraduate student interviewed learners from project schools on questions similar to selected test items in order to investigate learners’ thinking in relation to introductory algebra. We draw on these studies to illustrate our claims about learners’ thinking and strategies in relation to their errors. We explicitly acknowledge the postgraduate students as authors of the work reported here. However we have retranscribed and reanalysed all interview extracts that are included in this article.
Ethical clearance was obtained from the university (2010ECE60C) and from the Gauteng Department of Education for this research. Informed consent was obtained from schools, parents and learners for both the tests and the interviews. Names of learners and schools have been kept confidential and we have not reported separately on particular schools. Pseudonyms have been used in all learner transcripts. Permission was obtained from the postgraduate students to use their data as part of the wider project research.
As noted above, the validity of the items for the SA context was confirmed through piloting. The reliability of the coding was increased through a moderation process: all learner interviews were retranscribed and they were then reanalysed by two of the authors to confirm the initial interpretation.
Since the CSMS items had been developed for Year 8 and Year 9 learners in the UK, it was necessary to validate it for older learners in SA. We therefore conducted a simple Rasch analysis, using Winsteps (Version 3.73)
We used three fit statistics to judge the quality of the selected CSMS items as an instrument for measuring learners’ understanding of algebra in the SA context: infit mean square (MNSQ), outfit MNSQ and itemscale correlation. Since the test is a low stakes diagnostic test, we took a relatively liberal approach and so values of infit and outfit MNSQs higher than 2.0 or lower than 0.5 were regarded as a cause for possible concern (Linacre,
A summary of learners’ performance on the items across the three years is provided in
Percentage of correct responses to each item per grade.
Item  Percentage of correct responses  

Grade 9  Grade 10  Grade 11  
1.1  Simplify 2 
86  73  86 
1.2  Simplify 2 
34  37  46 
1.3  Simplify ( 
25  21  35 
1.4  Simplify (2 
44  43  52 
1.5  Simplify ( 
6  11  30 
1.6  Simplify 3 
8  14  30 
1.7  Simplify 
31  42  59 
1.8  Simplify 3 
34  43  56 
1.9  Simplify ( 
7  12  30 
2  Which is larger, 2 
0  1  1 
3.1  Add 4 to 8  82  74  78 
3.2  Add 4 to 
36  46  61 
3.3  Add 4 to 3 
26  37  56 
4.1  Multiply 8 by 4  70  70  77 
4.2  Multiply 
9  22  34 
4.3  Multiply 3 
64  64  66 
5.1  If 
82  85  87 
5.2  If 
30  40  46 
5.3  If 
15  21  41 
6.1  Determine 
78  85  86 
6.2  Determine 
8  14  30 
7.1  If 
50  60  71 
7.2  If 
32  52  66 
8.1  Area of rectangle 3 × 4  52  59  61 
8.2  Area of rectangle 6 × 10  52  58  61 
8.3  Area of rectangle 
62  70  67 
8.4  Area of rectangle 5 × ( 
2  5  11 
9.1  Perimeter, if sides 9 + 1 + 10 + 2  95  92  89 
10.1  Perimeter, if sides 
81  86  88 
10.2  Perimeter, if sides 
56  64  75 
10.3  Perimeter, if sides 
56  50  64 
10.4  Perimeter, if 
12  18  23 
11  Cakes cost 
20  22  28 
The general performance of the learners over the three years is low but this was not unexpected. There is a general increase from Grade 9 to Grade 11, although there are six items where performance dipped from Grade 9 to Grade 10 and two items where performance dropped from Grade 10 to Grade 11. Several of these are discussed in later sections. The four items that Grade 11 learners found most difficult were Q2, Q8.4, Q10.4 and Q11. More than 16% did not attempt the latter three items in Grade 11. By contrast fewer than 1% of learners omitted Q2 in Grade 11 although only 1% answered it correctly.
It is surprising that Q1.5, Q1.6 and Q1.9 were among the most difficult items for learners at Grade 11 level since these look similar to questions found in Grade 8 and Grade 9 textbooks in SA. Reflecting on Question 1 as a whole, it is noticeable that the performance on questions with brackets or negatives is well below the other items for all three years. For example Q1.3 and Q1.5 are very similar except for the operation in the bracket. Yet the performance on these two items is considerably different, particularly in Grade 9 and Grade 10. Other errors include inappropriate application of the addition law of exponents and the distributive law. These errors are discussed in more detail in the next section. The poor performance on Q1.2 appears to be related to learners’ expectation that they should do something with the unlike terms to produce an answer.
The slight drop in performance on Q3.1 from Grade 9 to Grade 11 is surprising, as is the small size of the gain on Q4.1, Q8.1 and Q8.2. One possible explanation is that asking simple arithmetic questions in the context of an algebra test confuses learners. Alternatively, given that the first part of our test consisted of curriculumlevel questions, some learners might have expected that these questions were more complex than intended.
With regard to Question 5, there is a noticeable contrast in performance between Q5.2 and Q5.3 in Grade 9 and Grade 10, although this was expected. It is also worth noting that responses to Q5.2 were unstable with many learners moving from a correct response in one year to an incorrect response the following year. For example, in Grade 9, 80 learners gave the correct answer but of these 44% got it incorrect in Grade 10. Similarly in Grade 10, 100 learners gave the correct answer but of these, 30% got it incorrect in Grade 11. It may be that this item is more errorprone than initially appreciated. If one merely glances at the question it is quite easy to focus on the increase on the left side and assume the same on the right side whereas it requires paying more careful attention to the relationships to recognise that such reasoning is faulty.
Question 6 and Question 7 involved substitution. Learners had particular difficulty with Q6.2. Given the prevalence of premature closure in their responses across the test, it is possible that many learners could not make sense of the equation
Question 9 and Question 10 required algebraic expressions for the perimeter of polygons. While performance improved systematically on Q10.2, performance on Q10.3 dropped in Grade 10 and was still poorer than Q10.2 in Grade 11. The higher number of errors is likely due to the combination of letters and numbers in Q10.3 with 23% of Grade 11 learners making conjoining errors and hence expressing the perimeter as 18
Question 11 was the only CSMS item selected that focused explicitly on letter as object. Not surprisingly many learners treated the letters as objects with at least 35% of learners each year stating that 4
While learners’ poor performance is reason for concern, an analysis of their errors provides insight into their reasoning and shows evidence of rational albeit incorrect strategies. These insights enable us to focus on what learners can do rather than on what they can’t do. In this section we focus on key errors and illustrate these with examples from Questions 1, 3, 4 and 5. We also include interview data from the postgraduate students’ research projects.
The most common errors involve conjoining and premature closure, negatives and subtraction, multiplication and indices, the equality relationship and evaluating letters rather than accepting an open expression as a final answer. We also provide evidence that learners change their strategies depending on particular details of a question.
Errors involving conjoining or premature closure occur in the responses to many items as shown in
Percentage of responses showing conjoining error.
Item  Typical errors  % of all responses  

Grade 9  Grade 10  Grade 11  
1.2  Simplify 2 
7 
53  50  42 
1.4  Simplify 2 
7 
25  18  14 
3.2  Add 4 to 
9 
11  12  7 
3.3  Add 4 to 3 
7 
25  32  17 
5.3  If 


8 
17  11  10 
In Q1.2 it could be argued that the nature of the question prompts learners to do something and so a reasonable response is to conjoin 2
They said add 5 to 3
They say multiply 3
When asked to ‘multiply
Then [
Okay, how did you get the 6?
Okay, I said 2 times 3 which is 6. Then I brought down the
The learners ‘simplify’ the binomial by attending first to the numbers, then appending the letter and then multiplying by the constant. Their use of language shows explicitly that when they conjoin unlike terms, they ignore the letter and then append it as one writes units in a measurement problem. Both learners use ‘put’ to explain how the letter comes to be placed next to the number and Shenaaz also says ‘I brought down the
The error of attending first to numbers is persistent in terms of the proportion of errors, as illustrated in
Percentage of errors showing focus on numbers.
Item  Typical errors  % of all errors  

Grade 9  Grade 10  Grade 11  
3.2  Add 4 to 
9 
32  38  34 
3.3  Add 4 to 3 
7 
47  59  49 
4.2  Multiply 

46  59  54 
In the sections that follow we continue to point out errors involving conjoining although they are no longer in the foreground. It is worth noting at this point that when learners conjoin, their talk typically reflects an operational view of the expression they are dealing with. This is evidenced in treating subtraction as ‘take away’ and in counting the numbers of things they are working with, for example the numbers of
Learners’ errors in simplifying algebraic expressions suggest they may not be paying attention to signs and operations. This is more visible when the question involves subtraction and negatives and it appears that learners are focusing mainly on letters and numbers. For example in Q1.5 more than 10% of Grade 9 and Grade 10 learners gave answers of
Kalidheen (
It’s going to be 8 minus 2
So you’ve got subtract 2
Still 8.
So you left out the 2
Cos, the 2
Ok, so what happens if I make this 2
2
…
And then, if I had … ok let’s say I change this to 2
2
Ok, so, what would happen here? What will the answer be?
No mam, it will be left like this (i.e. 2
Simon provides the correct answers when both terms contain letters. However, when only one term contains a letter, he eliminates the term because it cannot be subtracted from 8 ‘while there’s no
In contrast, other interviews show evidence of learners paying explicit attention to signs and operations yet overgeneralising methods from equation solving when working with expressions (e.g. Gumpo,
They said simplify. The first thing that I did, I grouped the like terms then I got the answer.
How did you group your like terms?
They said 3
Let us go back to where you added the like terms. You said when
Because that is how mam taught me that when
Oh! When it is an equation. Is this an equation?
Yes, ma’am.
Why do you say it is an equation?
Because, ma’am, it has the variables.
Themba is confident in his selection and use of a strategy, albeit an inappropriate one. He changes the sign if a term moves to another position in the expression and his justification reveals some confusion between an equation and an expression. It also reflects inappropriate criteria for defining an equation and incorrect description of the objects he is working with, such a referring to a term as an equation.
There is much evidence of learners applying the distributive law and the addition law of exponents inappropriately across various items in Question 1, particularly in Grade 10. Such errors account for the dip in the percentage of correct responses in Q1.1 and Q1.3 in Grade 10. By Grade 11 these errors were less frequent and this partially explains the substantial increase in the percentage of correct responses in items involving brackets.
Errors involving the addition law for exponents, that is,
Percentage of incorrect responses involving exponential laws.
Item  Typical responses  % of all responses  

Grade 9  Grade 10  Grade 11  
1.1  Simplify 2 
7 
10  17  7 
1.4  Simplify 2 
7 
10  18  15 
1.5  Simplify ( 

8  17  13 
Many learners treated the presence of brackets as a signal to multiply, not paying attention to the operations adjacent to the brackets. Consequently there is much evidence of the incorrect application of the distributive law in questions involving brackets. This is clearly illustrated in Q1.5 and Q1.6 as shown in
Percentage of incorrect responses involving distributive law.
Item  Typical responses  % responses  

Grade 9  Grade 10  Grade 11  
1.5  Simplify ( 

3  15  12 
1.6  Simplify 3 
±3 
3  15  9 
1.9  Simplify ( 

10  26  21 
Of all the CSMS test items, Question 1 was most similar in form to SA curriculum items in algebra. Consequently, learners saw opportunity to use the algebraic laws and procedures they had been taught and this may account for the inappropriate use of such curriculum knowledge. Exponential laws are introduced informally in Grade 9, then formalised for integer exponents in Grade 10 and extended for rational exponents in Grade 11. The increased attention given to the laws in Grade 10 may explain the spike in this error. The drop in Grade 11 may reflect that this knowledge is now more stable. The trend in Q1.4 and Q1.5 is similar.
The distributive law is first introduced in Grade 8 and reinforced in Grade 9 for more complex algebraic expressions. It is therefore surprising that the misapplication of the law appears more frequently in Grade 10 and not in Grade 9 as was the case with the exponential law. The reduction in errors involving the distributive law suggests that learners are more familiar with the correct application of the law in Grade 11.
In Q1.9 there is evidence of both the addition law of exponents and some attempt at the distributive law. It appears that that many learners treated the question as the product of binomials: (
When learners are not yet able to deal with a lack of closure in algebraic expressions, a common strategy is to evaluate the letters (Küchemann,
Both strategies are reported in other studies that have made use of the CSMS items (MacGregor & Stacey,
In the interview extracts we gain insight into learners’ reasoning about Q5.3 or similar items from the Honours projects. Thabo, a Grade 9 learner interviewed by Brown (
If you can take a look at
Thabo says ‘it actually tell us’ indicating that he perceives the value of
So how did you get 3?
I just guessed the 3. I just thought this 6, they are two variables, meaning I add two numbers to get 6.
Ok.
3 plus 3, and here also I said 3 because there are 3 numbers, I also used the same number that I thought they used here.
In both cases learners ‘know’ that they need to obtain values for the letters and both make use of an equal splits strategy. Thabo assumes the values are implied in the question and recognises that he can multiply to obtain the sum. Lindo guesses a value of 3 but in another section of the interview acknowledges that she could have chosen different values.
Another strategy, which was more common among Grade 11 learners, involved treating the expression as an equation and solving for the unknown. This is illustrated in Du Plessis (
I thought, I thought eh here neh, since
Bokang’s response to ‘if
Bokang interprets the question to mean that he needs to find the value of
There is evidence across several interview items that learners change their strategies depending on the specifics of the question. For example, Du Plessis (
The first question says simplify where possible. The equation is 2
…
Which ones are the like terms?
2
When you say term, what do you mean?
I mean the number.
When dealing with subtraction, Musa reasons differently. He is given 2
2
Musa’s strategy for adding terms is to identify like terms although he does so based on an incorrect definition. When subtracting, he makes no mention of like terms. Instead he assigns 1 to
The quantitative and qualitative analyses provide complementary insights into learners’ errors in basic algebra. From the quantitative evidence we see that conjoining errors become less frequent by Grade 11 but they are still persistent. Another persistent error is the focus on numbers and the lack of attention to operations. Learners’ talk confirms that they are separating numbers from letters when dealing with expressions.
Errors related to the learning of new procedures appear to depend on
Learners’ performance on items involving subtraction and negatives improved substantially from Grade 10 to Grade 11 but was still below performance on items without brackets and negatives. Further research needs to focus on learners’ reasoning when dealing with subtraction and negatives both in number work and algebra. The interviews show evidence of learners talking of subtraction as ‘take away’, even when dealing with algebraic terms. This restricted view of subtraction may be a consequence of limited exposure to other subtraction structures such as reduction, comparison and inverseofaddition (Haylock,
There is widespread evidence in the tests and interviews of learners evaluating letters rather than accepting expressions such as 8 +
The interview data provides evidence of learners incorrectly naming mathematical objects, for example referring to like terms as those with coefficients other than one, regardless of the letter. In other work emanating from the project we have argued for paying greater attention to the words learners use to name or refer to the mathematical symbols, expressions, graphs and for more opportunities for learners to talk mathematics (Adler & Ronda,
One of the most alarming observations each year was the high percentage of errors that were coded ‘other’, meaning that there was insufficient trend to add a particular code for the error. Although the diversity of such errors reduced by Grade 11, the apparent idiosyncrasy of the errors remains a cause for concern. However, the interviews across all postgraduate projects confirmed that learners are frequently able to justify their strategies and answers with some rational connection to the mathematics they have learned. This highlights the need to gain deeper insight into learners’ reasoning as they work with basic algebra and hence for teachers to provide more opportunity for learners to explain their strategies verbally.
In this article we have taken a first step in identifying and describing the algebraic errors that learners make as they move from Grade 9 to Grade 11. We have also provided some evidence of their reasoning in relation to these errors. This enables us to see what learners
Explanations for learners’ errors depend on theoretical orientations but there is agreement across perspectives that learners need to move from operational to structural ways of thinking about the symbols and their relationships. This will require deliberate instruction with tasks that push learners towards a structural view. Furthermore, increased attention to learners’ ways of speaking about algebra will provide greater insight into their thinking. There is no ‘quick fix’ to deal with learners’ errors in algebra, particularly when one sees how some errors persist deep into senior secondary mathematics. However, the insights provided by the research reported here give some indication of useful starting points such as specific attention to the meaning of brackets and a stronger focus on negatives and different views of subtraction.
This work is based on the research of the Wits Maths Connect Secondary Project at the University of the Witwatersrand, supported by the FirstRand Foundation Mathematics Education Chairs Initiative and the Department of Science and Technology and administered by the National Research Foundation. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the institutions named above. The authors wish to thank Vasen Pillay for the vital role he played as data manager in the project.
The authors declare that they have no financial or personal relationship(s) that may have inappropriately influenced them in writing this article.
C.P. conducted data analysis and wrote the majority of the article; J.H. did Rasch analysis, wrote sections of the article relating to CSMS design and the validation test for SA; Y.S. retranscribed interview data, checked accuracy of transcripts and conducted data analysis; J.A. was project leader and provided critical comments on drafts of the article.
In South Africa, all students are required to take mathematics for all 12 years of schooling. Typically they will choose between Mathematics or Mathematical Literacy at the end of Grade 9. Many learners change from Mathematics to Mathematical Literacy in Grade 10 or Grade 11 if they are not coping with Mathematics.
Following Linacre (