This case study carried out during the 2020 coronavirus disease of 2019 (COVID-19) lockdown used online data collection means to investigate the distribution of cognitive demand levels of probability and counting principles (PCP) learning tasks in a popular online Grade 12 mathematics textbook, based on the PCP teachers’ rating. The teachers’ cognitive demand ratings were categorised following Stein’s mathematical task framework. Five mathematics teachers from four secondary schools in two provinces in South Africa participated in the study by filling in an online questionnaire. We developed a rating framework named the mean cognitive demand rating (MCDR) to help us interpret the teachers’ perception of the tasks in terms of cognitive demand to the learners. Data from the teachers’ ratings revealed nearly 65% of the PCP learning tasks in the online textbook were rated as high. Analysis of secondary data from Department of Basic Education diagnostic reports from 2014 to 2020, however, suggests no association between teachers’ rating of learning tasks and learner performance.

This study draws attention to a long-standing underperformance in the topic of probability and suggests classroom-based study that focuses on the learners’ rating of the learning tasks themselves to understand clearly how best to support them.

The worldwide spread of the coronavirus disease of 2019 (COVID-19) in the year 2020 brought changes to the way societies live, work and study. The institutions affected by COVID-19 responded by moving from physical, human-to-human interaction to virtual and online platforms. During 2020, South African schools were shut down for two and a half months before Grade 7 and Grade 12 pupils were permitted to return to school. During the shutdown, learning continued virtually and online for some schools, particularly the relatively well-resourced schools (Mohohlwane, Taylor, & Shepherd,

Online learning resources include resources such as online video lessons on YouTube, digital textbooks (DT), and study guides. These resources can be accessed by learners working from home on a computer or on a smartphone connected to the internet. In this study, we focus on only one online resource, namely the DT used by Grade 12 mathematics learners and teachers in South Africa.

Due to the closure of schools in 2020 due the prevalence of COVID-19, we did not conduct classroom-based study involving the Grade 12 learners. We noticed that this online textbook was being heavily used by both Grade 12 teachers and learners and decided to investigate the teachers’ rating of learning tasks in the probability and counting chapter of the book. We were able to carry out an online survey with the Grade 12 mathematics teachers who are involved in teaching probability and counting principles (PCP).

The DT approved by the DBE is freely downloadable on any mobile device such as tablets and mobile phones. The DT comprises 9 mathematics topics (sequences and series, functions, finance, trigonometry, polynomials, analytical geometry, Euclidean geometry, statistics, and probability) that are taught at Grade 12. The 9 topics are each written following the same format; thus, each topic begins with the revision of related concepts, followed by the content notes, a couple of worked out examples, and exercises at the end. The exercises have answers to enable learners to cross-check their solutions. In this article, we only discuss the topic of PCP.

Probability theory is a mathematical modelling (Blum et al.,

Regarding counting principles, these are techniques for determining without direct enumeration, the number of possible outcomes of a particular experiment (May, Masson, & Hunter,

By multiple representations, we mean techniques of teaching PCP that include various objects such as graphs, diagrams, texts, and 3D visualisations to facilitate learners’ grasping of the underlying meaning of the concepts.

Probability and fundamental counting principles are relatively new topics in the South African mathematics syllabus (Zondo, Zewot & North,

There is also a lack of research to inform the teaching and learning of PCP at the school level. For example, a database search by the first author of articles in

One of the proposals from the education authorities to try and reverse the poor performance at Grade 12 is the suggestion that the teaching and learning of PCP should incorporate multiple representations of tasks. Based on the mathematical task theoretical framework (Stein & Smith,

The Curriculum Assessment Policy Statements (CAPS) diagnostic reports (DBE,

This case study investigated the Grade 12 mathematics teachers’ rating of learning tasks in a PCP chapter in a popular Grade 12 mathematics online textbook. Teacher ratings were interpreted following Stein and Smith’s (

The idea of probability is empirical. That is, probability describes what happens in very many trials, and we must observe many trials to pin down a probability. In this article, we use the definition based on the notion of proportion or relative frequency. Relative frequency of a score is obtained by dividing the frequency of that score by the total number of scores. Similarly, the probability of an experiment yielding a particular result (e.g. a coin toss yielding heads) can be defined as the number of equally likely and mutually exclusive outcomes, divided by the total number of possible, equally likely, and mutually exclusive outcomes. By equally likely, we mean that in the long run each of the possible outcomes will occur with approximately equal frequency (May et al.,

Probability knowledge for teaching (PKT) includes content knowledge of probability, and various ways of presenting this content to the learners so that learning takes place (Batanero, Chernoff, Engel, Lee, & Sanchez,

Probability pedagogical content knowledge is knowledge about presenting the concepts to the learners so that learners easily understand them (Kazima & Adler,

In the mathematical task framework (MTF), a task is defined as a segment of classroom activity that is devoted to the development of a particular mathematical idea (Stein & Smith,

The mathematics tasks framework.

This article will limit the discussion to phase 1 of Stein and Smith (

As shown in

Stein et al. (

Cognitive demand levels used in mathematical task framework.

Levels of demands of tasks |
---|

Involves reproducing previously learned facts, rules, formulas, or definitions. Cannot be solved because a procedure does not exist. Involves the exact reproduction of previously seen material, and what is to be reproduced is clearly and directly stated. Tasks have no connection to the concepts that underlie them. |

Algorithmic. Use of the procedure either is specifically called for or is evident from prior instruction. Require limited cognitive demand for successful completion. Have no connection to the concepts or meaning that underlie the procedure being used. Are focused on producing correct answers rather than developing mathematical understanding. Require no explanations or, if any, explanations that focus solely on describing the procedure that was used. |

Focus learners’ attention on the use of procedures for the purpose of developing deeper levels of understanding of mathematical concepts and ideas. Suggest explicitly or implicitly pathways that are broad and have connections to the underlying conceptual ideas. Can be represented in multiple ways, such as visual diagrams, symbols, and graphs that help develop meaning. General procedures may be applied, but the procedure cannot be fused mindlessly. Learners need to engage with conceptual ideas that underlie the procedures to complete the tasks. |

Require complex and non-algorithmic thinking. Pathways to solutions are not explicitly suggested by the task, or by the task instructions. Require learners to explore and understand the nature of mathematical concepts, processes, or relationships. Demand self-monitoring or self-regulation (Tanner & Jones, 2005) of one’s own cognitive processes. Require learners to access relevant knowledge and experiences and make appropriate use of them in working through the task. Require learners to analyse the task and actively examine task constraints that may limit possible solution strategies and solutions. Require considerable cognitive effort and may involve some level of anxiety for the learner because of the unpredictable nature of the solution process required. |

The MTF is used to classify PCP tasks found in a DT in terms of either high or low cognitive demand levels. Tasks that are set at a high cognitive demand level require multiple strategies to solve (Stein et al.,

The probability that Jabu likes tea is 0.6 and the probability that Jabu likes coffee is 0.3. If the probability that Jabu likes tea, coffee or both is 0.7, determine the probability that Jabu likes tea and coffee.

This task does not demand much more than using a formula and substituting in the respective values, then solving for the unknown. Let T represent tea, and C represent coffee,

Tasks of high cognitive demand level require some thinking and reasoning to solve (Stein et al.,

Government data in Country Z show that 10% of adults are full-time students and that 35% of the adults are age 50 years or older. Explain why we cannot conclude that because (0.10) (0.35) = 0.035, therefore about 3.5% of adults are college students aged 50 years or older.

One reason is that the two events are not necessarily independent, because not all 10% of adult full-time students are above 50 years of age. Moreover, it is reasonable to expect that younger adults are more likely than older adults to be college students. Hence, P(college student|over 50 years) < 0.10. This example fits in level 3 or level 4 of Stein et al.’s (

The PCP tasks used in this article are obtained from a digital Grade 12 mathematics textbook. The textbook is endorsed by the DBE in South Africa. The book is freely available to South African users. Users are free to download and read the book on their mobile devices or print and read offline. The only restriction is for users to keep the book’s cover, title, contents, and short-codes unchanged.

We chose the digital book from among other books for three reasons. First, the book is used by many Grade 12 mathematics teachers and learners in South Africa, so it is a popular learning resource. Second, the book is freely available. Third, the book covers all mathematics topics taught in Grade 12 in South Africa. In this article, we focus only on the topic of PCP.

The PCP section is divided into eight sub-topics. For the purposes of this article, we limited our discussion to only four sub-topics, namely: the fundamental counting principles, factorial notation, tasks involving the application of counting principles, and tasks involving application of probability. We picked a total of 48 different learning tasks and asked five senior mathematics teachers at Grade 12 to rate the tasks according to the four levels of cognitive demand developed by Stein et al. (

This study is a case study taking a descriptive statistical approach. This approach enables us to transform qualitative data into quantifiable form and use it to make sense of the cognitive demand levels of learning tasks in PCP.

Participants in the study are five secondary mathematics and probability teachers, pseudo-named A, B, C, D, and E to ensure anonymity. Initially, seven secondary mathematics teachers (six male and one female) were contacted by email to take part in the study. A questionnaire with clear instructions was emailed to all the seven teachers to complete. However, only five teachers, all male, from four secondary schools in two provinces in South Africa (Gauteng and KwaZulu-Natal) returned the questionnaire. The five questionnaires were entered into a spreadsheet (

Cognitive demand rating of 48 probability and counting principles tasks in the digital textbook by five senior mathematics teachers.

SN | Exercise (Task #) | Task cognitive demand rating by teacher |
Mean rating | Mean rating (to the nearest digit) | ||||
---|---|---|---|---|---|---|---|---|

A | B | C | D | E | ||||

1 | 10.4 (1) | 2 | 1 | 1 | 2 | 1 | 1.4 | 1 |

2 | 10.4 (2) | 1 | 2 | 2 | 2 | 1 | 1.6 | 2 |

3 | 10.4 (3) | 1 | 2 | 2 | 2 | 2 | 1.8 | 2 |

4 | 10.4 (4) | 3 | 4 | 3 | 3 | 2 | 3.0 | 3 |

5 | 10.4 (5) | 2 | 1 | 3 | 3 | 3 | 2.4 | 3 |

6 | 10.4 (6) | 4 | 3 | 3 | 3 | 3 | 3.2 | 3 |

7 | 10.4 (7) | 2 | 2 | 4 | 3 | 4 | 3.0 | 3 |

8 | 10.5 (1) | 2 | 1 | 1 | 1 | 1 | 1.2 | 1 |

9 | 10.5 (2) | 2 | 2 | 1 | 1 | 1 | 1.4 | 1 |

10 | 10.5 (3) | 2 | 4 | 2 | 1 | 2 | 2.2 | 2 |

11 | 10.6 (1) | 1 | 1 | 2 | 1 | 1 | 1.2 | 1 |

12 | 10.6 (2) | 2 | 1 | 3 | 1 | 1 | 1.6 | 2 |

13 | 10.6 (3) | 1 | 1 | 3 | 1 | 2 | 1.6 | 2 |

14 | 10.6 (4) | 3 | 2 | 3 | 2 | 2 | 2.4 | 2 |

15 | 10.6 (5) | 3 | 1 | 3 | 3 | 3 | 2.6 | 3 |

16 | 10.6 (6) | 2 | 2 | 3 | 3 | 3 | 2.6 | 3 |

17 | 10.6 (7) | 2 | 3 | 3 | 3 | 3 | 2.8 | 3 |

18 | 10.6 (8) | 3 | 2 | 3 | 3 | 4 | 3.0 | 3 |

19 | 10.6 (9) | 3 | 2 | 3 | 3 | 4 | 3.0 | 3 |

20 | 10.6 (10) | 4 | 2 | 4 | 3 | 4 | 3.4 | 3 |

21 | 10.6 (11) | 4 | 2 | 4 | 3 | 4 | 3.4 | 3 |

22 | 10.7 (1) | 3 | 3 | 2 | 3 | 2 | 2.6 | 3 |

23 | 10.7 (2) | 3 | 3 | 3 | 3 | 2 | 2.8 | 3 |

24 | 10.7 (3) | 3 | 3 | 3 | 3 | 3 | 3.0 | 3 |

25 | 10.7 (4) | 3 | 2 | 3 | 3 | 3 | 2.8 | 3 |

26 | 10.7 (5) | 3 | 3 | 3 | 3 | 3 | 3.0 | 3 |

27 | 10.7 (6) | 3 | 3 | 3 | 3 | 3 | 3.0 | 3 |

28 | 10.8 (1) | 4 | 3 | 3 | 3 | 2 | 3.0 | 3 |

29 | 10.8 (2) | 3 | 2 | 3 | 3 | 2 | 2.6 | 3 |

30 | 10.8 (3) | 3 | 2 | 3 | 3 | 3 | 2.8 | 3 |

31 | 10.8 (4) | 3 | 3 | 3 | 3 | 3 | 3.0 | 3 |

32 | 10.8 (5) | 4 | 2 | 4 | 3 | 4 | 3.4 | 3 |

33 | 10.8 (6) | 4 | 3 | 4 | 3 | 4 | 3.6 | 4 |

34 | 10.8 (7) | 4 | 3 | 4 | 3 | 4 | 3.6 | 4 |

35 | 10.8 (8) | 4 | 4 | 4 | 3 | 4 | 3.8 | 4 |

36 | 10.9 (1) | 4 | 4 | 3 | 3 | 3 | 3.4 | 3 |

37 | 10.9 (2) | 4 | 3 | 3 | 3 | 3 | 3.2 | 3 |

38 | 10.9 (3) | 3 | 3 | 3 | 3 | 3 | 3.0 | 3 |

39 | 10.9 (4) | 3 | 2 | 3 | 3 | 3 | 2.8 | 3 |

40 | 10.9 (5) | 4 | 3 | 3 | 3 | 3 | 3.2 | 3 |

41 | 10.9 (6) | 3 | 2 | 3 | 3 | 3 | 2.8 | 3 |

42 | 10.9 (7) | 3 | 3 | 3 | 3 | 4 | 4.0 | 4 |

43 | 10.9 (8) | 3 | 3 | 4 | 3 | 4 | 3.4 | 3 |

44 | 10.9 (9) | 2 | 4 | 4 | 3 | 4 | 3.4 | 3 |

45 | 10.9 (10) | 3 | 2 | 4 | 3 | 4 | 4.0 | 4 |

46 | 10.9 (11) | 4 | 3 | 4 | 3 | 4 | 3.6 | 4 |

47 | 10.9 (12) | 3 | 3 | 4 | 3 | 4 | 3.4 | 3 |

48 | 10.9 (13) | 3 | 4 | 4 | 3 | 4 | 3.6 | 4 |

1, Memorisation; 2, Procedure with no connection; 3, Procedure with connection; 4, Doing mathematics.

The mean cognitive demand ratings (MCDR) by five senior PCP teachers from four schools in two provinces in South Africa were received by email by both authors of this article. Each teacher rated the 48 tasks independently of the other teachers. The first author entered the original data received from all teachers in

In this article, we have categorised MCDR 1 and 2 as low, and MCDR 3 and 4 as high (Stein et al., _{i}

Frequency table of the mean cognitive demand ratings of probability and counting principles learning tasks.

Level | Frequency | Percentage | Cumulative percentage |
---|---|---|---|

1 = Memorisation | 4 | 8.3 | 8.3 |

2 = Procedures without connections | 6 | 12.5 | 20.8 |

3 = Procedures with connections | 31 | 64.6 | 85.4 |

4 = Doing mathematics | 7 | 14.6 | 100 |

Bar graph of the mean cognitive demand rating of probability and counting learning tasks in the digital textbook by probability and counting principles teachers.

From

From

From

The code to a safe consists of 10 digits chosen from the digits 0 to 9. Assuming that none of the digits is repeated, determine the probability of having a code with the first digit even and none of the first three digits is 0.

Such a task includes procedures with connection, but it also requires some reasoning skills from the learner to solve.

This case study focused on and investigated Grade 12 senior mathematics teachers’ rating of learning tasks in a PCP chapter of a popular Grade 12 online textbook. Teacher ratings of tasks were interpreted following Stein and Smith’s (

On the first question, 65% of the learning tasks in the chapter on PCP were rated by the teachers in this sample as procedures, but with some connections to other concepts and representations, which supported learning. Characteristics of such tasks include use of procedures, but after obtaining the numerical solutions, learners are expected to interpret the solutions. Other examples include interpreting the concepts of probability that have been represented in a diagram such as a tree diagram. We argue concepts such as the ones in our example engage learners beyond the procedures and can help them to understand underlying concepts in the tasks (Stein et al.,

However, findings in this study also revealed that teachers in this study rated 79% of the learning tasks in the DT as having high cognitive demand. Only 21% of tasks in the DT the teachers rated as having low cognitive demand. If the teachers’ seemingly favourable rating of the learning tasks is true, the question that remains unanswered is: what explains learners’ general underperformance in the PCP topic in the Grade 12 national examination?

Under the new CAPS syllabus (DBE,

Grade 12 learners’ mean pass rate in probability (2014–2020).

Year | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 |
---|---|---|---|---|---|---|---|

Probability | 39 | 28 | 65 | 41 | 31 | 21 | 18 |

This leads us to the second research question, which is: What are the implications of the teacher rating of the learning tasks in the online mathematics textbook on improved performance in PCP? Drawing on the secondary data from the CAPS document, and from our data from the teachers’ ratings of tasks, we can only offer two reflections on this question. First, the exceptionally low performance in probability in 2020 by Grade 12 learners partly speaks to the learning difficulty that learners could have faced during the COVID-19 closure of schools, but this observation has no direct link to our current data on teacher rating. The apparently favourable teacher rating of the online tasks is probably an indication of the confidence that teachers had (or still have) in the tasks. However, the learner performance as shown in

The study was carried out during the restrictions due to COVID-19, where physical contacts were restricted as recommended by the health authorities to keep individuals and the public safe from contracting the disease. Communication during the data gathering process depended mainly on email with an attached questionnaire for the teachers, and follow-up phone calls. We contacted seven PCP teachers, but in the end only five teachers returned the questionnaire. Although only five teachers responded, percentage wise, it still represented a reasonable percentage considering that our initial target was seven senior teachers of probability at Grade 12. We were not able to observe the teaching of PCP in the classrooms for the same reason explained above. Finally, this study focused only on the learning tasks that are available to learners in the online textbook, so we missed the teaching tasks and the nuances that the teachers incorporate in their actual lessons. Obviously, we also missed observing the tasks that learners implement in their learning in the classrooms (Stein et al.,

The study opened our eyes to the challenges in the teaching of probability that we only have been hearing about but have not investigated for ourselves. This study suggests that teachers’ rating of tasks does not count until reflected in learner output in terms of learners’ performance in the tasks. It can also be argued that learners’ rating of learning tasks should precede the teachers’ rating. In other words, teachers should rely on, and respond to, the learners’ rating of learning tasks. One direct indicator is the learner scores in the tasks that are assigned to them. We recommend empirical classroom-based studies that support teachers with different ways of teaching PCP. One possibility is writing PCP teaching support materials that complement the online materials and the CAPS-recommended materials, focusing on the understanding of meanings in PCP, and their applications.

G.E. would like to thank Wits University, Faculty of Humanities, for the small research grant for this report.

The authors have declared that no competing interests exist.

S.M. compiled the data and wrote the first draft. G.E. added more research information on the draft, wrote and edited the final manuscript.

This article followed all ethical standards for research without direct contact with human or animal subjects.

G.E. received project fund number GEKL020 from the Faculty of Humanities Research Grant.

Data sharing is not applicable as no new data were created.

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.