There is an articulation gap for many students between the literacy practices developed at school and those demanded by higher education. While the school sector is often well attuned to the school-leaving assessments, it may not be as aware of the implicit quantitative literacy (QL) demands placed on students in higher education. The National Benchmark Test (NBT) in QL provides diagnostic information to inform teaching and learning. The performance of a large sample of school-leavers who wrote the NBT QL test was investigated (1) to demonstrate how school-leavers performed on this QL test, (2) to explore the relationship between performance on this test and on cognate school-leaving subjects and (3) to provide school teachers and curriculum advisors with a sense of the QL demands made on their students. Descriptive statistics were used to describe performance and linear regression to explore the relationships between performance in the NBT QL test and on the school subjects Mathematics and Mathematical Literacy. Only 13% of the NBT QL scores in the sample were classified as proficient and the majority of school-leavers would need support to cope with the QL demands of higher education. The results in neither Mathematics nor Mathematical Literacy were good predictors of performance on the NBT QL test. Examination of performance on selected individual items revealed that many students have difficulty with quantitative language and with interpreting data in tables. Given that QL is bound to context, it is important that teachers develop QL practices within their disciplinary contexts.

Many students leaving the school system in South Africa, while prepared for leaving school, are to some extent unprepared for higher education. There is therefore great inequality in the experiences of and outcomes for students in the higher education sector. According to Scott, Yeld and Hendry (

The real key to whether a student will pass or fail relates to the literacy practices she brings with her to the University from her school and home environments, and the extent to which these have commonalities with the literacy practices of her chosen discipline. (McKenna,

The importance of quantitative literacy (QL) for higher education is widely recognised (see for example, Steen,

One of the goals of the NBTP is to provide lecturers and curriculum developers in higher education with information about the capabilities of students ‘to inform the nature of foundation courses and curriculum responsiveness’ (Griesel,

A second goal of the NBTP is to ‘assess the relationship between higher education entry level requirements and school-level exit outcomes’ (Griesel,

Since QL involves dealing with and communicating quantitative information in various academic contexts, it is not only in the school subjects of Mathematical Literacy and Mathematics that QL is developed. Teachers in many other subjects, for example Geography, the sciences or Economics, have a very important role to play in developing this vital literacy by, for example, expecting learners to interpret a variety of representations of quantitative information, by using (and expecting learners to use) correct and precise language when expressing quantitative ideas and by assessing learners’ ability to practise and express quantitative reasoning in context.

In this article, we provide an overview of the results of the NBTP QL test written by a large sample of successful school-leavers across South Africa who were intending to apply to enter higher education institutions in 2015. These learners all achieved NSC results that would allow them to enter higher education. We also report on the associations between performance on the NBT QL test and the NSC subjects Mathematics and Mathematical Literacy. We further analyse the performance of these students on subgroups of items and on a selected number of key items. This analysis is not intended to give a comprehensive exposition of the QL competencies relevant to higher education, but is intended to alert school teachers and curriculum specialists to the requirements of higher education and to provide some examples of areas where lack of student competence will have detrimental effects on their success in higher education, if not addressed.

Higher Education South Africa commissioned the National Benchmarks Tests Project in 2005 with its main aim being to assess the academic proficiency of prospective students wishing to enter higher education. The tests assess proficiency in QL, as well as academic literacy (AL) for all students and mathematics (MAT) for those students intending to enter courses or programmes that have a significant mathematical component. These tests are designed by academics in higher education to provide information complementary to that provided by the NSC, to assist with selection and placement of students into appropriate courses and programmes. Another function of the NBTP is to provide information to inform curriculum development in higher education.

The NBTs provide a measure of students’ readiness for higher education and the competencies that are assessed in the NBTs are regarded as key areas in which students entering higher education should have minimum levels of proficiency. The content and competencies assessed in each domain are described by Griesel (

In particular, the NBTP QL test aims to measure the levels of proficiency in QL of school-leavers who are aspiring to enter higher education. The construct informing the design of this test is outlined by Frith and Prince (

In practical terms the NBTP QL test assesses students’ ability to competently interpret and reason with quantitative information that is presented in a variety of modes. For example, the test specifications include that they must understand and use a range of quantitative terms and phrases, read and interpret tables, graphs, charts, diagrams and texts and integrate information from different sources. The test also assesses the ability to apply quantitative procedures in various situations, to do simple calculations and estimations which may involve multiple steps and to formulate and apply simple formulae. Students are also required to identify trends and patterns in various situations, interpret two-dimensional representations of three-dimensional structures and to reason logically. The questions are designed to assess QL practices and do not assume that students have the knowledge of any particular school subject.

In this section, we will outline the theoretical considerations about the nature of QL that underpin the construct of the NBTP QL test. There has been an ongoing debate about what constitutes QL, especially in England and Australia (where it is usually referred to as ‘numeracy’) and in the United States (where it is usually called ‘quantitative literacy’). One aspect of this debate is to do with the relationship of QL with mathematics. Hughes-Hallett (

Mathematics focuses on climbing the ladder of abstraction while quantitative literacy clings to context. … Mathematics is about general principles that can be applied in a range of contexts; quantitative literacy is about seeing every context through a quantitative lens. (p. 94)

Some authors prefer to conceptualise QL as a social practice (Street,

At the very least then, the definitions garnered from this debate would agree that numeracy is to do with ‘using maths in context’ and that to be numerate is to have the ‘capacity to use maths effectively in context’. (Johnston,

It follows that QL cannot be taught as a generic skill and that learned rules will not be sufficient to enable the solution of QL problems. Thus, in almost all of the questions in the NBTP QL test, writers have to apply quantitative methods and reasoning within a realistic (mostly relevant to higher education) context. There is a wide range of both the contexts used for the items and the kinds of competencies required by them, in order that the test has both face validity and relevance for all disciplines in higher education.

The definition of QL that is the foundation of the construct of the NBTP QL test, is strongly influenced by the definition of numerate behaviour underlying the assessment of numeracy in the Adult Literacy and Lifeskills (ALL) Survey (Gal, Van Groenestijn, Manly, Schmitt & Tout,

Quantitative literacy is the ability to manage situations or solve problems in practice, and involves responding to quantitative (mathematical and statistical) information that may be presented verbally, graphically, in tabular or symbolic form; it requires the activation of a range of enabling knowledge, behaviours and processes and it can be observed when it is expressed in the form of a communication, in written, oral or visual mode. (Frith & Prince,

The construct informing the test design is based on the idea that each item can be described in terms of three dimensions of what it assesses: the main mathematical and statistical ideas, the underlying reasoning and behaviours (competencies) and the level of cognitive complexity. The construct does not specify the contexts for the items, but rather that there is a range of different kinds of contexts. This is necessary for face validity and because familiarity (or unfamiliarity) of the context could affect the manner in which a candidate responds to it.

With the introduction of the NSC in 2008, one national set of Grade 12 examination question papers was introduced. After a review of the NSC curricula, the Curriculum and Assessment Policy Statement (CAPS) was introduced and implemented in 2012 in Grade 10, with this cohort being the first to write the NSC examinations based on the CAPS in 2014.

The scoring of the NSC assessments, administered by the Department of Basic Education (DBE), is norm referenced and therefore the rating codes associated with them cannot easily be used to assess whether candidates meet a certain standard in a subject or domain. For the NSC, the final subject score is made up of the course mark and the examination mark and then the scores are ‘standardised’ or ‘normed’ to the five-year rolling average score for each subject. While a candidate may perform well compared to the norm, they may still fail to meet a particular standard in the domain being tested.

The achievement scale for NSC subjects is shown in

The achievement scale for the National Senior Certificate.

Rating code | Description | Score |
---|---|---|

7 | Outstanding achievement | 80–100 |

6 | Meritorious achievement | 70–79 |

5 | Substantial achievement | 60–69 |

4 | Adequate achievement | 50–59 |

3 | Moderate achievement | 40–49 |

2 | Elementary achievement | 30–39 |

1 | Not achieved | 0–29 |

On completing the NSC, a candidate can qualify for higher certificate, diploma or degree study. The criteria in

Criteria for higher certificate, diploma and degree study.

Qualification | Minimum entry requirement |
---|---|

Higher certificate | Pass NSC with at least rating of 2 (30–39%) for the language of learning and teaching of higher education institution. |

Diploma | Pass NSC with an achievement rating of 3 (40–49%) or better in four subjects. At least rating of 2 (30–39%) for the language of learning and teaching of the higher education institution. |

Bachelor degree | Pass NSC with an achievement rating of 4 (50–59%) or better in four subjects from the designated list. At least rating of 2 (30–39%) for the language of learning and teaching of the higher education institution. |

NSC, National Senior Certificate.

All NSC candidates must write the examinations for either Mathematics or Mathematical Literacy, which are both cognate with, but not the same as QL, as can be seen from their descriptions in the NCS CAPS documents.

The NSCS CAPS document for the subject Mathematics defines Mathematics as:

a language that makes use of symbols and notations for describing numerical, geometric and graphical relationships. It is a human activity that involves observing, representing and investigating patterns and qualitative relationships in physical and social phenomena and between mathematical objects themselves. (DBE,

The CAPS document claims that studying Mathematics will develop a student’s ability to think logically, critically and creatively and to be able to solve problems, in order to obtain a better understanding of the world around us. This focus on problem-solving and critical thinking in order to understand real-world phenomena has strong similarities with the definition of QL, but the main focus of the subject is in fact on learning the discipline of mathematics itself in order to ensure ‘access to an extended study of the mathematical sciences and a variety of career paths’.

On the other hand, the NCS CAPS for Mathematical Literacy states that the competencies it develops should:

allow individuals to make sense of, participate in and contribute to the twenty-first century world – a world characterised by numbers, numerically based arguments and data represented and misrepresented in a number of different ways. (DBE,

It suggests that these competencies, which include the ability to reason, solve problems, interpret information and use technology, should be developed by exposing learners to both elementary mathematical content and authentic real-life contexts. This exposure is intended to enable the learner to be a ‘self-managing person, a contributing worker and a participating citizen in a developing democracy’ and an ‘astute consumer of the mathematics reflected in the media’. The emphasis on using mathematical knowledge and skills in context is what makes this subject similar to QL, but in higher education the contexts are academic disciplinary contexts, not necessarily everyday life-related contexts, as emphasised in the NSC CAPS document for Mathematical Literacy.

This brief overview of the NSC Mathematics and Mathematical Literacy subjects and the method of assessment used for the NSC is important in order to reveal the complementary nature of the information derived from standardised benchmark or criterion-referenced assessments, such as the NBTs and a norm referenced assessment, such as the NSC. This complementary information about student competence is particularly useful for making decisions about whether students should be placed in extended or regular degree curriculum structures and for providing information to inform teaching and learning in higher education.

The NBT QL test results were analysed for a large sample (

There are 50 multiple choice items in the QL test, which are selected in accordance with the specification table (Frith & Prince,

The NBTs are administered at test centres under standardised conditions by specially trained invigilators. The items in the QL test comprise two out of the seven sections of the NBTP Academic and Quantitative Literacy test which is administered in paper and pencil format. Thirty minutes is allocated for the completion of each of the 25-item sections that make up the QL test. Calculators are not used, but students are only required to calculate with simple numbers, for example, with fractions that can easily be simplified by cancellation. Many questions can be answered by estimation.

Students writing the tests record their responses on mark-reading (bubble answer) sheets which are then scanned using optical scanner technology. Responses are dichotomised (either 1 for right or 0 for wrong). The unidimensional three parameter Item Response Theory (IRT) model (Yen & Fitzpatrick,

National Benchmark Tests Project quantitative literacy test benchmarks for degree study, set in 2012.

Performance band | Score range (%) | Description |
---|---|---|

Proficient | 70–100 | Future academic performance is unlikely to be adversely affected by weaknesses in QL. If admitted, students may be placed into regular programmes of study. |

Intermediate | 38–69 | Academic progress is likely to be adversely affected by QL challenges. These students’ needs should be addressed through appropriate provision (e.g. foundation, extended or augmented programmes, special support provision). |

Basic | 0–37 | It is highly likely that students will not cope with degree-level study without extensive and long-term support (e.g. non-credit preparatory courses, special support provision or Further Education and Training provision, i.e. training after school but not part of higher education). |

QL, quantitative literacy.

It has also been useful to differentiate between different levels of support that would be most appropriate for students with scores in the ‘Intermediate’ category and so this level is divided into ‘Upper Intermediate’ and ‘Lower Intermediate’ bands, as shown in

National Benchmark Tests Project quantitative literacy test degree Intermediate benchmarks and how they should be interpreted.

Intermediate performance band | Score range (%) | Description |
---|---|---|

Intermediate Upper | 54–69 | Students are likely to need complementary support (additional tutorials, workshops, augmented courses, language intensive work). |

Intermediate Lower | 38–53 | Students need to be placed in an extended programme. |

In addition to the analysis that is routinely done to generate scores for NBT results, for this article each item in the QL test was assigned to a discrete subgroup of items based on the main competency that the item was designed to assess, as follows:

Computing: interpreting problem statements and calculating or estimating (e.g. calculating areas or percentages).

Knowing: recalling simple facts and applying them (e.g. mean and median of a distribution).

Translating: identifying alternative representations (e.g. identifying the graphical representation of a relationship described verbally).

Using data: deriving information from data representations (e.g. reading required value/s off charts or tables).

Reasoning: reasoning and synthesising (e.g. identifying, reading off and calculating with more than one appropriate value from tables or charts; reasoning about rates).

Extrapolating: predicting and visualising (e.g. recognising patterns or predicting terms in a sequence; visualisation in three dimensions).

Performance on each subgroup was then calculated separately so as to obtain more detailed diagnostic information about students’ competencies. For this analysis, the scores used were not generated using the three parameter IRT model as for the overall NBT score, but merely using the percentage of the sample who answered each item correctly.

In order to examine how students responded to individual items, the proportion of students who chose each alternative answer was recorded. This was done for the whole cohort and separately for the students whose scores for the whole test were in each of the proficiency bands.

The statistical package R (R Core Team,

Ethical clearance for this research was obtained from the Research Ethics Committee of the Faculty of Higher Education Development at the University of Cape Town. This included approving the consent declaration signed by prospective students writing the NBT, which allows the use of their results for research purposes and assures anonymity in the use of these data.

The results are presented for a large sample of school-leavers who wrote one version of the NBTP QL test and who qualified to progress to some kind of higher education. These results are therefore not representative of all NSC candidates, but only of those who were qualified to enter higher education.

We first present some information about the characteristics of the students in the sample. We then present the overall distribution of scores on the entire QL test for the whole sample as well as for NSC Mathematics candidates and NSC Mathematical Literacy candidates separately. This will be followed by some results showing the relationship between performance on NSC Mathematics or Mathematical Literacy and on the NBT QL test. In addition, we report on the performance of the whole sample on subgroups of items in the NBT QL test, defined in terms of the competencies they assess. Finally, we discuss the performance of the students in the different proficiency bands on individual example items, in order to highlight some particular areas of difficulty they experienced.

Some characteristics of the students in the sample are shown in

Demographic characteristics of the total sample of National Benchmark Tests Project quantitative literacy test candidates (

Variable | Number | Percentage (%) |
---|---|---|

Male | 2 874 | 38.5 |

Female | 4 590 | 61.5 |

Total | 7 464 | 100.0 |

African | 4 554 | 61.0 |

Mixed race | 785 | 10.5 |

Indian | 841 | 11.3 |

White | 1 244 | 16.7 |

Not specified | 40 | 0.5 |

Total | 7 464 | 100.0 |

Afrikaans | 220 | 3.0 |

English | 2 808 | 38.6 |

isiNdebele | 78 | 1.1 |

isiXhosa | 1 019 | 13.7 |

isiZulu | 984 | 13.2 |

Sesotho | 457 | 6.1 |

Sesotho sa Leboa | 523 | 7.0 |

Setswana | 433 | 5.8 |

siSwati | 140 | 1.9 |

Tshivenda | 397 | 5.3 |

Xitsonga | 208 | 2.8 |

Other | 125 | 1.7 |

Total | 7 464 | 100.0 |

The data in

Distributions of National Benchmark Test quantitative literacy scores for National Senior Certificate Mathematics, National Senior Certificate Mathematical Literacy candidates and for the whole sample.

Descriptive statistics for the distributions of the quantitative literacy performance of the whole sample (

Variable | Number | Mean (%) | SD (%) | Minimum (%) | 1st quartile (%) | Median (%) | 3rd quartile (%) | Maximum (%) |
---|---|---|---|---|---|---|---|---|

Whole sample | 7 464 | 47.5 | 16.5 | 16.0 | 34.0 | 44.0 | 58.0 | 98.0 |

NSC Mathematics | 6 271 | 49.6 | 16.8 | 16.0 | 35.0 | 46.0 | 61.0 | 98.0 |

NSC Mathematical Literacy | 1 193 | 36.7 | 9.1 | 17.0 | 30.0 | 34.0 | 41.0 | 87.0 |

NSC, National senior certificate.

The most obvious observation to be made is that the NBT QL scores of the NSC Mathematical Literacy candidates are considerably lower than those of the NSC Mathematics candidates. The median score for the whole sample (44%) is approximately in the middle of the ‘Lower Intermediate’ proficiency band, while the median score for NSC Mathematics candidates is within the ‘Lower Intermediate’ band, but the median for NSC Mathematical Literacy candidates is well within the ‘Basic’ band. The third quartile for all the distributions illustrated in

Percentage distribution of National Benchmark Test quantitative literacy scores by proficiency category for the whole sample, National Senior Certificate Mathematics and Mathematical Literacy candidates.

Numbers and percentage distribution across quantitative literacy performance categories for the total sample and of subsets defined by National Senior Certificate mathematics subject written.

Variable | Whole sample (%) | Basic (%) | Intermediate Lower (%) | Intermediate Upper (%) | Proficient (%) |
---|---|---|---|---|---|

Whole sample | 7 464 (100.0) | 2 662 (35.7) | 2 472 (33.1) | 1 391 (18.6) | 939 (12.6) |

NSC Mathematics | 6 271 (100.0) | 1 895 (30.2) | 2 115 (33.7) | 1 330 (21.2) | 931 (14.8) |

NSC Mathematical Literacy | 1 193 (100.0) | 767 (64.3) | 357 (29.9) | 61 (5.1) | 8 (0.7) |

NSC, National Senior Certificate.

The majority of the scores are in the ‘Basic’ and ‘Lower Intermediate’ bands and less than 15% of the scores are classified as ‘Proficient’. The most striking difference is seen when comparing the distributions for those students who wrote NSC Mathematics and NSC Mathematical Literacy: the proportion of scores that are ‘Basic’ is more than twice as large for Mathematical Literacy as for Mathematics candidates (over 60% and about 30%, respectively). Less than 1% of the scores for Mathematical Literacy candidates fall in the ‘Proficient’ category. From this it is clear that the NSC Mathematical Literacy subject does not prepare students for the quantitative demands of higher education study. It cannot however be concluded that the somewhat better performance on the NBT QL test of students who did NSC Mathematics can necessarily be ascribed to their having taken this subject, as QL is also developed in other subjects such as in the physical, earth and life sciences.

In this section, we will further explore the relationship between performance in the NSC mathematical subjects and in the NBT QL test.

Distributions of National Senior Certificate Mathematics and National Senior Certificate Mathematical Literacy scores and of National Benchmark Test quantitative literacy scores for National Senior Certificate Mathematics candidates and National Senior Certificate Mathematical Literacy candidates.

Descriptive statistics for the National Senior Certificate Mathematics and Mathematical Literacy results for the whole sample (

Variable | Number | Mean (%) | SD (%) | Minimum (%) | 1st quartile (%) | Median (%) | 3rd quartile (%) | Maximum (%) |
---|---|---|---|---|---|---|---|---|

NSC Mathematics | 6 271 | 59.8 | 18.9 | 4.0 | 46.0 | 60.0 | 74.0 | 100.0 |

NSC Mathematical Literacy | 1 193 | 65.8 | 13.3 | 28.0 | 56.0 | 66.0 | 76.0 | 95.0 |

NSC, National Senior Certificate.

In

In

National Senior Certificate Mathematics and National Benchmark Test quantitative literacy scores (

Percentage distribution of National Benchmark Test quantitative literacy scores by proficiency band for different National Senior Certificate Mathematics levels achieved.

The same data (as in

However, it is most noteworthy that, even for those who obtained level 7 results for NSC Mathematics, less than 50% obtained a ‘Proficient’ NBT QL score, while the proportion of ‘Proficient’ scores for the level 6 (‘Meritorious achievement’) Mathematics results is even smaller, at just under 25%. Thus, based on the NBT QL scores, the majority of even the best NSC Mathematics performers will require some kind of additional support to cope with the quantitative demands of higher education study.

In

National Senior Certificate Mathematical Literacy and National Benchmark Test quantitative literacy scores (

Percentage distribution of National Benchmark Test quantitative literacy scores by proficiency band for different National Senior Certificate Mathematical Literacy levels achieved.

The proportions illustrated in

Each NBT QL item was assigned to a single subgroup based on a judgement of the main competency which the item was designed to assess. This results in six discrete subgroups of items, with the numbers of items in each subgroup as shown in

Distributions of the scores of the entire sample for each subgroup of items based on the main competency they assess.

Classification of the National Benchmark Test quantitative literacy items according to the main competence required.

Subgroup | Mathematical or statistical idea | Number of items |
---|---|---|

Extrapolating | Predicting and visualising | 5 |

Knowing | Recalling simple facts and applying them | 6 |

Using data | Deriving information from data representations | 15 |

Computing | Interpreting problem statements and calculating or estimating | 10 |

Translating | Identifying alternative representations | 8 |

Reasoning | Reasoning and synthesising | 6 |

In all subgroups except ‘extrapolating’, the upper quartile is at or below the top of the ‘Upper Intermediate’ band, and the median is at or below the top of the ‘Lower Intermediate’ band, indicating that at least three-quarters of the scores are not in the ‘Proficient’ band, and at least 50% are not in the ‘Upper Intermediate’ or ‘Proficient’ bands. The subgroups on which the performance was the weakest are ‘computing’ and ‘knowing’. For ‘computing’ the median is the lowest, at 40%, and about a quarter of the scores are below 30%. This reflects students’ poor number sense (as calculators were not used in the test session and many items required estimation) as well as difficulty with interpreting problem statements, especially in some cases where the context may have been unfamiliar. The strongest performance was on the subgroup ‘extrapolating’ which contains several items that require predicting future terms in a given sequence. These presumably draw on skills that are better developed at school. The median scores for ‘reasoning’, ‘knowing’, ‘translating’ and ‘using data’ are all about 50% and near the top of the ‘Lower Intermediate’ band.

In

Mean scores (and confidence intervals) for each subgroup of items based on the main competency they assess reported separately for the National Senior Certificate Mathematics and Mathematical Literacy candidates.

The mean scores for the subgroups for the NSC Mathematics candidates follow the same pattern that is seen in

The examples described in this section illustrate how a detailed examination of the proportions of students who chose different alternative answers in certain items can be used to gain rich insights into students’ abilities. In these items some of the alternative (incorrect) answers are obtained through applying common misconceptions or fallacious thinking, so examining the patterns of responses can be revealing. However, it is difficult to report this kind of information without also describing precisely the structure and content of the items, and for security reasons this is undesirable. Thus, we will try to illustrate how diagnostic information relevant to higher education studies is revealed, without publicising the actual test items.

A table from a reading that was given to first-year medical students and which contains both numbers and percentages is shown in

Homicide and suicide by gender and age (preliminary NMSS data, first quarter 1999).

Age ranges | Homicides |
Suicides |
||
---|---|---|---|---|

Males (%) | Females (%) | Males (%) | Females (%) | |

< 15 | 10 (76.9) | 3 (23.1) | 1 (50.0) | 1 (50.0) |

15–24 | 166 (91.2) | 16 (8.8) | 22 (91.7) | 2 (8.3) |

25–34 | 231 (91.3) | 22 (8.7) | 23 (85.2) | 4 (14.8) |

35–44 | 141 (82.9) | 29 (17.1) | 11 (68.8) | 5 (31.2) |

45–54 | 48 (84.2) | 9 (15.8) | 9 (81.8) | 2 (18.2) |

55–64 | 20 (87.0) | 3 (13.0) | 8 (100) | 0 (0.0) |

65+ | 3 (42.9) | 4 (17.1) | 5 (62.5) | 3 (37.5) |

For a question requiring interpretation of a percentage in a table, the proportions of students who chose each alternative answer, for the total cohort and for each proficiency band.

Less than half of all students could identify the correct description of the meaning of a particular percentage value in the table (alternative D). Only about 60% of those scoring overall in the ‘Upper Intermediate’ category and about 40% in the ‘Lower Intermediate’ category were able to do this.

In the bottom two categories more than half of the students chose alternative A (58% and 54% in the ‘Basic’ and ‘Lower Intermediate’ categories respectively). This was equivalent to their choosing ‘23.1% of the female homicides were under 15 years of age’ in the above example, showing that they identified the correct row and column headings, but could not identify what the ‘whole’ was that the percentages were a ‘part’ of.

Students in higher education will have to interpret tables of data and percentages in many disciplines and most lecturers will assume that they can understand these representations. The data in

^{2}) there are in 2 cm^{2} (with the fact that there are 10 mm in 1 cm given). The alternative answers were A 20, B 40, C 200 and D 400. Only one-third of all students and less than two-thirds of those with scores in the ‘Proficient’ category answered this correctly. Students with scores in the bottom two categories preferred alternative A (the answer 20), indicating that they treated all units mentioned as linear, ignoring the references to square units. Even those in the ‘Upper Intermediate’ category were almost as likely to choose alternative A as the right answer (between 35% and 40% of them). Alternative D (answer 400) was chosen by between 10% and 20% in all proficiency bands. In this case students were aware that squaring the numbers was appropriate, but applied it inappropriately to the value 2, as well as in order to convert the units. Similarly, alternative B (answer 40) was chosen by between 10% and 20% of the students with scores in the lower three proficiency bands, indicating that they were aware of the need for squaring, but not aware of which numbers to square in order to convert the units. These results indicate that students are unable to think flexibly about square units in the metric system, or that many of them do not read questions carefully (interpreting mm^{2} as mm and cm^{2} as cm).

For a question requiring converting to square units, the proportions of students who chose each alternative answer, for the total cohort and for each proficiency band.

The pattern of performance on this question shows that most students are not able to do a simple conversion from linear to square units, which is a competency that they would most likely be expected to have in many scientific and technical courses in higher education.

Given values in a chart for the proportion of the population in several provinces that is, say, over 15 years of age, as well as the values of the total population for those provinces, one might ask which province had the greatest number of people over 15 years of age. If in addition the proportions are all similar, but the total populations are significantly different, one can conclude that the province with the largest overall population also has the largest number of people over 15 years of age.

The results for a task similar to the one just described are shown in

For a question requiring proportional reasoning, the proportions of students who chose each alternative answer, for the total cohort and for each proficiency band.

The proportion of all students who could in fact answer correctly by using the proper reasoning about proportions was probably smaller than 40%, as it was possible to arrive at the correct answer by answering the question equivalent to ‘Which of the provinces had the greatest number of people?’ not ‘Which of the provinces had the greatest number of people

The pattern of performance on this task illustrates the difficulty that many people have with reasoning with proportions, understanding the distinction between absolute and relative quantities and the language used to make this distinction. These difficulties are further analysed by Frith and Lloyd (

Given data for the percentage distribution of the population in the provinces of South Africa and the proportion of the population that is over 15 years of age in each province (for example in a stacked bar chart), a question might ask what percentage of the total population is over 15 years of age and lives in a particular province. This requires combining the percentage of the population that lives in the given province and the percentage of that provincial population that is over 15 years old, by calculating a percentage of a percentage.

For a question requiring calculating a percentage of a percentage, the proportions of students who chose each alternative answer, for the total cohort and for each proficiency band.

The pattern of performance on this question illustrates the extent to which students struggle to interpret the precise language used to describe percentages and, as in the previous example, that many of them are unable to integrate information from two different data representations.

In this article, the results are presented for a large sample of school-leavers from across South Africa who wrote one version of the NBTP QL test in 2014 and who obtained NSC results that qualified them for entry into higher education in 2015. Nearly 70% of the scores for the test are in the lower two proficiency bands, with 36% in the ‘Basic’ band and 13% in the ‘Proficient’ band. This suggests that the majority of candidates aiming to enter higher education are in need of some kind of supplementary support for developing their QL, while more than a third will require extensive support. This support could be in the form of foundation courses, supplementary tutorials integrated into disciplinary curricula or online provision. Teachers and lecturers should be mindful of the assumptions they make about students’ QL competencies and should at all times attempt to make the implicit literacy practices of their disciplines more explicit.

Students who wrote NSC Mathematics perform considerably better on the NBT QL test than those who wrote NSC Mathematical Literacy. Nearly two-thirds of those who wrote NSC Mathematical Literacy have scores in the ‘Basic’ category, compared to 30% of the NSC Mathematics candidates. Less than 1% of those who wrote NSC Mathematical Literacy have a ‘Proficient’ score, indicating that in general the school subject Mathematical Literacy does not prepare students to cope with the QL demands of higher education.

Comparison of the NBT QL scores and the NSC Mathematics results reveals that the NSC Mathematics result is a very poor predictor of the QL score. For any particular NSC Mathematics score there is a large range of NBT QL scores, but this range is particularly large for the lower NSC Mathematics scores. In general, the performance on the NBT QL test is lower than for the NSC Mathematics. A similar weak relationship is observed when the NBT QL scores are compared to the NSC Mathematical Literacy scores. In addition, the difference in overall performance is much greater (with a median score for the NBT QL test approximately half of the median for NSC Mathematical Literacy).

The scores on subgroups of items classified according to the main competencies they were designed to assess are also considered. Three-quarters of the candidates score below the ‘Proficient’ level in all of these subgroups except the one mainly requiring recognising patterns or predicting terms in a sequence, a competency that is probably quite well developed in school mathematics. The weakest performance is in the subgroup that requires candidates to interpret problem statements and calculate or estimate answers. This could reflect difficulties experienced with interpreting the problem statements (the majority of students are not first-language English speakers), but probably also points to students’ poor number sense and dependence on calculators, as most questions in this subgroup require estimation. This kind of analysis of the scores on subgroups of items provides diagnostic information that can inform the design of interventions to address the specific needs of students in terms of the competencies to be developed. Due to the context-bound nature of QL, teachers and lecturers need to use this information to develop their own curriculum solutions in their own disciplinary and classroom contexts. Where large numbers of students are not first-language English speakers, and the possibility exists that language difficulties contributed to poor performance, it should possibly be assumed that some form of AL intervention will also be required in order to help students cope with the language demands of their quantitative studies.

In the last part of the article, close examination of patterns of performance on examples of individual items are presented to illustrate how diagnostic information can be derived from this kind of analysis. These examples show that many students have difficulty with quantitative language, especially the language used to describe percentages and absolute and relative quantities. Students generally also have difficulties with interpreting data in tables, especially percentages, and many struggle with proportional reasoning. These are all examples of concepts and competencies essential for practising QL in many academic disciplines. In a society where many arguments and the understanding of many situations and problems draw on quantitative data, competencies like these are also essential for effective and critical citizenship. It would therefore be essential to ensure that students develop these competencies in all appropriate contexts in school and also as graduate attributes in higher education.

These results stress that there is a lack of alignment between the exit level outcomes from schooling and the expectations of higher education in terms of QL. Higher education institutions need to recognise this fact and modify curricula accordingly. At the same time it would be productive if the school sector could give more attention to the development of this vital literacy, which is needed not only for higher education, but also for critical citizenship.

We thank the NBT project team at the Centre for Educational Testing for Access and Placement at the University of Cape Town who provided the opportunity to conduct this research, with the goal of contributing to the NBT project’s purpose of assessing the relationship between entry level proficiencies and school-level exit outcomes.

The authors declare that we have no financial or personal relationships that might have inappropriately influenced us in writing this article.

R.P. and V.F. contributed equally to the conceptualisation and execution of this research and the production of the manuscript.