The rational number knowledge of student teachers, in particular the equivalence of fractions, decimals, and percentages, and their comparison and ordering, is the focus of this article. An instrument comprising multiple choice, short answer and constructed response formats was designed to test conceptual and procedural understanding. Application of the Rasch model enables verification of whether the test content was consistent with the construct under investigation. The validation process was enabled by making explicit the expected responses according to the model versus actual responses by the students. The article shows where the Rasch model highlighted items that were consistent with the model and those that were not. Insights into both the construct and the instrument were gained. The test items showed good fit to the model; however, response dependency and high residual correlation within sets of items was detected. Strategies for resolving these issues are discussed in this article. We sought to answer the research question: to what extent does this test instrument provide valid information that can be used to inform teaching and learning of fractions? We were able to conclude that a refined instrument applied to first-year students at university provides useful information that can inform the teaching and learning of rational number concepts, a concept that runs through mathematics curricula from primary to university. Previously, most research on rational number concept has been conducted on young learners at school.

Venkat and Spaull (

Application of the Rasch model enabled a finer analysis of the test construct, the individual item and person measures, and the overall test functioning through making explicit the expected responses according to the model versus the actual responses by the students. In addition, the test as a whole was investigated for properties that are requirements of valid measurement such as the local independence where each item functions independently of each of the other items.

This article reports on how students displayed gaps in their rational number knowledge base but focuses primarily on the validation of the instrument. The following questions are answered:

To what extent does the test provide valid measures of student proficiency?

How might the test be improved for greater efficiency of administration, and greater validity for estimating student proficiency?

The aims of the immediate analyses were to:

Evaluate the assessment tool in terms of fit to the model, both item and person fit, thereby checking whether the tool was appropriate for this student cohort.

Provide detailed descriptions of selected items in relation to the students taking the test.

The validity and reliability of the assessment tool were analysed through the Rasch model incorporating both the dichotomous and partial credit model using Rasch Unidimensional Measurment Models (RUMM) software (see Andrich, Sheridan, & Luo,

In an attempt to clarify the assessment, how it was conducted and its purpose, we provide the justification for the exercise. It was critical to ensure that certain conditions were satisfied in order to safeguard the effectiveness of the assessment as well as the validity of the test items. Stiggins and Chappuis (

The learning and teaching of rational number concepts is particularly complex. The representation of a fraction

Besides the features mentioned previously, there are different representational systems for rational numbers, namely common fractions, decimal fractions and percentages. While there is equivalence across the three systems within the triad fractions-decimals-percentages, this equivalence is not obvious at face value unless the student has understood the organising principles of each system. For instance, the denominator of a percentage representation is always 100, for common fractions the choice of denominator is infinite, while for decimal fractions, the denominator is 1 (one).

The apparent simplicity of the percentage because of its everyday use belies the complexity of this ‘privileged proportion’ (Parker & Leinhardt,

Hiebert and Lefevre (

Stacey et al. (

Research shows that in most cases both teachers and learners appear to have instrumental understanding of fractions, but do not really know why the procedures are used (Post, Harel, Behr, & Lesh,

The rich theorising of and research into rational numbers provides the theoretical base for the assessment instrument, which therefore meets the requirement for measurement to define clearly what is to be tested (Wright & Stone,

The primary study (Maseko,

The assessment tool was administered to the whole population of students that were admitted into the Foundation Phase teacher training programme (

The main research study comprised five conceptual categories that facilitated the analyses. The categories are understanding rational number concepts: definitions and conversions (14 items); manipulating symbols (operations) (17 items); comparing and sequencing rational numbers (15 items); alternate forms of rational number representation (35 items); as well as solving mathematical word problems with rational number elements (12 items). The items were drawn from selected projects, for example ‘the rational number project’ (Cramer, Behr, Post, & Lesh,

The items were primarily informed by the conceptual categories above, and could be identified according to the following requirements:

The items demanded a demonstration of procedural as well as conceptual understanding.

The items included fraction, decimal and percentage representations.

Items were generated with the specific purpose of evoking misconceptions.

The items were comprehensive, covering most concepts and sub-concepts within the three representational systems – fractions, decimal fractions and percentages.

The format of the test item types included multiple choice items, short answer, as well as extended response items.

The reason for such a comprehensive selection of items was that the lecturers needed to identify the many difficulties and misconceptions the students could bring into their first semester mathematics class. A range of difficulty that would include learners of current low proficiency, and high proficiency, was also required. Also, at the time of setting the items, the instructors were not sure from which categories the difficulties would emerge.

The Rasch model was applied in this study in order to either confirm or challenge the theoretical base, to check the validity of the instrument, and to measure the students’ cognition of rational number concepts. The hypothesis was that the assessment tool would function according to measurement principles. The Rasch model provided information of where the item functioning and student responses were unexpected. Possible explanations could then be inferred, and presented, as well as provide some indications for the refinement of the test instrument.

There are other theories developed that can be used to validate and authenticate tests, such as Classical Test Theory (CTT) (Treagust, Chittleborough, & Mamiala,

This study has been cleared by the University of Johannesburg Ethics Committee, with the ethical clearance number SEM 1 2018-021.

The first analysis showed the test instrument to have a sound conceptual base and to be well targeted to the cohort, with a range of items, such that the students of current lower proficiency could answer a set of questions with relative ease, while students of high proficiency would experience some challenging items.

Summary statistics of fractions, decimals and percentages.

Statistic | Value |
---|---|

Item mean | 0.0000 |

Item standard deviation | 1.6000 |

Person mean | –0.4238 |

Person standard deviation | 0.9686 |

Person separation index | 0.9114 |

Power of analysis of fit | Excellent |

As observed in the person-item map

Rasch model – Person-item original map.

Easier items are located at the lower end of the map (Item 65 and Item 66), while the difficult items are located at the higher end (Item 27 and Item 28). Similarly, learners of high proficiency are located higher on the map, 2.903 and 1.733, while learners of low proficiency are located at −2.159 and −2.143. The mathematical structure of the Rasch model is such that where a person’s proficiency location is aligned with an item difficulty location, an individual of that proficiency level has a 50% probability of answering an item of that difficulty level correctly (Rasch,

The individual items when constructed were initially reviewed by the lecturers. The application of the Rasch model provided empirical output calibrating a relative location and giving the probability that a person located at a certain proficiency location will get the item correct within the instrument.

Item 63 (43. Fraction form of 0.21) at position −0.646 is shown on the category probability curve (

Item 63 – category probability curve.

In

When an item is difficult or easy for the students, the curves show a shift of the meeting point away from the zero position (0) on the

Item 58 – difficult category probability curve.

Item 39 – easy category probability curve.

Very few students are to the right of position +3, implying that it was only students located at +3, or higher, that had a greater than 50% probability of answering the item correctly.

Item 39 (

In this next discussion we compare two students, one located at +3 and another located at −3, on Item 63 (

Item 63 – category probability curve.

In summary, applying the Rasch model to a data set is essentially testing a hypothesis that invariant measurement has been achieved. Where there are anomalies, the researcher is required to investigate the threat to valid measurement. The model enables the researchers to identify the items that did not contribute to the information being sought or those items that were deemed faulty in some respect. Likewise, where students’ responses to the question were unexpected the researchers were also alerted. The Rasch model is to some extent premised on the Guttman pattern, which postulates that in addition to some difficult questions, a person of greater proficiency should answer all the items correctly that a person of lower proficiency answers correctly. Likewise, easier items should be answered correctly by low proficiency learners, and also by moderate proficiency and higher proficiency learners. While a strict Guttman pattern is not possible in practice, the principle is a good one (Dunne, Long, Craig, & Venter,

We briefly report on six students against four questions close enough to their locations to illustrate the relationship of person proficiency to item difficulty as seen through the Guttman pattern model.

The student of low proficiency (A, location −2.159) struggled with the range of items that included the easiest of the items. The other student categorised as of low proficiency (B, location −2.143), offered no response to these particular items. From the person-item map, we would expect students at these locations to have a 50% chance of answering correctly, meaning that if there were 100 students at that location approximately 50 could have answered the items correctly.

Of the two students in the moderate category, one of the students (C, location 0.003) did not attempt the easiest item (location −2.234) (missing response), while the other student (D, location 0.029) answered this item far below his location correctly. The next two items which were above the two students’ locations were either not answered or answered incorrectly.

The two students located in the high proficiency category are located at 1.733 logits (E) and 2.903 logits (F), more than a logit apart. We therefore deal with them separately. Student E answered the easiest item correctly and this was to be expected; however, the next easiest item was answered incorrectly. In theory the student should have had a greater than 50% correct response. The difficulty of the third item is aligned with the proficiency of Learner E. In theory Learner E has a 50% chance of answering Item 57 correctly. Item 58 has a greater difficulty by a large margin. One would expect the student to perhaps get this incorrect.

Student F (location 2.903) answered three items correctly but was not able to answer Item 58 (location 2.903) correctly. According to the model the student had a 50% probability of answering this item correctly, as it is located at the same point on the scale. In the case of the most difficult item, Item 58, the requirement was to make decisions on converting the existing form before comparing and sorting the elements in ascending order. The cognitive demand required the students to connect their knowledge and make decisions in the process of working out the solution.

Six students, low, moderate and high proficiency vs performance on four items.

Weakest to strongest student | Low proficiency |
Moderate proficiency |
High proficiency |
|||
---|---|---|---|---|---|---|

Easiest to difficult item | –2.159 (A) | –2.143 (B) | 0.003 (C) | 0.029 (D) | 1.733 (E) | 2.903 (F) |

0 | MR | MR | 1 | 1 | 1 | |

0 | MR | 0 | 1 | 0 | 1 | |

0 | MR | MR | 0 | 1 | 1 | |

0 | MR | MR | 0 | 0 | 0 |

MR, missing response.

It was noted in the first analysis that there were two items that did not function as expected. These two items were removed from this analysis, although for future testing they may be refined. One multiple choice item was removed due to an error. The second item, Question 8A, was revised as shown below and was reserved for the next cycle. Item 88 (Question 8A) was found to be a misfit as the grammatical representation of the mathematical idea is confusing. The original and possible revised versions are briefly discussed below.

1.5 150%

The responses to item 8A produced the distribution displayed in

Item 88 (8A) – item characteristic curve.

The black dots represent the means of the 5 class intervals into which the students were divided. The allocation to class intervals is decided by the researcher. The black dots representing students’ mean responses did not follow the expected pattern according to the model. The expectation is that students of lower ability will be less likely to answer an item correctly than those of higher ability. The analysis revealed that learners of lower proficiency (four × marks left of 0 logits) on the test as a whole performed relatively higher than the students of higher proficiency (one × mark at about 1 logits). This anomaly was investigated, and it was found that the grammar and length of the instructions appeared to have interfered with the understanding of the question. For the next three items in Question 8 the instructions did not seem to mislead the students. When the instructions were revised and reduced to ‘Use

A further check on the validity of the test required an investigation of local independence. In any test, one expects that each item would contribute some information to the test construct (Andrich & Kreiner,

In this instrument analysis, we checked the residual correlations of the items and found high correlations, both positively correlated sets of items and negative correlations across some items. The implications of such a threat to local independence is that there are many items contributing the same information, as in a high positive correlation, and those with a negative correlation are ‘pulling in the other direction’. A resolution of this threat is to remove the items that seem to test the same thing or create subtests of items that are highly correlated, by investigating both the item context and the statistics it conveys.

In a second round, eight items were removed due to redundancy. In order to resolve response dependency, 18 subtests were created. These subtests were then checked for ordered or disordered thresholds. For illustrative purposes four sets of items are discussed.

Question 6: Item 6a ‘Draw a representation of fraction

On investigating the subtest, Question 6 (combined a and b), which required students to both draw a representation and explain the meaning of

(a) Subtest 3 (question 6), (b) Subtest 3 (question 6) – re-scored.

Question 27 required the students to provide the fraction and percentage form for 0.75 as individual responses, but the correct answer depended on whether the student knew how to perform the conversions to both forms of fractions from the decimal form, that is, fraction and percentage form. This was the second set of items observed to be highly correlated and was subsumed into a subtest. For this subtest (see

Subtest 12 (question 27A and 27B).

The next subtest was created by subsuming four items into one set. The four sections of the question asked similar questions, which were to convert from an improper fraction to a mixed fraction. These four items – 11A =

(a) Subtest 5 (question 11) – original score category probability curve, (b) Subtest 5 (question 11) – re-scored category probability curve.

The final subtest was made up of four different question items, where the requirement was to order a combination of the fractions-decimals-percentages representations in ascending or descending order (See

Final subtest.

Here it appeared that although these items were highly correlated, they increased in complexity. This subset functioned as expected in that the categories mark increase in proficiency with a clearer differentiated distribution of the curves (

Subtest 17 (questions 39–42) – re-scored category probability curve.

As exhibited in the examples above, the investigation of specific subsets, from both a conceptual perspective and a statistical perspective, was conducted in order to ascertain which items could reasonably be subsumed into subtests. The subtests that functioned as expected were retained, but for those whose categories were for some conceptual reason not functioning according to measurement principles, the rescoring of the subtests items was implemented. The process reported in this article works together with a qualitative investigation that was done in the main study, and also formed part of improving the functioning of the instrument (Dunne et al.,

The outcome, after this final analysis, was a test with 50 items, including both dichotomous and polytomous items, 22 of which were multiple choice format and 28 constructed response format.

Revised person-item map.

A comparison of the initial and final analyses.

Initial analysis | Final analysis | ||
---|---|---|---|

Item mean | 0.0000 | Item mean | 0.0000 |

Item standard deviation | 1.6302 | Item standard deviation | 1.4933 |

Person mean | −0.4172 | Person mean | 0.0990 |

Person standard deviation | 0.9442 | Person standard deviation | 0.7580 |

Person separation index | 0.9088 | Person separation index | 0.8606 |

Power of analysis of fit | Excellent | Power of analysis of fit | Excellent |

As stated in the introduction, this article forms part of a larger study into the student understanding of rational number, fractions, decimals and percent. The purpose of the investigation was to gather information about the cohort entering the Foundation Phase teacher development on working with rational numbers, especially fractions-decimals-percentage. This article reported on how the instrument was functioning to assess their knowledge level of work done at school. The assessment tool covered understanding rational number concepts, manipulating symbols (operations), comparing and sequencing rational numbers, alternate forms of rational number representation, as well as solving mathematical word problems with rational number elements. It is clear that the number of items does not impact the quality of the test. Beyond a certain amount, some of the items might be redundant. One has to check if the test instrument as a whole is fit for purpose. Beyond the total score obtained by each student in the test, the Rasch model indicates a position on a unidimensional scale where the student’s proficiency level is differentiated. The power and usefulness of the Rasch model is that it supports the professional judgement of the subject expert in making decisions about the validity of items (Smith & Smith,

The Rasch model was applied in this study in order to confirm or challenge the theoretical base, to check the validity of the instrument and to quantify the students’ cognition of rational number concepts.

The application of the Rasch measurement model enabled checking whether the test content was consistent with the construct under investigation, and supported expectations of a sharper understanding of these students in terms of proficiency level within a set of items in the test. The outcome showed the data to fit the model, the person separation index was high, and the target was appropriate, thereby confirming the theoretical work that supported the design of the test.

This work is based on a doctoral study for which the University of Johannesburg has provided financial support.

The opinions, findings, conclusions and recommendations expressed in this manuscript are those of the authors and do not necessarily reflect the views of the University of Johannesburg.

The authors declare that they have no personal relationships that may have inappropriately influenced the writing of this manuscript.

This study is based on the submitted doctoral research of J.M. The conceptualisation of this manuscript was done by all three authors. K.L. provided input on the literature and methodology. C.L. assisted with the Rasch analysis. The final product received input from all three authors.

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Data sharing is not applicable to this article as no new data were created or analysed in this study.

The opinions, findings, conclusions and recommendations expressed in this manuscript are those of the authors and do not necessarily reflect the views of the University of Johannesburg.