This article reports on the development of a custom-made questionnaire. The questionnaire was developed with the aim to compile guidelines for the professional development (PD) of mathematics teachers for the pedagogical use of information and communication technology (ICT) integration in teaching and learning. During the standardisation and validation of the questionnaire it was distributed to 179 schools and 300 teachers in eight educational management districts in the Western Cape, South Africa. The extracted factors had a reliability level higher than 0.8, which indicates the items in the questionnaire are significant to address the research problem and the questionnaire is valid for ICT PD.
Information and communication technology (ICT) is increasingly becoming embedded in teaching and learning; it is thus imperative that teachers integrate the underpinning concepts and skills of teaching with technology into sound classroom pedagogy. Cassim (
The question that guided this research was: What are the processes to develop, validate and standardise a custom-made instrument for PD of mathematics teachers towards ICT integration? This question is complex and relates to the two separate social realities of objectivist and subjectivist assumptions (Burrell & Morgan,
Two phases of the multi-mode research.
Paradigm characteristics | Phase I (Interpretivist) | Phase II (Structuralist) |
---|---|---|
Ontology | Understand the social phenomena through the eyes of various participants in the social activity (Patton, |
Rigid and actual nature of reality which exists outside the human mind (Mertens, |
Epistemology | Derive meaning from symbols and the interpretive processes when they share and interact in their environment (Merriam, |
Know the world they live in, change the world, identify the structures in society and analyse the structures in society (Mack, |
Human nature | Expert authors in PD, mathematics, ICT and ODL identified by means of a systematic literature review | Mathematics teachers in eight educational management districts in the Western Cape |
Methodology | Qualitative | Quantitative |
Document analysis | Questionnaire dissemination |
PD, professional development; ODL, open distance learning; ICT, information and communication technology.
The research therefore comprised a fully multimode equal status design of the qualitative and quantitative phases (Leech & Onwuegbuzie,
Survey research comprises the use of survey interviews or questionnaires. A prerequisite for the design of a good survey is deciding what is to be measured (Fowler,
Surveys generally collect trustworthy data relating to a large number of variables (McMillan & Schumacher,
The researchers, to no avail, searched for an existing questionnaire which could be used or modified to measure the envisaged variables of this study. Choosing a suitable questionnaire is a complex process; even more aspects come into play when designing and developing a survey (Creswell,
The researchers followed the 14 stages of Cohen et al. (
Stages of questionnaire development.
Stages | Focus of each stage |
---|---|
Stage 1 | Define the aim |
Stage 2 | Determine which type of survey to use |
Stage 3 | Formulate research question |
Stage 4 | Identify the aspects on which to focus |
Stage 5 | Clarify what information is needed to address the research question |
Stage 6 | Determine the target population |
Stage 7 | Compile questions and the metrics |
Stage 8 | Make the instrument |
Stage 9 | Determine data collection strategies |
Stage 10 | Pilot the instrument |
Stage 11 | Prepare for the data collection |
Stage 12 | Collect the data |
Stage 13 | Analyse the data |
Stage 14 | Report the findings |
A questionnaire aims to gather data from a large number of respondents on the same phenomena with the intention of describing the nature of the current circumstances or to determine whether relationship exists between specific events (Cohen et al.,
Mainly two types of questionnaires exist: cross-sectional (obtain data on present trends, attitudes and beliefs of the participants; Creswell,
The design of a questionnaire involves the translation of the research question and objectives into information embedded in the questions. It is therefore essential that the research question should be linked to the rationale of the research. The question for this study was:
Four key variables (governance, school environment, PD and ODL) emerged as themes from the document analysis from Phase I (adjustable exploration). Each of these variables represents a section of the questionnaire with multiple sub-questions; two variables (PD and ODL) were grouped into a single section with multiple sub-questions (Neuman,
In order to address the key aspects in the research, certain categories of questions should be posed. Neuman (
Sampling is an integral part in the survey approach (Cohen et al.,
The target population is a specified large group of many subjects from which a researcher draws a sample (Neuman,
The systematic literature analysis revealed various ICT projects in South Africa since the launch of the e-education policy (DOE,
The researchers considered five key factors when selecting the sample: (1) the sample size, (2) the symbolisation and limitation of the sample, (3) access to the sample, (4) the use of a sample strategy and (5) the conduct of research. There is no clear-cut method to select an appropriate sample (Neuman,
Stages in sample selection.
Stage | Activities |
---|---|
1 | Identify WCED schools as target population |
2 | Identify senior phase (Grades 7–9) mathematics teachers as subjects from the population |
3 | Select sample with the assistance of co-promoter, a senior statistician |
4 | Access WCED EMIS database to locate sample schools |
5 | Identify participating schools through systematic random cluster sampling |
6 | Build redundancy through oversampling |
7 | List contact information of participating WCED schools |
8 | Adjust sample according to circumstantial conditions at grassroots |
EMIS, Education Management Information System.
The WCED is divided into eight education districts: four rural districts, which correspond to one or more municipalities, and four urban or metro districts located within the City of Cape Town. Each educational management district has a district director, a circuit team with a circuit team manager who coordinates the tasks of the curriculum advisors, Special Needs Education professionals, Institutional Management and Governance planning and a School Governance and Management team (WCED,
Babbie (
General question format. The arrangement of the questions in the questionnaire is of equal importance to the nature and wording of the questions asked. Jumbled and shortly phrased questions confuse respondents and can lead to respondents not replying to questionnaire items (Babbie,
Usable format for respondents. The researchers provided tick boxes to capture responses easily. For Parts C–F of the questionnaire, the researchers used a five-point Likert scale (1 = strongly agree; 2 = agree; 3 = strongly disagree; 4 = disagree; 5 = do not know).
Ordering of items in the questionnaire. The researchers placed the demographic data at the beginning of the questionnaire (Parts A and B) to provide an easy start and put the respondents at ease. The more complex questions on attitudes, attributes, opportunities, self-classification and knowledge followed in Parts C– G.
Questionnaire instructions. A cover letter introduced the background and the rationale for the research, followed by basic instructions on how to complete the questionnaire.
Pretesting of the questionnaire. To ensure that the questions were clear and unambiguous, the researchers approached mathematics teachers to assess the readability and layout of the questionnaire.
Data processing. A data processing pane on the questionnaire was created: (1) to assist the researchers during the capturing and checking of the data and (2) for respondents to see which data would be used for the analysis (Babbie,
Seven guidelines to formulate questions.
Guideline | Rationale |
---|---|
Formulate clear questions | Respondents interpret the text in the same way; no unclear words |
Avoid double-barrelled questions | Question are limited to one single idea or question |
Match the level and tone of questionnaire to the characteristics of the target respondents | Respondents should feel compelled to supply trustworthy responses |
Ensure the relevance of the questions | Respondents should be able to respond to the questions based on their lived experiences |
Pose short, simple questions | Shorter questions ensure respondents are able to read and respond quickly |
Avoid negative items | Respondents cannot misinterpret the question and be unsure how to approach the question |
Avoid biased items or terms | Avoid formulating questions in order to influence the intended outcomes |
The researchers compiled the questionnaire according to the seven identified parts.
Part A requested personal information of the respondents: (1) gender, (2) age, (3) home language, (4) language of instruction, (5) years of teaching experience in the various grades, (6) highest qualification or highest professional qualification and (7) subject specialisation. Part B comprised demographical information relating to the: (1) school district, (2) nearest town or city, (3) geographical location (rural or urban area), (4) quintile of the school, (5) number of mathematics classes per grade, (6) number of mathematics classes teachers teach, (7) availability of computers for teaching, learning and administration and (8) access to the Internet for administration, teaching and learning. For some questions, the respondents had to select the appropriate option or fill in the applicable response.
Parts C–F comprised complex questions on attitudes, attributes, opportunities, self-classification and knowledge relating to the research. Part G relates to PD models. Parts C–F posed questionnaire items relating to the four themes (governance; school environment; ODL; PD). The selected quotations from the qualitative analysis illustrate the underlying constructs. A closed form method of a five-point Likert scale (1 = strongly agree, 2 = agree, 3 = strongly disagree, 4 = agree, 5 = do not know) pinpoints respondents’ inputs.
Description of compilation of questions for Parts C–F.
Question | Themes from inductive systematic literature analysis | |
---|---|---|
C1 | The WCED motivates for the use of ICT in mathematics teaching | Promote effective practices in the use of ICTs in teaching and learning (DOE, |
C2 | The WCED allocates funds for ICT mathematics training | The DBE provides access to technology, including teacher development (DOE, |
C3 | The WCED provides funding for ICT resources | Building an integrated e-education system requires better investment in the education sector (DOE, |
D1 | I use social software for personal use, e.g. email, Facebook, Twitter, Mxit, WhatsApp, BBM | Many researchers recognise the value of social software in ICT PD (Ala-Mutka, Punie & Redecker, |
D2 | I use the Internet to find mathematics resources | The Internet can be a tool to search for new developments in mathematics teaching and learning (Da Ponte, Oliveira & Varandas, |
D3 | I use ICT in mathematics teaching to achieve the learning outcomes | During training teachers use ICT in problem-solving, producing mathematics lessons and creating new problems (Swan et al., |
E1 | My school negotiates with service providers for reliable Internet access | Stakeholders negotiate easier access to reduce costs for Internet-enabled devices (Department of Higher Education and Training, |
E2 | My school has an ICT policy | Every South African learner will have the capability to function within an ICT environment by 2013 (DOE, |
E3 | My school has teachers who use ICT innovatively | Schools should have skilled and capable teachers who use ICT to enhance teaching and learning (DOE, |
F1 | I develop my mathematics competencies myself | Teacher's own attitude regarding their development (Da Ponte, |
F2 | I know about the current trends in mathematics education | The structure of the teachers’ activity has undergone important changes involving new curriculum goals and professional responsibilities (Da Ponte, |
F3 | I am innovative with ICT in my mathematics teaching | Teachers to explore new ways of using this technology (Da Ponte, |
Part C of the questionnaire addressed the governance of ICT implementation at provincial (WCED), district and school level. The majority of the items in this section relate to the objectives of the three-phase ICT integration plan as stipulated in the
Aspects relating to the school environment were grouped as one part of the questionnaire with various sub-questions (Part D). The questions relate to aspects regarding the extent to which mathematics teachers: (1) create a classroom environment where they utilise the Internet and ICT applications, (2) download resources, (3) develop lessons with the help of ICT, (4) stimulate their learners to be creative, (5) use the vast array of resources to improve their academic performance, (6) achieve the assessment standards and (7) have a positive attitude towards the use of ICT for teaching and learning of mathematics.
In Part F, the researchers aimed to: (1) confront issues about PD of mathematics teachers, (2) gain insight into the opinions of mathematics teachers on future PD opportunities, (3) learn about the context in which PD activities should occur and (4) assess the modes in which PD should take place.
The final part of the questionnaire (Part G) included four PD models conceptualised using the multiple PD models and frameworks identified in the literature review. The researchers adapted a variety of PD models, tested in other education systems across the world by the expert researchers in this particular field, to suit the background and context of South African schools. The four models included the best practices for PD that could work in the South African context.
The data collection of a survey can be conducted via a postal interviews, personal interview, telephone, and Internet-based surveys. Each of these strategies has its own strengths and weaknesses (Cohen et al.,
The validation of the questionnaire before distribution to the participants is critical. Two components have to be piloted: (1) the instructions to the respondents and (2) the questionnaire itself (McMillan & Schumacher,
Before data collection the researchers obtained ethical clearance from the university's ethics committee and permission from the WCED to distribute the questionnaire in the eight education districts. Ethical clearance ensures that the respondents are protected from harm and that the researcher ensures the respondents of confidentiality, anonymity and the non-traceability of their participation in the research. The researchers planned the data collection in accordance with the schedule of the district offices and contacted the relevant parties (subject advisors, schools) via email and followed up by telephone.
The first author confirmed the appointments for the data collection sessions and visits to schools telephonically and travelled in 37 days a total of 9287 km in order to collect the data. This was a considerable task in terms of man-hours and cost. The researchers collected data from farm, semi-urban, urban, former Model C and independent schools across the districts.
Various statistical procedures (descriptive statistics, factor analysis and structural equation modelling) were followed to analyse the collected data.
The biographical information was presented with frequencies and percentages.
Descriptive statistics for Parts A and B.
Items | Frequencies | Percentages | |||
---|---|---|---|---|---|
A1 | Gender | Male |
135 |
45 |
|
A2 | Age | 20–29 |
66 |
22 |
|
A3 | Total number of years teaching | 0–9 years |
113 |
38 |
|
A4 | Total number of years teaching mathematics | Grade 7 | 0 years |
230 |
77 |
Grade 8 | 0 |
73 |
24 |
||
Grade 9 | 0 years |
50 |
17 |
||
A5 | Home language | English |
58 |
19 |
|
A6 | Language of instruction | English |
133 |
44 |
|
A7 | Qualifications | DE III |
21 |
7 |
|
A8 | Subject specialisation | Mathematics |
252 |
84 |
|
B1 | School district | Metro Central |
40 |
13 |
|
B4 | School quintile | 1 |
52 |
17 |
|
B5 | Type of school | Farm |
15 |
5 |
|
B6 | Number of learners at school | 0-500 |
53 |
18 |
|
B10 | Number of computer laboratories | 0 |
15 |
5 |
|
B11 | Computers at my school available for | Administration |
286 |
95 |
|
B12 | Computers with Internet available for administration |
Yes |
286 |
95 |
|
B13 | I rate my computer literacy level as | Poor |
24 |
8 |
|
B15 | Personal computers available | School | Yes |
195 |
65 |
Home | Yes |
258 |
86 |
||
B16 | Internet access | Home | Yes |
195 |
65 |
B17 | Interactive whiteboards at school | Yes |
163 |
54 |
|
B18 | Online course | Yes |
32 |
11 |
Only 7% of the mathematics teachers held a NQF level five qualification; 14% of the mathematics teachers had postgraduate degrees (
The researchers applied
>0.90: very highly reliable
0.80–0.90: highly reliable
0.70–0.79: reliable
0.60–0.69: marginally reliable
=0.60: low reliability
The questionnaire included subscales; therefore the reliability of each set of factors was calculated individually. The factors were extracted according to Kaiser's criteria and 10 of the 11 extracted factors had a reliability level higher than 0.8, which indicates the items in the questionnaire are significant to address the research problem.
Pattern matrix of factor analysis for Parts C–F.
Number of items | Factors in Part C | Factors in Part D | Factors in Part E | Factors in Part F | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 1 | 2 | 1 | 2 | 1 | 2 | 3 | |
1 | 0.569 | - | - | - | 0.692 | - | 0.661 | - | 0.655 | - | - |
2 | 0.652 | - | - | - | 0.480 | - | 0.839 | - | 0.684 | - | - |
3 | 0.898 | - | - | - | - | 0.619 | 0.514 | - | 0.803 | - | - |
4 | 1.004 | - | - | - | - | 0.713 | 0.662 | - | 0.773 | - | - |
5 | 0.906 | - | - | - | - | 0.865 | - | 0.742 | 0.716 | - | - |
6 | 0.763 | - | - | - | - | 0.814 | - | 0.828 | 0.831 | - | - |
7 | - | 0.309 | - | - | - | 0.856 | - | 0.732 | 0.753 | - | - |
8 | - | 0.486 | - | - | - | 0.614 | - | 0.461 | 0.723 | - | - |
9 | - | 0.520 | - | - | - | 0.345 | - | 0.934 | - | 0.447 | - |
10 | - | 0.628 | - | - | - | 0.903 | - | 0.704 | - | 0.755 | - |
11 | - | 0.819 | - | - | - | 0.901 | - | - | - | 0.769 | - |
12 | - | 0.826 | - | - | - | 0.909 | - | - | - | 0.918 | - |
13 | - | 0.533 | - | - | - | - | - | - | - | 0.755 | - |
14 | - | - | 0.485 | - | - | - | - | - | - | 0.380 | - |
15 | - | - | 0.617 | - | - | - | - | - | - | 0.529 | - |
16 | - | - | 0.610 | - | - | - | - | - | - | - | –0.427 |
17 | - | - | 0.833 | - | - | - | - | - | - | - | –0.330 |
18 | - | - | 0.653 | - | - | - | - | - | - | - | –0.892 |
19 | - | - | 0.404 | - | - | - | - | - | - | - | –0.451 |
20 | - | - | - | 0.440 | - | - | - | - | - | - | - |
21 | - | - | - | 0.691 | - | - | - | - | - | - | - |
22 | - | - | - | 0.653 | - | - | - | - | - | - | - |
23 | - | - | - | 0.657 | - | - | - | - | - | - | - |
Cronbach's alpha | 0.95 | 0.85 | 0.81 | 0.88 | 0.58 | 0.94 | 0.80 | 0.88 | 0.92 | 0.87 | 0.87 |
Mean | 3.26 | 2.92 | 3.24 | 3.24 | 3.19 | 2.71 | 3.26 | 2.62 | 3.19 | 3.23 | 3.26 |
Standard deviation | 1.204 | 0.768 | 0.977 | 0.926 | 0.601 | 0.709 | 0.917 | 0.774 | 0.548 | 0.450 | 0.503 |
Four factors were extracted with the factor analysis. The Kaiser Meyer Olkin (KMO) measure of 0.912 indicated adequate data for factor analysis with a total variance of 59%. Communalities varied from 40% for Factor 1 and 50% for Factor 2 to 55% for Factor 3 and 59% for Factor 4. The factors showed a high reliability of 0.95 for Factor 1, 0.85 for Factor 2, 0.81 for Factor 3 and 0.88 for Factor 4. The four factors were named: responsibility of DBE) (Factor 1), responsibility of management (Factor 2), responsibility of teaching and learning (Factor 3) and policy initiatives (Factor 4).
The mean of 3.26 for
Two factors were extracted with a KMO measure of 0.906 indicating adequate data for factor analysis with a total variance of 59%. Communalities varied from 53% for Factor 1 to 61% for Factor 2. The factors showed a marginal reliability of 0.60 for Factor 1 and 0.71 for Factor 2. Factor 1 relates to
A mean of 3.19 for TK indicates that quite a few teachers regard TK as more important than TPACK. They believe that if they understand ICT, they can (1) use it in their daily lives, (2) employ it in their work environment and (3) identify when ICT can complement other methods to achieve the learning outcomes for mathematics. The mean of 2.71 for TPACK indicates that some mathematics teachers regard ICT pedagogical skills to impart knowledge and skills as less important (
Two factors were extracted with the KMO measure of 0.893 indicating adequate data for factor analysis with a total variance of 55%. Communalities varied from 46% for Factor 1 to 55% for Factor 2. The factors showed a high reliability of 0.92 for Factor 1 and a reliability of 0.78 for Factor 2. Factor 1 relates to
A mean of 3.26 for contributors to SPI indicates that holistic growth through self-awareness and constructive socialisation is critical to many mathematics teachers. Mathematics teachers should have Internet access so that they can create a network with other mathematics teachers to discuss their uncertainties and share their best practices. A mean of 2.62 for ICT and SPD indicates that mathematics teachers regard their holistic growth through social interaction as more important than the extent to which they use ICT to add to their SPD (see
Three factors were extracted with the KMO measure of 0.927 indicating adequate data for factor analysis with a total variance of 55%. Communalities varied from 50% for Factor 1 to 57% for Factor 2 and 61% for Factor 3. The factors showed a high reliability: of 0.92 for Factor 1, 0.87 for Factor 2 and 0.87 for Factor 3. Factor 1 relates to
Means of 3.19 for teachers’ expectations for PD, 3.23 for building a SPI and 3.26 for PD models and frameworks indicate that mathematics teachers in the WCED regard all these as important during PD. Mathematics teachers expect the DBE, provincial departments and schools to work together to develop an ICT strategic plan, which includes a PD model that is versatile, so that they can develop at their own pace, perceptive to address subject-specialised training and insightful to their developmental needs (
The aim of the structural equation model was to determine whether there were statistically significant relationships between the four themes.
Standardised regression weights and correlations between governance, school environment, open distance learning and professional development.
Activity system | Standardised regression weight | |
---|---|---|
Environment←Governance | 0.61 | ***** |
Environment←Open distance learning | 0.97 | ***** |
PD←←Open distance learning | –0.90 | 0.594 |
PD←←Environment | 1.07 | 0.533 |
*****,
PD, professional development
Chi-square test statistic divided by its degrees of freedom (CMIN/DF) value of 3.40, which was less than 5, which indicated a good fit.
CFI value of 0.88, which indicated an acceptable overall fit (Mueller,
Structural equation model from governance, school environment, open distance learning and professional development.
Root mean square error of approximation measures the differences in the observed values in the model. A value of 0.11 was not smaller than 0.1 for unacceptable fit (Blunch,
The structural equation model illustrates the validation of how the four themes were supportive towards achieving the object of the research. Even though ODL has a negative influence on PD; ODL is a new mode of service delivery in terms of PD in the current education system in South Africa. When teachers become acquainted with PD through ODL, and the benefits thereof, their perceptions and attitudes towards ODL as a method of PD may change.
The first author wrote a full report on the findings from the statistical analysis with clear guidelines for the PD of mathematics teachers for the pedagogical use of ICT in ODL. Additionally the research: (1) developed a model for identity essentials for PD for mathematics teachers in South Africa and (2) compiled strategies on how to conduct systematic literature and quantise literature data through exploratory factor analysis. The researchers are in the process of: (1) distributing the questionnaire in Finland to do a comparison between PD needs of mathematics teachers in the two contexts and (2) constructing national guidelines for PD for ICT integration.
With the 14 stages of questionnaire development, the research developed, validated and standardised an instrument for PD of mathematics teachers for the pedagogical use of ICT, which (1) enables each context to assess the PD requirements, (2) gives access to the developmental needs of mathematics teachers to allow for the creation of context-specific PD programmes and (3) can consequently be used by other researchers to compare the contexts of other South African provinces, as well as to explore and describe PD needs in diverse contexts.
The factor analysis extracted factors with high reliability which accentuates the value of the analysis. From the results it became clear that fundamentally ICT integration and implementation initiatives start with the DBE. Before developing future PD which aligns with the continuous teacher PD management system and Phase III of the e-education policy, the DBE should appoint a DBE panel comprising national and provincial ICT coordinators. Once all the groundwork has been done they can plan future ICT initiatives, develop PD models that suit the South African school context and supply the provincial education departments and schools with guidelines for PD. The DBE should invest in the provision of ICT equipment and human capital, reinstate the laptop initiative for teachers and supply schools with networked computer facilities so that they can explore online platforms for PD.
This work is based on research conducted for a PhD study. The research is in part funded by the National Research Foundation of South Africa. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and therefore the NRF does not accept any liability in regard thereto.
This publication is based on research that has been supported in part by the University of Cape Town's Research Committee.
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
V.L. (North-West University) was the main author of the article. A.S.B. (North-West University) assisted with the qualitative analysis and to create constructs. S.E. (North-West University) was the co-promoter and statistician and assisted with the structure, scales and quantitative analysis. H.D.N. (North-West University) assisted with the interpretation of the quantitative analysis.
A quintile is a cut-off point to receive a subsidy from the government for the provision of resources.