Speaker
Description
Learning achievements are usually assessed by administering tasks to students- ranging from individual assessment to Large-scale surveys. Task design and administration are key issues in an assessment process. In the last decades, quantitative research investigated systemic changes in assessment modality, such as how the switching from a paper-and pencil administration of a task to a computer-based testing impacts on students’ answers, or from static to interactive administration. Less research has been done at item level on the impact of changes in the formulation of a task.
Students’ answers to an item depend on a very complex system of relationships between the elements of the formulation, personal features, teachers’ style, and the learning noosphere. It is a somehow chaotic system where no direct cause-and-effect relationships are usually clearly identifiable. A classical research trend in mathematics education is how the linguistic formulation of a word problem impact students’ answers. This issue, of course, has many significant consequences, for instance, on how items administered in international surveys are translated into the different languages.
Qualitative research investigated how a change in the formulation of a problem may actually change the problem itself, for a particular student. Several theoretical frameworks are available, based on qualitative evidence, but measurement is rarely used in Mathematics Education studies, since the optimal situation is difficult to achieve. It is of course impossible to administer two different versions of the same task to the same student, since he/she should “forget” to have answered to that task the first time, before answering to the second version.
To overcome such a limit, we developed and administered four anchored Mathematics achievement tests, consisting of two sections. The first one was given by a core set (named Core Test) of common items (used to anchor tests) and representing the entire achievement test from both a statistical and mathematical point of view (i.e., both in terms of estimated parameters and mathematics items’ content). Compared with previous studies in Mathematics Education, our “Core Test” consists of items from Large-scale assessment, each of which administered by INVALSI to the entire Italian students’ population in previous main studies. Results from data analysis confirmed the “robustness” of our Core Test that can thus serve both as a benchmark and as a definition of the latent trait. This measurement step can be inserted both in explanatory and in exploratory plans. The second section of each achievement test was given by alternative formulations of the same stem-items: each stem-item was modified by performing on it a single, well-individuated variation, thus resulting in a quasi-experimental design which can detect emerging phenomena due to changes in formulation (hence overlapping this difficulty at least at a systemic level).
We present examples of application of this methodological strategy, showing how it may contribute to better define the summative aspects of assessment, help in interpreting students’ behaviors when answering to an assessment question, thus giving a contribution to formative assessment, and help task-assessment designers.
| Research Strand | Methods and Methodological Aspects in Science Education Research |
|---|
