Speaker
Description
In the context of students' ability assessment, considering collateral information in addition to item responses can be helpful in increasing the accuracy of the measurement. In this vein, the evaluation of students' abilities via computer based-devices has made response time data available at the item level (Wang et al., 2019). Besides, the literature (Becker et al., 2022) has highlighted an item position effect when the same items are presented in different positions within multiple test forms.
With the present contribution, we contribute to this research line by proposing a structural equation model (SEM) to jointly consider item responses, response times and item positioning in students' ability assessment. In particular, we assume that the response process is driven by two underlying latent variables: the first latent variable, denoted by $\Theta_i$, represents the ability of individual $i$ that is measured by the test items; the second latent variable, denoted by $\eta_i$, refers to the speediness of individual $i$ to answer the test items.
We formulate the statistical model assuming that the item responses are directly affected by the ability $\Theta_i$, whereas the response times depend both on the ability $\Theta_i$ and on the speediness $\eta_i$. Accordingly, response accuracy tends to increase with the ability level of individual $i$ while response time tends to decrease with the speediness and ability levels. Moreover, we suppose that item positioning affects both item responses and response time. Under this setting, the correlation between $\Theta_i$ and $\eta_i$ is modelled through the cross-relation function that models the relationships between $\Theta_i$ and the observed response times.
The empirical application of the proposed model was carried out on first-year Psychology students at the University of Naples Federico II, attending the introductory Statistics course. The test administered was composed of 30 multi-choice questions developed according to three of the five Dublin descriptors: Knowledge (10 items), Application (10 items) and Judgement (10 items). For each question, students' answers were coded as correct (2 credits), partially correct (1 credit) and wrong (0 credits). Data were collected through Moodle platform, which also provided the response time.
References
Becker, B., Van Rijn, P., Molenaar, D., and Debeer, D. (2022). Item order and speededness: Implications for test fairness in higher educational high-stakes testing. Assessment & Evaluation in Higher Education, 47(7):1030–1042.
Wang, C., Weiss, D. J., and Su, S. (2019). Modeling response time and responses in multidimensional health measurement. Frontiers in psychology, 10:51.
