BEGIN:VCALENDAR VERSION:2.0 PRODID:-//CERN//INDICO//EN BEGIN:VEVENT SUMMARY:"I don’t know": dealing with uncertainty in students' perceived competence in STEM DTSTART;VALUE=DATE-TIME:20230130T141500Z DTEND;VALUE=DATE-TIME:20230130T144500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-642@cern.ch DESCRIPTION:Speakers: Rosa Fabbricatore (Università degli Studi di Napoli )\nThe importance of STEM (science\, technology\, engineering\, and mathem atics) subjects has been growing in different economic sectors and in gene ral for society as a whole (Murphy et al.\, 2019). Nevertheless\, there ar e significant concerns in STEM education (Höhne et al.\, 2019) mainly rel ated to the students' difficulties in achieving successful academic perfor mances\, the high dropout rates\, and a consistent gender discrepancy (Pri ulla et. al.\, 2021). Students’ motivation to enrol in STEM courses is g enerally related to career expectation and their perceived competence in s cientific subjects (Franks et al.\, 2019). Some students are\, however\, c autious about or unsure of their abilities to succeed in mathematics and s cience\, leading to “don’t know” responses when questioned about the ir competencies. The aim of this contribution is to analyse uncertainty in students’ perceived competence from an integrated assessment perspectiv e. We employ a within-item multidimensional latent class IRT model (Bacci et al.\, 2014) to analyse item responses about self-concept in mathematics and science\, taking into account the “Don’t Know” option.\n\nhttps ://indico.unina.it/event/60/contributions/642/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/642/ END:VEVENT BEGIN:VEVENT SUMMARY:An analysis of the differences in Italian students’ performance in STEM and no-STEM courses DTSTART;VALUE=DATE-TIME:20230130T134500Z DTEND;VALUE=DATE-TIME:20230130T141500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-641@cern.ch DESCRIPTION:Speakers: Marialuisa Restaino (DISES Università di Salerno)\n In recent years\, exploring the determinants that may influence students ’ achievement has received much attention. Empirical studies have found that the most important factors which affect student performance are stude nts’ characteristics\, family background\, school attended\, and regiona l residence.\nThis paper aims at investigating the differences in students ' performances among Science\, Technology\, Engineering and Mathematics (S TEM) courses by using regression models. To measure performance (universit y success)\, we focus on the number of ECTS credits earned during the firs t year\, since it represents an important moment in the students’ path a t university.\nThe analysis concerns students enrolled at 3-year STEM degr ees in an Italian university located in the South of Italy during the last 5 years\, with a focus on the number of university credits earned during the first year (a good predictor of the regularity of the career). In part icular\, the main purpose is to estimate the probability of getting at lea st a certain number of credits at the end the first year and identify the factors which might affect it. As threshold\, we opt for choosing that fix ed by the Italian National Agency for the Evaluation of Universities and R esearch Institutes (ANVUR) in the Annual Review Report ("Scheda di monitor aggio annuale") for evaluating the university courses.\nThe data are colle cted from the Student Information System (ESSE3)\, a student management sy stem used by most Italian universities\, which manages the entire career o f students from the enrollment to graduation. It contains information abou t students’ high school diplomas\, personal characteristics\, exams\, ab road experience\, internship\, and degrees. Using students’ identificati on numbers\, we merged the dataset which contains students’ demographic information and the dataset which reports all exams taken by students. Sin ce the aim is to estimate the probability of earning a given number of cre dits\, the dependent variable is the number of credits accrued at the end of the first academic year\, while the independent variables\, among the o thers\, are the gender\, course of study\, type of diploma\, final high sc hool mark\, students’ age\, residence place\, average exams mark\, and t he distance between residence place and university.\n\nhttps://indico.unin a.it/event/60/contributions/641/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/641/ END:VEVENT BEGIN:VEVENT SUMMARY:Clustering methods to study student reasoning lines: theoretical a spects and experimental results DTSTART;VALUE=DATE-TIME:20230130T113000Z DTEND;VALUE=DATE-TIME:20230130T121500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-661@cern.ch DESCRIPTION:Speakers: Onofrio Rosario Battaglia (Università di Palermo)\ nStudying reasoning lines students deploy when dealing with problematic si tuations is\, in Education Research\, one of the most important aims. The reasoning lines can be achieved by analysing the answers that students giv e to a questionnaire\, although it becomes increasingly complicated as the number of students to be analysed increases. In my speech I focus on a qu antitative method based on clustering by discussing theoretical and method ological aspects. The method can allow a researcher to analyse a set of an swers given to a questionnaire\, even in the case of a large sample of stu dents. Moreover\, clustering is not presently common to the physics educat ion research community. I describe in detail two different clustering meth ods\, a hierarchical one and a non-hierarchical one. I introduce a binary coding that makes the answers quantitatively analysable. Moreover\, a corr elation coefficient and a metric suitable for measuring student similarity in the case of binary coding are presented. Then\, criteria for choosing the optimal number of clusters for both the clustering methods are discuss ed. For the same purpose\, a new coefficient is introduced to measure the total amount of information one can obtain from a clustering solution. I s how that each cluster can be characterised by its centroid. It summarizes the most frequent answers given by students in a given cluster. An example of a clustering procedure for experimental data is given. The comparison between the results obtained through the two clustering methods shows a go od agreement exhibiting robustness in the proposed method.\n\nhttps://indi co.unina.it/event/60/contributions/661/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/661/ END:VEVENT BEGIN:VEVENT SUMMARY:Use of Multiple correspondence analysis to investigate students’ mental models about quantum entities DTSTART;VALUE=DATE-TIME:20230130T161500Z DTEND;VALUE=DATE-TIME:20230130T163500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-646@cern.ch DESCRIPTION:Speakers: Giovanni Giuliana (University of Camerino)\nhttps:/ /indico.unina.it/event/60/contributions/646/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/646/ END:VEVENT BEGIN:VEVENT SUMMARY:Investigating secondary students’ identification with physics th rough structural equation modeling DTSTART;VALUE=DATE-TIME:20230130T155500Z DTEND;VALUE=DATE-TIME:20230130T161500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-645@cern.ch DESCRIPTION:Speakers: Danilo Catena (University of Udine)\nIn STEM educati on\, the identity framework is often used to investigate students’ inten tion to pursue a STEM-related career. The purpose of this study was to exp lore the relationships between physics identity\, interest\, recognition\, and performance-competence in physics. The analysis was based on a Likert -scale survey aimed at measuring the addressed constructs\, administered o nline to N = 1135 Italian undergraduate and high school students. We valid ated a structural model in which performance-competence play the role of t he independent variable\, physics identity represents the dependent variab le\, while interest and recognition act as total mediators. We also consid ered two moderating variables for this model\, gender and previous experie nce with physics\, finding significant gender differences only for the pat h mediated by recognition\, while very significant differences were found depending on different experience with physics. Results have implications for instruction in terms of understanding the mechanism underlying the pro motion of students’ identity development in physics.\n\nhttps://indico.u nina.it/event/60/contributions/645/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/645/ END:VEVENT BEGIN:VEVENT SUMMARY:Assessment of social and emotional skills in cross-national settin gs: an OECD approach DTSTART;VALUE=DATE-TIME:20230131T080000Z DTEND;VALUE=DATE-TIME:20230131T084500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-662@cern.ch DESCRIPTION:Speakers: Milos Kankaras (Faculty of Philosophy\, Department o f Psychology\, University of Montenegro\, Montenegro)\nhttps://indico.unin a.it/event/60/contributions/662/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/662/ END:VEVENT BEGIN:VEVENT SUMMARY:The black box is not that black anymore: The use of eye-tracking i n STEM education research DTSTART;VALUE=DATE-TIME:20230131T084500Z DTEND;VALUE=DATE-TIME:20230131T093000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-663@cern.ch DESCRIPTION:Speakers: Martin Rusek (Charles University\, Faculty of Educ ation. Prague\, Czech Republic)\nDespite many attempts to measure pupils a nd students' learning outcomes\, their performance in problem solving has largely remained something of a black box. This talk will therefore focus on the potential of using eye-tracking data in combination with the think- aloud method. Case studies will be used to show how to analyze data from a ctual science education research. Finally\, further possibilities of using eye-tracking in STE(A)M education research will be outlined.\n\nhttps://i ndico.unina.it/event/60/contributions/663/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/663/ END:VEVENT BEGIN:VEVENT SUMMARY:Automated Assessment Of Open-Ended Question Of Invalsi Tests DTSTART;VALUE=DATE-TIME:20230131T154500Z DTEND;VALUE=DATE-TIME:20230131T163000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-664@cern.ch DESCRIPTION:Speakers: Michele Marsili (Invalsi)\nThis work describes the n ew procedures of automated corrections of freeform answers given by the 8t h\, 10th and 13th grade students to open-ended questions in CBT Computer B ased Test) INVALSI tests. INVALSI team\, composed of statistical and compu ter scientists\, responsible of open-ended question correction\, has imple mented an algorithm to process text strings of different complexity. \nBef ore survey distribution\, the correction team and the items authors group discuss to define the correction criteria\, that is a set of rules to dete rmine the correct or incorrect classification for each answer given by the students for a specific item. The discussion produced\, moreover\, the in dications to remove useless elements for the classification\, then transla ted in operations of the algorithm on the textual data such as punctuation detection and removal\, special characters\, articles\, conjunctions\, wo rd lemmatisation\, etc. The answer strings were subsequently processed by a “data cleaning” operation\, that was focused on the automated correc tion of spelling and typing errors\, by detection and substitution of “o ut-of-vocabulary” words (OOV words). \nAfter the “data cleaning” pha se\, the correction criteria fixed by the experts have been translated in logical IT patterns\, aiming to uniquely defining the set of admissible wa ys to give a correct answer. The last test phases of the algorithm were ch aracterized by a constant exchange of information about the encoding\, amo ng the authors’ team and the correction team\, this passage being critic al to refine the logical rules used for correction and to get more consist ency and precision between the encoding produced by the algorithm and the authors’ indications. \nThe final test of the algorithm ends with a comp arison between the manual encoding by video correction and the one process ed by the algorithm on a set of items already processed in a former test: the algorithm is accounted as accurate enough and aligned to the indicatio ns of authors’ team when the complete accordance of the two encoding was achieved. \nThe methodological approach\, countable as a method of superv ised automated correction\, represents a valid compromise between a manual encoding and a totally automated one\, typical of the machine learning al gorithms.\nThis method has indeed the benefit of considerably reduce the h ours/man needed to correct the open-ended answer items\, when compared to a manual procedure\, and get a better accuracy reducing the wrong encoding matches\, when compared to a non-supervised automated procedure. A compar ison between supervised and non-supervised automated procedure has been ev entually done to evaluate the distance between the two methodological appr oaches.\n\nhttps://indico.unina.it/event/60/contributions/664/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/664/ END:VEVENT BEGIN:VEVENT SUMMARY:Evolution acceptance and high school students. Methodological con siderations and new perspectives of investigation DTSTART;VALUE=DATE-TIME:20230130T084500Z DTEND;VALUE=DATE-TIME:20230130T093000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-660@cern.ch DESCRIPTION:Speakers: Giuseppe Pellegrini (Observa Science in Society)\nTh e idea of biological evolution is not accepted by many people around the w orld\, with a large disparity amongst countries. Some factors may act as o bstacles to the acceptance of evolution\, such as religion\, a lack of ope nness to experience\, and not understanding the nature of science. Althoug h the strength of the association between evolution acceptance and non-sci entific factors varies among studies\, it is often assumed that resistance to evolution is the by-product of a religious background. Some studies ar e even more specific and try to associate the acceptance of evolution with precise religious affiliations. \nIn my speech I will propose an introduc tion of the main tests used internationally to measure evolution acceptanc e and some statistical tools that allow our research team to verify the re levance of sociocultural factors in predicting such acceptance. \nStarting from an in-depth reflection on the factors that influence the knowledge a nd acceptance of evolutionary theories\, the Italian-Brazilian research gr oup formulated a research question to address this complex picture conside ring the same religious affiliation in two different countries with deep s ociocultural differences. Catholic Christians in Italy and Brazil have sev eral similarities\, including many family connections owing to immigration history. Brazil is the country with the highest number of Catholic Christ ians in the world\, and Italy is the hub of Roman Catholicism.\nThese cond itions allowed to conduct a survey on the adolescent population with two s tatistically significant national samples in 2014.\nWe adopted a clear def inition of evolution acceptance despite the complexity of the discussion\n on the subject in different languages\, taking acceptance as the expressio n of explicit recognition\nof the objective validity of known scientific s tatements about evolution under absolute\nanonymity. This definition consi ders two steps. The first is associated with scientific statements\nabout evolution\, which must be clear and well known\, avoiding issues under dis cussion\,\nfor instance\, about the origin of life. Students must show not only a positive attitude towards\nevolution but also express clearly that a statement based on biological evolution is considered a\nvalid premise to construct a judgement about the real world. The second step refers to o bjective conditions in which a person may admit his/her positive judgment about a certain scientific statement.\nWe aimed to explore the strength of associations among nationality\, religion\, and the acceptance of evoluti on by students using multiple correspondence analysis (MCA) and statistica l tools\, with\nnationwide samples from two different countries.\nIn our r esearch we found that wider sociocultural factors predict the acceptance o f evolution to a higher degree than a religious background. Roman Catholic students showed significant differences between the two countries\, and t he gap between them was wider than between Catholics and non-Catholic Chri stians within Brazil. Our conclusions support those who argue that religio us affiliation is not the main factor in predicting the level of evolution acceptance.\nThe sociocultural environment and the level of evolutionary knowledge seem to be more important in this regard. These results open up new interpretative perspectives and provide a better understanding of atti tudes towards evolution.\n\nhttps://indico.unina.it/event/60/contributions /660/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/660/ END:VEVENT BEGIN:VEVENT SUMMARY:What I wish I had known as a Rasch beginner…. DTSTART;VALUE=DATE-TIME:20230131T150000Z DTEND;VALUE=DATE-TIME:20230131T154500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-659@cern.ch DESCRIPTION:Speakers: William Boone (Miami University )\nThis talk will pr esent a summary of what I wish I had known 30+ years ago as I began my Ph. D. studies with Ben Wright and first learned Rasch methods. In this talk I will present key errors beginners commonly make when they start to use Ra sch. Naturally I will briefly discuss the ins and outs of such beginner Ra sch errors…my goal..to aid Rasch novices so that they can confidently\, quickly and correctly use Rasch. I will also share and discuss what my fam ily calls “Bill’s Rasch elevator speech”…such a speech is the one I can give very quickly (60 seconds) in an effort to help others understan d why Rasch is important. Quickly explaining Rasch to those who do indeed need your Rasch help (but they do not know they need your assistance) is a critical skill that one must develop. It is an art to succinctly explain Rasch to others\, be it in person\, or in papers. Even as a beginner with Rasch\, by mastering your own elevator speech you can increase the number of people who seek your Rasch expertise\, be it in academia or the private sector.\n\nhttps://indico.unina.it/event/60/contributions/659/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/659/ END:VEVENT BEGIN:VEVENT SUMMARY:Exploring the impact of student characteristics and social context on mathematical literacy DTSTART;VALUE=DATE-TIME:20230130T131500Z DTEND;VALUE=DATE-TIME:20230130T134500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-640@cern.ch DESCRIPTION:Speakers: Cristina Davino (Dipartimento di Scienze economiche e statistiche -)\nThe diagnosis of the a-priori weaknesses of students bas ed on their features and past experience (e.g. age\, sex\, type of diploma ) together with the features of the social context represents one of the p reliminary assessment activities that can be taken into consideration so a s to define different typologies of students and then develop a tailored l earning programme for each student group. This paper is based on the study of a specific aspect of the school-leaving student's ability\, namely mat hematical skills. It is a matter of fact that the results of several asses sment exercises show that the mathematical knowledge of students leaving s econdary school is becoming worse and worse. The aim of the paper is to ev aluate if and how the socio-demographic features of students impact on the ir mathematical ability. Exploiting data of the INVALSI tests and the pote ntialities of quantile regression we will also explore if this impact is d ifferent in case of observed or unobserved heterogeneity of the student po pulation.\n\nhttps://indico.unina.it/event/60/contributions/640/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/640/ END:VEVENT BEGIN:VEVENT SUMMARY:The Calculation Of Socio-Economic-Cultural Status Indicator Of Lea rners From Invalsi Data DTSTART;VALUE=DATE-TIME:20230131T143000Z DTEND;VALUE=DATE-TIME:20230131T145500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-658@cern.ch DESCRIPTION:Speakers: Emiliano Campodifiori (INVALSI)\nThe socio-cultural and economic characteristics of students and the families from which they come play a very important role on the learning levels achieved\, even fr om the early years of schooling. It is well established\, in fact\, that s tudents who live in conditions of higher economic as well as social and cu ltural advantage have a better chance of achieving more satisfactory resul ts during their education. It is now well known in the literature that the socio-cultural and economic condition\, the so called background\, has a significant predictive value on learners' achievement (INVALSI\, 2008\; OE CD 2007). \nHaving established this\, it is essential to have an instrumen t\, i.e.\, an indicator\, to measure in some way the background of the lea rners of interest. The definition of an indicator of socio-economic-cultur al status and its calculation poses general and technical problems. \nThis paper presents the method and techniques used to calculate an indicator o f socio-economic-cultural status (ESCS) of pupils who participated in the National Assessment Service (NES) surveys. The calculation of ESCS is base d on discrete indicators such as parents' level of education (HISEI) and t heir employment status (PARED)\, but also on a continuous indicator that c an express a proximity measure of the material conditions in which the pup il lives outside school (HOMEPOS). \nThe latter indicator is calculated fr om the data of the regionally stratified sample of students participating in the INVALSI tests and using techniques related to the methodological fr amework of Rasch Analysis. The paper also illustrates some operational sol utions to overcome\, at least in part\, the problem of missing data\, alwa ys present in large-scale surveys\, especially for so-called context varia bles. Finally\, some initial analyses are proposed about the relationship between ESCS and learning levels achieved in Italian and Mathematics.\n\nh ttps://indico.unina.it/event/60/contributions/658/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/658/ END:VEVENT BEGIN:VEVENT SUMMARY:A Counterfactory Approach To The Evaluation Of Educational Policie s: Methodology\, Variables And Results DTSTART;VALUE=DATE-TIME:20230131T140500Z DTEND;VALUE=DATE-TIME:20230131T143000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-657@cern.ch DESCRIPTION:Speakers: Andrea Bendinelli (INVALSI)\nThe fight against earl y school leaving\, the prevention of school failure\, the raising of basic skills levels and support for learning are among the main objectives of t he interventions financed through projects activated in favor of schools. The study is aimed at quantifying the effect on the levels of competence a chieved by students in the INVALSI tests through interventions aimed at (i ) improving school performance and (ii) reducing early school leaving thro ugh the construction of a longitudinally anchored and counterfactual analy tical system able to evaluate the progression and/or changes in students' skills over time in three areas: Italian\, Mathematics and English. This e valuation is carried out in a comparative way between the students of the schools involved in the Projects (hereinafter "treated cases") with respec t to those not involved (hereinafter referred to as "untreated cases"). Th e purpose of effect evaluation is to verify the ability of an intervention to change the behavior or conditions of a specific target population in t he desired direction. The method used\, the Difference-in-Difference (Angr ist & Pischke\, 2009\; Keele\, 2020)\, makes it possible to estimate the v alue of the outcome variable of the students treated in the event that the y would not have received the treatment (counterfactual). The net effect o f the treatment is therefore obtained as the difference between the score observed on the "treated" students and the score that would have been obse rved in the absence of the treatment. To estimate the effect of interventi ons on the results of standardized tests\, a quasi-experimental evaluation design was introduced by adopting a counterfactual approach. In counterfa ctual analysis\, three groups are referred to: (i) control group\, (ii) fa ctual group\, and (iii) counterfactual group. The control group\, built us ing the Propensity Score Matching technique (Martini A.\, 2011)\, is made up of students who have not received the treatment and who present charact eristics (socio-demographic\, such as gender\, age\, geographical location \, citizenship\, socio-economic status and culture of the family of origin \; and scholastic\, previous level of competence and regularity with respe ct to the course of studies) completely similar to the characteristics of the students who received the treatment and who make up the current group. Finally\, the counterfactual group\, constructed with the help of a contr ol group\, is the one for which it is estimated what the variation over ti me of the students' competence in the treated group would have been if the y had not received the treatment. The data used selected from the national surveys to which INVALSI administers to students in the various school le vels covered by the Survey\, thanks to the presence of a longitudinal stud ent code\, it was possible to follow the students over time to evaluate th e variation in the outcomes following specific interventions. It was also possible to proceed with a clustering of the group of treaties to evaluate the effectiveness of policy interventions on sub-populations on the basis of the different characteristics present within them. Thanks to the subdi vision into subpopulations\, it was possible to conclude that the efficacy of the treatment had different outcomes according to the identified subgr oup. The results will also be discussed based on the literature on the sub ject.\n\nhttps://indico.unina.it/event/60/contributions/657/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/657/ END:VEVENT BEGIN:VEVENT SUMMARY:ASSESSING THE GENDER GAPS IN MATHS COMPETENCE: AN OVERVIEW OF WHAT WE KNOWN FROM INVALSI DATA DTSTART;VALUE=DATE-TIME:20230131T113000Z DTEND;VALUE=DATE-TIME:20230131T121500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-637@cern.ch DESCRIPTION:Speakers: Stefania Mignani (Department of Statistical Sciences \, University of Bologna)\nAmongst the extensive research conducted in the context of gender gaps in educational context\, an issue that is frequent ly examined is the role of pre-university education and of the study of ST EM subjects\, in particular mathematics. The impact of how this discipline is taught\, along with the relative attitudes to and interest in this sub ject\, seem to be the main factors pushing young people\, and girls in par ticular\, away from the study of STEM subjects.\nIn Italy several studies have been conducted to investigate the gender gap in mathematics using sta ndardized large-scale data from INVALSI tests.\nThe presentation will disc uss a review on recent learning results in mathematics of young Italian st udents both in a cross sectional context and in a longitudinal perspectiv e.\nFor cross-sectional data the focusing on the differences in top and lo w performances may confirm or not whether the gender differences are more evident within groups of students characterize by different achievement le vel.\nThe cross-section analyses provide only snapshots of the level of co mpetencies acquired at a specific grade in a given year. On the contrary\, a longitudinal approach could evaluate how the performances change over t ime.\nBesides\, INVALSI makes data available including not only the test r esponses\, but also a set of variables dealing with socio-demographic and economic characteristics\, the educational path and\, for some grades\, st udent self-reported information about individual and emotional aspects. In the literature\, an interesting issue deals with the emotional aspects in duced by the test administration. The emotional component behaviour can pa rtially explain the gender gap performance.\nThe results of the illustrate d studies offer room to enrich the debate around the international recomme ndations to promote gender equality in education by ensuring equal opportu nities in making educational choices between boys and girls and making the study of STEM subjects equally inclusive and attractive\n\nhttps://indico .unina.it/event/60/contributions/637/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/637/ END:VEVENT BEGIN:VEVENT SUMMARY:Clustering educational data: a high school students' performance a nalysis DTSTART;VALUE=DATE-TIME:20230131T105500Z DTEND;VALUE=DATE-TIME:20230131T112000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-636@cern.ch DESCRIPTION:Speakers: Matteo Farnè (University of Bologna)\nIn this talk\ , we first briefly discuss the definitions of Educational Data Mining and Learning Analytics\, providing an overview of the most used statistical me thods in both fields\, as well as a synopsis of their differences and simi larities. The possible aims of the different methods\, aimed at uncovering hidden patterns in educational data\, are stressed pointing to the possib le services provided to the whole school community. In the following\, we focus on clustering methods for educational data: we start from traditiona l methods (hierarchical\, partitive\, and density-based)\, we proceed with dimension reduction methods\, such as factorial k-means and reduced k-mea ns\, and we present some methods which incorporate the longitudinal dimens ion. Then\, we present a pilot analysis carried out on a dataset reporting the performance of a class of high school students in three periods (whic h were treated as three separate datasets)\, using hierarchical clustering \, partitive clustering (k-means)\, factorial k-means and reduced k-means techniques. The goal of the analysis is to show how the composition of gro ups and the number of groups vary in each period and which are the factors that influence the creation of groups. The partitions obtained with these algorithms were compared in terms of reliability using the average silhou ette width index. Reduced k-means and k-means generated similar results an d we can say that these results were the most acceptable considering the a verage silhouette width. Hierarchical clustering generated the same result s as the former algorithms only in the first two periods of time. The resu lts generated by factorial k-means differ from the other methods and as su ggested by the values of the average silhouette width\, it is not the best algorithm for clustering on the dataset available to us. The underlying m eaning of clusters over time and the reasons behind statistical results ar e discussed and analyzed in detail\, with the aim to highlight possible st udent group structures present in a high school class.\n\nhttps://indico.u nina.it/event/60/contributions/636/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/636/ END:VEVENT BEGIN:VEVENT SUMMARY:An application of Differential Item and Bundle Functioning analysi s to the study of gender differences in Mathematics Education: implication s for educational practitioners. DTSTART;VALUE=DATE-TIME:20230131T103000Z DTEND;VALUE=DATE-TIME:20230131T105500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-634@cern.ch DESCRIPTION:Speakers: Clelia Cascella (INVALSI)\nThe study of gender diffe rences in mathematics attainment has exponentially increased over time as closing the gap between boys and girls is a priority for both research and policy. Mathematics has in fact been listed by the European Commission as one of the ‘key competences’ necessary to all for personal fulfilment and development. Nonetheless\, there are still several countries around t he world in which boys significantly outperform girls in mathematics with negative implications at both individual and collective level (i.e.\, in t erms of expected employability and wage as well as in terms of societal an d economic development\, especially considering that the demand of STEM-re lated job is going to increase).\nThe current research builds on a Marie C urie project aimed at exploring the possible association between gender di fferences (particularly\, female underachievement) in mathematics and envi ronmental socio-cultural and economic factors. Building on results showing that the more traditional the ‘field’ in which students grow their ge nder identity\, the better the boys’ attainment in mathematics compared with girls\, the current study goes a step further by showing how measurem ent can be used to help educational practitioners in understanding the rel ationship between students’ gender and their attainment in mathematics.\ nWithin the framework of the Rasch analysis\, educational research has fre quently employed Differential Item Functioning (DIF) analysis to measure t he (possible) association between the probability of successfully encounte ring each single (mathematics) item and a single students’ characteristi c (such as gender). Differential Functioning occurs when examinees from di stinct groups\, but matched on ability\, have different probabilities of a nswering an item correctly. Differential functioning can relate to a singl e item or to a bundle of items. The former is referred as Differential Ite m Functioning (DIF)\, the latter as Differential Bundle Functioning (DBF). \nIn the current study\, after showing results from a systematic literatu re review about the employment of DBF in educational research\, I present an application of both DIF and DBF to Large-Scale Assessment (LSA) data\, collected in Italy by the Italian national institute for the evaluation of educational system\, in 2017\, at Grade 10 (on average\, 15-years old stu dents). Results were used to discuss (i) strengths and weaknesses of both DIF and DBF\; (ii) similarities and differences of these two analyses and\ , thus\, how each of them can be employed to support educational practitio ners in dealing with gender differences\; as well as (iii) the use of LSA data in Mathematics Education research.\n\nhttps://indico.unina.it/event/6 0/contributions/634/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/634/ END:VEVENT BEGIN:VEVENT SUMMARY:Mathematics achievement at the end of upper secondary school durin g COVID-19 pandemic: insights from the INVALSI national assessment DTSTART;VALUE=DATE-TIME:20230131T100500Z DTEND;VALUE=DATE-TIME:20230131T103000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-635@cern.ch DESCRIPTION:Speakers: Marta Desimoni (INVALSI)\nEducation stakeholders in times of the Covid-19 pandemic require a broad empirical base to recover f rom the crisis and to strengthen the resilience of education systems in th e future. National assessments of students learning outcomes are potential ly useful sources of education data during the COVID-19 pandemic\, playing a pivotal role in monitoring school systems and improving education quali ty. In Italy\, the National Institute for the evaluation of the education and training educational system (INVALSI) every year carries out a standar dized national assessment of students’ achievements in primary and secon dary education. From the school year (SY) 2018-2019\, the INVALSI national testing program has been extended to the last year of upper secondary sch ool\, thus allowing to depict of an overall picture of students’ mathema tics achievement at a key point of transition to tertiary education and em ployment. Results are reported not only in terms of numerical scores but a lso as proficiency levels\, to offer substantial information on the profic iency status at the system level. The attribution of an explicitly describ ed level is also supposed to allow the students\, their families\, and the teachers to have more significant and useful feedback compared to a simpl e score\, thanks to the direct link to the content area covered by the tes t. The present work aims at providing an overall picture of students’ p erformance in the INVALSI mathematical assessment at the end of upper seco ndary school in the school year 2020-21\, about one year after the COVID-1 9 outbreak in Italy\, taking into account their pre-pandemic mathematics p roficiency level (Grade 10\, s.y. 2017-18). Protective factors and possibl e sources of inequalities in achievements during the COVID-19 crisis are a lso explored\, by adopting a multilevel approach. The overall differences between pre-pandemic and pandemic cohorts in Italy suggest a pandemic achi evement gap in mathematics at the end of upper secondary school\, with a h igher number of students resulting as low performers\, i.e. “who are lik ely to use basic skills and procedures mainly acquired in lower secondary school and\, partly\, at the end of the first two years of upper secondary school […]” or maximum “knowing the basic mathematical concepts as outlined in the national guidelines for mathematics in the first two years of upper secondary school” […]. Considering their pre-pandemic starti ng points (G10) in a retrospective perspective\, it emerges that although most of the G13 low performers were already struggling with mathematics\, about one out of three moved from intermediate-high (G10 scale) to lowest levels (G13 scale). More encouraging results are those from another subgro up of pandemic-cohort students who maintained intermediate to high perform ance in mathematics (with respect to G10 and G13 scales)\, suggesting posi tive patterns of adaptation in the context of adversity due to the COVID-1 9 crisis. Multilevel analyses provide further insights into the relevance of different variables in supporting the students’ relative progresses i n mathematics during the pandemic\, both at the individual and the context ual level.\n\nhttps://indico.unina.it/event/60/contributions/635/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/635/ END:VEVENT BEGIN:VEVENT SUMMARY:An extensive questionnaire about emergency remote teaching: more t han 3000 engineering students respond about their perceptions on online d idactic activities. DTSTART;VALUE=DATE-TIME:20230130T151500Z DTEND;VALUE=DATE-TIME:20230130T153500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-628@cern.ch DESCRIPTION:Speakers: Roberto Mazzola (Department of Physics\, Politecnico di Milano)\nBy 11 March 2020\, the phrase “COVID-19” had officially e ntered everyday life across most of the word. Each level of education sudd enly faced new changes and new challenges. Emergency remote teaching becam e widespread\, and new methodologies to deliver classes and courses were a dopted by educational institutions. In February 2020\, the Politecnico di Milano introduced a series of focused and systemic actions in order to sup port the passage to completely online teaching and to ensure the continuit y of the activities that were previously developed in the classroom. In ou r work\, we focus on the impact of the remote learning experience on STEM disciplines\, by analysing the perceptions of engineering students enrolle d at the Politecnico di Milano. The subjects were recruited from all engin eering courses\, from the first to the fifth year\, and were asked to comp lete a multidimensional survey. The questionnaire was proposed in July 202 1 at the end of the second semester\, referring to the didactic activities held in the 2019-2020 and 2020-2021 academic years. More than 3000 studen ts completed the entire survey\, by answering the questions that were prop osed in a Likert scale\, from 1 to 5. This large sample was composed of 66 % male and 33% female students. Almost the 70% of the students were attend ing Bachelor's degree courses\, while the remaining were attending Master' s degree courses. The survey was composed of 66 questions regarding the pe rceptions and the challenges of online education\, compared with the “st ate of the art” before COVID-19\, divided into 6 main groups: Remote Tea ching\, Subjective Well Being\, Metacognition\, Self-Efficacy\, Identity a nd Socio-Demographic information. In this work we described the entire sur vey and we focused on the items concerning Metacognition and Self Efficacy but we also gave a first glance to the results concerning the remote teac hing section. We performed preliminary analysis\, by computing frequency d istribution and descriptive statistics concerning the remote teaching sect ion\, then\, by using Cronbach's alpha test\, confirmatory factor analysis and the t-test\, we performed a more in-depth analysis concerning the out comes of metacognition and self-efficacy. Data analysis shows that student s clearly appreciated how the Politecnico di Milano dealt with the organiz ation of the mandatory online courses\, but in the same time the results i ndicates that they complained about their relationships with classmates du ring remote teaching. Data also suggests an improving in effective learnin g strategies performed by students. We can say that students\, in some sen se\, overcame the difficulties due to the emergency remote teaching by imp roving their cognitive processes. Potentially\, this last result could hav e an important impact in teaching methodologies\, adding a point in favour of distance learning. But the topic is so sensitive and discussed that it surely needs to be investigated in more details.\n\nhttps://indico.unina. it/event/60/contributions/628/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/628/ END:VEVENT BEGIN:VEVENT SUMMARY:Investigating through worksheets the impact of short formative mod ules on prospective teachers' didactic projects about specific topics: qua litative analysis and data visualization. DTSTART;VALUE=DATE-TIME:20230130T161500Z DTEND;VALUE=DATE-TIME:20230130T163500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-630@cern.ch DESCRIPTION:Speakers: Francesca Monti (University of Verona)\nWe describe how we approached a study on a group of 65 primary school prospective teac hers (PPTs) aimed at investigating through post-assessment questions\, wor ksheets and final interviews how they would project a learning path on the subject of "heat\, temperature and energy" after following a short didact ic module involving the use of infrared thermal cameras and temperature on line sensors(1).\nThe intervention was proposed inside the laboratory part of the Physics Education course of the combined bachelor and master degre e in Primary School education of the Italian University of Verona. The mod ule was organized into four phases. The first and third phases were develo ped as interactive experiment-based lectures following active learning str ategies to introduce the operational definition of temperature\, the conce pt of energy\, the use of the online sensors and the use of the thermal ca meras. In the second and fourth lab-work phases\, four experiments were au tonomously carried out by the PPTs divided into groups.\nData collection w as made at the end of the course. In the post-assessment questions\, PPTs were asked to list the ways they know for measuring temperature. In the wo rksheets\, they were asked to plan a didactic project for their pupils abo ut thermal phenomena\, highlighting the addressed concepts and the corresp onding related activities. Final interviews were mainly focused on whether and how PPTs used or would use a thermal camera in their proposed activit ies and\, if so\, related to which concepts.\nThe investigation on how muc h the formative module and methodologies did help PPTs in understanding th e proposed concepts was made through the following research questions: 1) Do PPTs cite (in the proposed post-assessment question) thermal cameras am ong the possible instruments for measuring temperature? 2) Which concepts and corresponding activities do PPTs choose (in the worksheets) in constru cting their didactic project on thermal phenomena? 3) Do (from the workshe ets) or would (from the final interviews) PPTs propose the use of thermal cameras in their activities and\, if so\, related to what concepts?\nData analysis was performed following an iterative process of Qualitative Analy sis(2) by identifying directly from students’ answers a set of categorie s and refining them through successive re-readings of students’ reports. Attention was paid: a) on the instruments (online sensors and thermal cam eras) as cited in the post-assessment questions and as utilized in the act ivities proposed in their planned learning paths\; b) on the concepts addr essed by PPTs as related to the instruments utilized in the activities. Th e frequencies and percentages of the categorized answers were visualized a s bar charts.\nWe will present some examples of the answers given by stude nts and a description of the process that lead us to the final categorizat ion and interpretation of the obtained results.\n\n(1)Monti and Daffara (2 021) Journal of Physics Conference Series 1929(1)012020\n(2)Miles\,Huberma n and Saldaña (2014) “Qualitative data analysis: a methods sourcebook (Third Edition)”. Thousand Oaks\, California: Sage Publications\, Inc.\n \nhttps://indico.unina.it/event/60/contributions/630/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/630/ END:VEVENT BEGIN:VEVENT SUMMARY:Peer learning in higher education: An effective response to the un iversity students’ dropout problem DTSTART;VALUE=DATE-TIME:20230130T155500Z DTEND;VALUE=DATE-TIME:20230130T161500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-627@cern.ch DESCRIPTION:Speakers: Matteo Bozzi (Politecnico di Milano\, Department of Physics\, Milan\, Italy)\nThe process of globalisation has shown that coun tries can play a key role on the international stage provided that their c itizens are given the opportunity to build a significant background in sci ence\, technology\, engineering and mathematics (STEM). One of the critica l issues regarding the low number of degrees in STEM faculties is the cons iderable dropout rate in the first years of higher education. In this cont ext\, Italy is no exception with about 20% of its university students who drop out within the first two academic years. In a recent study based on a dministrative data from the Italian Politecnico di Milano\, it has been po inted out that the most important factor which allows to predict Politecni co di Milano student dropout is the number of university educational credi ts gained across the first term of the first academic year. Since active m ethods employed in academic courses appear to enhance students’ learning more than traditional lectures even in the context of large size classes\ , their use could lead to a decrease in the dropout rate. In the academic year 2021-2022 we carried out a case study which involved about two hundre d freshmen attending the “Fisica Sperimentale A+B” course at Politecni co di Milano. In addition to traditional lectures and drills\, seven peer learning sessions were offered to these students. During each peer learnin g session\, the learners answered a questionnaire consisting of three mult iple choice items based on some Physics topics and whose provision was imp lemented by using the students’ response system Socrative. Immediately a fter the questionnaire\, freshmen in the classroom would discuss the quizz es in small groups for few minutes. At the end of this debate\, they retoo k the same questionnaire. Finally\, the instructor briefly illustrated the correct as well as the incorrect alternatives of each item and the percen tage of answers ascribed to each possible option were shown to the student s. Considering that some students attended that course in person while oth ers attended the lessons on line\, we had both an experimental and a contr ol group. Their initial knowledge in Physics was checked and compared thro ugh a questionnaire. This instrument was administered at the beginning of the academic course and based on multiple choice items on some Physics top ics which were completely different from the ones employed during the peer learning sessions. In order to evaluate the effectiveness of this educati on methodology\, we examined the freshmen’s achievement in their Physics course final examination during the first exam session. Our findings show that the success rate of the experimental group was higher than the contr ol group and this difference was statistically significant. Moreover\, the calculated effect size highlighted that the association between the final examination pass rate and the peer learning sessions attendance was relat ively strong. Furthermore\, we investigated the possible correlation betwe en the final examination pass rate and the number of peer learning session s attended by the students. On balance\, these results appear to confirm t hat our innovative educational methodology may be effective in the mitigat ion of the university students’ dropout rate.\n\nhttps://indico.unina.it /event/60/contributions/627/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/627/ END:VEVENT BEGIN:VEVENT SUMMARY:Artificial Neural Network Analysis of Physics and Engineering fres hmen FMCE responses DTSTART;VALUE=DATE-TIME:20230130T151500Z DTEND;VALUE=DATE-TIME:20230130T153500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-632@cern.ch DESCRIPTION:Speakers: Valentina Bologna (Physics Department\, University o f Trieste)\nWe administered FMCE concept inventory (Thornton and Sokolof\, 1998) at the beginning of college studies to young Physics and Engineerin g freshmen of the University of Trieste during the Academic Year 2021- 202 2. We wanted to spotlight if there were influences on the Physics knowledg e acquired involved by the context of instruction (Bao and Redish\, 2006). We provided the Italian translation of the Concept Inventory to administe r it to freshmen. The translation from the original English FMCE was perfo rmed using a conceptual translation model. To investigate conceptual coher ence\, we analysed the responses exploring if there was knowledge fragment ation in using the concepts\, thus looking for lack of conceptual change ( diSessa\, 1993\, 2014\; Vosniadou\, 1994).\nWe conducted two different kin ds of analysis. The first one was based on multivariate descriptive statis tics. Then we undertook an in-depth analysis based on the Artificial Neura l Network (ANN) method (Lamb et al.\, 2014). This versatile framework (Amo o et al.\, 2018) computes relationships based on the interaction of multip le connected processing elements (Lamb et al.\, 2014\; Pinkus\, 1999). We used ANN as an analytical data method to investigate possible statistical evidence of what descriptive analysis suggested. In ANN design\, we adopte d the Sanger rule\, also known as Sequential Principal Components Analysis \, which forces neurons to represent a well-ordered set of principal compo nents of the data set (Sanger\, 1989). Artificial Neural Network analysis supplied a deeper insight into data-set statistical correlation. The Artif icial Neural Network analysis validated what was depicted by descriptive s tatistical analysis. This gave us a noteworthy method to feature students ’ conceptual knowledge expressly about force and motion topics\, even if we need to observe a more representative sample for a more robust statist ical inference. One relevant result concerns the responses clustering acco rding to conceptual misunderstanding (Hammer\, 1998). This shows that most of the sampled freshmen begin their College studies in Physics and Engine ering still knowing force and motion concepts intuitive or naïve. Only a small group of students\, mainly belonging to the Physics freshmen group\, exhibit a robust conceptual building in the description of force/motion p henomena. This clearly occurs by probing the neural network and relying on how different groupings of students follow the answers identified. \nExpl oring the correlation between answers and past-curriculum\, we observed th at a long-exposure to physics curricular studies enacts a better conceptua l understanding\, as would be. What remarkable occurs is the negative tren d in students with a three-year past-curriculum. Even if they experienced Physics studies at a heightened performance of cognitive development (most ly between 17-19 years old)\, they would suffer the lack of conceptual cha nge in force/motion description. The difficulties are steady in students w ith only early two-year exposition to Physics studies. Finally\, by presen t-curriculum choice\, we confirmed the differences between students’ gro up affiliations. Students from the Civil\, Environment\, Electronic and In formatics Engineering group present more conceptual difficulties than Phys icists and Industrial Naval Engineers.\n\nhttps://indico.unina.it/event/60 /contributions/632/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/632/ END:VEVENT BEGIN:VEVENT SUMMARY:Combining qualitative and quantitative analyses to investigate the role of virtual simulations in the evolution of primary school students' mental models about electrostatics DTSTART;VALUE=DATE-TIME:20230130T153500Z DTEND;VALUE=DATE-TIME:20230130T155500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-629@cern.ch DESCRIPTION:Speakers: Giacomo Bozzo (Department of Biology\, Ecology and E arth Science. University of Calabria (UNICAL)\, Cosenza\, Italy )\nWe comb ined qualitative and quantitative analyses to investigate the role of virt ual simulations in the evolution of primary school students' mental models about electrostatics with respect to the target school scientific model o f microscopic charges by analyzing the answers of two groups of 9/10 years old pupils inside instructional sequences that combined two hands-on real (R) activities using balloons and jackets with the same two virtual activ ities (V) using PhET simulations. \n Five classes in three schools (a tota l of n = 83 students\, 43 males and 40 females) were chosen to carry out t he "first-real" (RV) sequence while five classes in three other schools (a total of n = 85 students\, 41 males and 44 females) carried out the "firs t-virtual" (VR) sequence. The two groups were organized so as to be homoge neous from an ethnic\, cultural\, social and economic viewpoint\, ensuring that the typology of students would not affect the results. \n Data colle ction was made through pre-observation and post-observation worksheets for each real and virtual activity concerning a descriptive dimension as well as an explanatory dimension\, leading to a total eight worksheets for eac h student. As for the descriptive dimension\, related to the multiple-choi ce question “What happens? Choose a description for the physical behavio r of the two objects”\, students’ answers were classified with respect to three categories: “Concordant”\, “Discordant” and “No Answer ”. The explanatory dimension\, investigated through the open question “Why does it happen? Justify your choice”\, required a qualitative and interpretative analysis of the texts and of the graphic representations p roposed by pupils. To classify students’ answers and establish a progres sion with respect to their degree of adequacy to the target model\, we def ined five categories associated to five levels from 0 (lowest adequacy) to 4 (highest adequacy) following an approach based on the empirical learnin g progressions.\n Quantitative data analysis was applied to compare the ef ficacy of the RV and VR sequences. In the case of the descriptive dimensio n\, we applied the chi-square test since our aim was to estimate whether t he difference between the percentages of concordant\, discordant and not g iven answers between the two sequences was statistically significant. In t he case of the explanatory dimension\, where we wanted to test the statist ical significance of the difference in the ranking of students’ levels f rom 0 to 5 between the two sequences\, we applied the Mann-Whitney U-test. In both cases\, the null-hypothesis was that the differences between the RV and the VR sequence were not statistically significant: the difference between the two sequences was considered as statistically meaningful when the probability that the null hypothesis was true (p-value) resulted lower than 5%.\n This mixed qualitative and quantitative approach allowed to sh ow that virtual simulations do improve the level of adequacy of students' answers to the target model\, but this improvement is not transferred to n ew phenomena\, so that the capability of relating the developed model to t he real world remained the same at the end of both sequences.\n\nhttps://i ndico.unina.it/event/60/contributions/629/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/629/ END:VEVENT BEGIN:VEVENT SUMMARY:Investigation of Systems Thinking Skills of students aged 11 to 14 years old DTSTART;VALUE=DATE-TIME:20230130T153500Z DTEND;VALUE=DATE-TIME:20230130T155500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-626@cern.ch DESCRIPTION:Speakers: Sara Zanella (Free University of Bozen)\nThe educati on of responsible citizens\, capable of becoming active members of society \, is one of the goals of the European Commission (Hazelkorn\, 2015). It a ims to create conditions for science education that foster students' under standing of the complexity of the world of which they are a part. Green Co mp\, a framework developed by the European Commission (Bianchi et al.\, 20 22) recognizes Systems Thinking as one of twelve competencies to increase and develop the knowledge and attitudes to work\, live and act in a sustai nable world. Systems Thinking is\, therefore\, an approach that enables st udents\, regardless of their future careers\, to develop the knowledge and skills to consciously take part in local and global challenges. As part o f my research project\, I\, therefore\, administered the Systems Thinking Assessment Italia (STAI) test to students in Lower Secondary School in the Province of Trento to investigate the Systems Thinking competencies of st udents aged 11 to 14. The STAI test is a translation and adaptation from G reek Cypriot to Italian of the Systems Thinking Assessment (STA) test (Κ ωνσταντινίδη\, 2015). It consists of 27 questions divided into four categories (elements in the system\, interactions in the system\, fl ows in the system\, dynamics in the system). Each category is further divi ded into skills\, the Systemic Thinker skills. The analysis covers a sampl e of 709 students from nine Secondary Schools (240 students in grade one\, 232 students in grade two\, 237 students in grade three). Rasch analysis\ , ANOVA on the sample\, ANOVA on the test\, and a frequency analysis show that the test in its entirety is appropriate for this age group but is una ble to discriminate the identified categories. Students show basic Systems Thinking skills that do not improve significantly over the three-year per iod\, confirming the lack of a program to develop these skills. This inves tigation is part of a doctoral project in its concluding stages entitled ' Systems Thinking in the Lower Secondary School: field survey in the Provin ce of Trento and intervention study'.\n\nhttps://indico.unina.it/event/60/ contributions/626/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/626/ END:VEVENT BEGIN:VEVENT SUMMARY:Open Research Using Invalsi Data And Multilevel Models DTSTART;VALUE=DATE-TIME:20230131T134000Z DTEND;VALUE=DATE-TIME:20230131T140500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-656@cern.ch DESCRIPTION:Speakers: Maraviglia Lorenzo (INVALSI)\nThe purpose of the se ssion is to show how\, thanks to the opportunities disclosed by the develo pment of practices related to the idea of Open Research\, it is possible ( even for an independent researcher) to address issues of great cognitive a nd practical importance through the combination of high-quality open data\ , accessible computing tools and advanced statistical models.\nSpecificall y\, the data taken into consideration are INVALSI open data on students’ mathematic skills\; the data manipulation resource is R\, a high-level pr ogramming language whose interface can be freely downloaded and installed by everybody\; the statistical tools belong to the family of multilevel re gression models\, which can now be implemented through very sophisticated Bayesian algorithms.\nThe combined use of data\, resources and models will be illustrated and developed in connection with the treatment of a very d elicate problem: the possibility that\, in our country\, students with a m igrant background may suffer a penalty which is not justified by their act ual level of skills and abilities but\, rather\, depends on some sort of b ias rooted in the evaluation systems implemented by the schools.\nThe topi c is of particular interest since the share of foreign students (or studen ts with a migrant background) is expected to increase sharply in the next decade. To give adequate relevance to the issue\, it must be observed that in our Country schools are the main – some would say the only – agenc y for the integration of immigrants. Therefore\, the perspectives of our s ociety crucially depend on how efficiently schools will be able to carry o ut their tasks.\nThe INVALSI data allow us to approach the problem of scho ols’ fairness in an objective way. The standardized nature of INVALSI t ests helps us to try to identify the presence of potential bias in the ove rall evaluation system. A possible research question can be formulated in these terms: Invalsi scores being equal\, do Italian and foreign students receive (approximately) the same marks by their teachers? \nThe INVALSI sc ore is aimed at assessing fundamental abilities – for example general ma thematic skills – thorough the administration of standardized tests\; on the other hand\, marks are the outcome of a complex evaluation carried ou t by teachers – a kind of assessment that considers many other factors\, for example behavioural ones. Any discrepancy between standardized scores and marks could be a sign of limitations in the former\, which is plausib le\, but also a clue of some bias in the way schools evaluate students wit h a migratory background (which cannot be excluded on an a priori base: af ter all\, despite a growing number of foreign students almost 100% of our teaching body is made by Italian teachers).\nIn order to address these iss ues\, and provide a contribution to the discussion\, it is necessary to ap proach the data with tools that allow us to extract as much information as possible from them\, without introducing further distorting factors. This is essentially an open process\, which must be subjected to strict critic al supervision and requires transparency\; the tools and techniques that w ill be illustrated seek\, precisely\, to build a transparent path to open discussion and criticism.\n\nhttps://indico.unina.it/event/60/contribution s/656/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/656/ END:VEVENT BEGIN:VEVENT SUMMARY:A Study Of Gender Differences In The Selection Of Stem Courses At University With Large Scale Assessment Data Of Invalsi DTSTART;VALUE=DATE-TIME:20230131T131500Z DTEND;VALUE=DATE-TIME:20230131T134000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-655@cern.ch DESCRIPTION:Speakers: Patrizia Falzetti (INVALSI)\, Patrizia Giannantoni ()\nIn the most recent years an effort of INVALSI has been addressed to th e match of its large scale population data with other sources of administr ative/survey data. Technical difficulties and privacy limitations made thi s process complex and challenging\, however\, a first linked dataset has b een recently developed joining together data from INVALSI\, MIUR and Unive rsity Register . This joint dataset allows researchers to have a wide set of panel data in which each student is followed from primary school until university enrolment and final exam. \nAbout 248.000 university students e nrolled in 2019/2020 are retrospectively match with their INVALSI test in the previous year (2018/19) and with Ministry of Education Dataset in the same year\, with a rate of perfect match of 96%.\nThe whole collection of data in the joint dataset included: \n• basic anagraphic data\, such as gender\, migration background\, socio economic status as well as parental education (INVALSI dataset\, validated through MIUR data on the same stude nts)\n• Performance at cognitive tests (INVALSI dataset)\n• Characteri stics of the school career and performance of the students: typology of hi gh school\, regularity\, teachers’ marks (INVALSI dataset)\n• Informat ion about university career: year of enrolment\, class of discipline\, dat e and mark of final examination\nIn this work we present preliminary resea rch lines using the joint dataset to gain a better understanding of gender inequalities in the choice of STEM at university. The reference populatio n is a subgroup composed only by students attending a scientific lyceum in grade 13 of 2018/19\, who are supposed to pursue with a scientific discip line at university. \nThe study delineates the significant differences be tween girls and boys in the selection of STEM (42\,6% versus 54\,6%)\, wi th a difference of about 20% in favour of boys.\nThe most analytical part of the research deals with models that aim to evaluate relevant factors fo r choosing STEM at University\, separately for girls and boys. We built up logistic regression models\, stratified for gender\, and estimated the im pact of factors pertaining to the demographic\, socio-economic and school/ skills domains. \nIn the development of these models\, we address and disc uss some methodological issues\, about multicollinearity of socio-economic variables and selection of best model with AIC criterion. \nOur research highlights that cultural factors\, i.e. mother’s educational attainment have a stronger positive impact in the choice of STEM compared to socio-ec onomic factors. In a similar fashion\, in the selection of a highly scient ific course\, the mark in Mathematics given by teachers is more influentia l than standardized test scores obtained in the INVALSI test\; with a pred icted probability of choosing STEM increasing from 0.30 to 0.62 passing fr om weak to excellent mark in Mathematics.\n\nhttps://indico.unina.it/event /60/contributions/655/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/655/ END:VEVENT BEGIN:VEVENT SUMMARY:Measuring the impact of changing the formulation of a mathematical task DTSTART;VALUE=DATE-TIME:20230130T110000Z DTEND;VALUE=DATE-TIME:20230130T111500Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-639@cern.ch DESCRIPTION:Speakers: Giorgio Bolondi (Free University of Bozen-Bolzano)\n Learning achievements are usually assessed by administering tasks to stude nts- ranging from individual assessment to Large-scale surveys. Task desig n and administration are key issues in an assessment process. In the last decades\, quantitative research investigated systemic changes in assessme nt modality\, such as how the switching from a paper-and pencil administra tion of a task to a computer-based testing impacts on students’ answers\ , or from static to interactive administration. Less research has been don e at item level on the impact of changes in the formulation of a task.\nSt udents’ answers to an item depend on a very complex system of relationsh ips between the elements of the formulation\, personal features\, teachers ’ style\, and the learning noosphere. It is a somehow chaotic system whe re no direct cause-and-effect relationships are usually clearly identifiab le. A classical research trend in mathematics education is how the linguis tic formulation of a word problem impact students’ answers. This issue\, of course\, has many significant consequences\, for instance\, on how ite ms administered in international surveys are translated into the different languages.\nQualitative research investigated how a change in the formula tion of a problem may actually change the problem itself\, for a particula r student. Several theoretical frameworks are available\, based on qualita tive evidence\, but measurement is rarely used in Mathematics Education st udies\, since the optimal situation is difficult to achieve. It is of cour se impossible to administer two different versions of the same task to the same student\, since he/she should “forget” to have answered to that task the first time\, before answering to the second version.\nTo overcome such a limit\, we developed and administered four anchored Mathematics ac hievement tests\, consisting of two sections. The first one was given by a core set (named Core Test) of common items (used to anchor tests) and rep resenting the entire achievement test from both a statistical and mathemat ical point of view (i.e.\, both in terms of estimated parameters and mathe matics items’ content). Compared with previous studies in Mathematics Ed ucation\, our “Core Test” consists of items from Large-scale assessmen t\, each of which administered by INVALSI to the entire Italian students ’ population in previous main studies. Results from data analysis confir med the “robustness” of our Core Test that can thus serve both as a be nchmark and as a definition of the latent trait. This measurement step can be inserted both in explanatory and in exploratory plans. The second sect ion of each achievement test was given by alternative formulations of the same stem-items: each stem-item was modified by performing on it a single\ , well-individuated variation\, thus resulting in a quasi-experimental des ign which can detect emerging phenomena due to changes in formulation (hen ce overlapping this difficulty at least at a systemic level). \nWe present examples of application of this methodological strategy\, showing how it may contribute to better define the summative aspects of assessment\, help in interpreting students’ behaviors when answering to an assessment que stion\, thus giving a contribution to formative assessment\, and help task -assessment designers.\n\nhttps://indico.unina.it/event/60/contributions/6 39/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/639/ END:VEVENT BEGIN:VEVENT SUMMARY:Spatial abilities and the gender gap in mathematics (Matabì) DTSTART;VALUE=DATE-TIME:20230130T104000Z DTEND;VALUE=DATE-TIME:20230130T110000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-633@cern.ch DESCRIPTION:Speakers: Barbara Romano (Fondazione Giovanni Agnelli)\nLearni ng mathematics is a general problem in Italy\, but it is more relevant for females. Male students typically outperform their female classmates in ma ths test scores from the earliest years of schooling\, and the gap worsens as school grades progress. \n\nThe different experiences and expectations about potential abilities of boys and girls are key to understanding the possible causes of the observed relative differences in education performa nce. Early life experiences are essential for developing a child’s cogni tive capacities\, and environmentally induced gender differences may arise when children are exposed to heterogeneous sources of development opportu nities. One of the sources of gender inequalities in mathematics is relate d to differences in the acquisition of visuospatial abilities between girl s and boys from a very young age.\n\nExisting works find that playing with specific toys facilitates learning in maths and science. Boys usually gai n more experience than girls because of different parental and educators ’ beliefs and behaviour regarding the gender-specific suitability of toy s. Moreover\, self-confidence and anxiety are not gender-neutral and can a ffect educational outcomes. \n\nThe Matabì project aims to enhance spatia l abilities and reduce the gender gap via construction play (i.e.\, by usi ng building toys and Lego Duplo brick sets). Using the bricks should help students process abstract concepts\, while the playful approach should red uce maths anxiety. Girls should benefit more from Matabì than males\, who are (on average) able to develop these skills early on in life. \n\nThe p roject started in October 2022\, so we would present the overall structure and what has been accomplished so far. This should be engaging in three r espects: the teachers' reinforcement of their spatial abilities\, the inst ruments used to collect the outcomes\, and the impact evaluation design co nducted with Randomized Control Trial (RCT).\n \nTeachers’ training incl udes a pre- and post-standardised test\, the mental rotation section of th e Revised Purdue Spatial Visualization Test.\nThe RCT involves around 60 c lasses in five schools in Torino (60 teachers and 1200 third and fourth gr ade pupils). Within each school\, we randomly assigned a third of the teac hers to the control group and two-thirds to the treatment group. \n\nThe s tudy will provide evidence of the effects of teaching methods exploiting c onstruction play. The primary research questions are:\n\n 1. What is the i mpact on the spatial abilities of teachers? \n 2. What is the impact on t he spatial abilities of pupils? \n 3. How does the impact differ by gende r?\n 4. How does the treatment affect the gender gap in mathematics?\n\nht tps://indico.unina.it/event/60/contributions/633/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/633/ END:VEVENT BEGIN:VEVENT SUMMARY:Do the epistemological bases of qualitative and quantitative parad igms in STEM education still hold today? DTSTART;VALUE=DATE-TIME:20230130T102000Z DTEND;VALUE=DATE-TIME:20230130T104000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-638@cern.ch DESCRIPTION:Speakers: Martina Caramaschi (Department of Physics and Astron omy - University of Bologna)\nRecently\, the Physical Review Physics Educa tion Research journal organised two calls for papers focused collection to “examine and challenge” quantitative (2017) and qualitative methods ( 2021) in that field. Both calls had a subtitle\, “a critical examination ”\, which foretells the need to focus on these two methodological paradi gms\, not taking them for granted\, inviting to reposition them in today ’s panorama.\n\nThe division of research into these two methodological p aradigms originates in the philosophical debate\, occurring at the end of the nineteenth century\, which aimed to confront the nature of humanistic knowledge with the scientific one. \nDespite its epistemological origin\, the debate between qualitative and quantitative approaches within the rese arch in science and STEM education focuses mainly on pure methodological a spects:\n\n -What methodological approaches are best suited to answer cert ain research questions?\n\n -Which methods could be used with certain type s of data?\n\n -What methodological approaches could best guarantee specif ic characteristics of the results (e.g. explaining a cause-effect relation ship\, being objective\, specific\, general\, etc.)? \n\nThe aims of the p resentation are: to point out the epistemological assumptions that stay be hind the traditional qualitative and quantitative methodological approache s\; to argue why the big data revolution has been questioning the basic ep istemological assumptions that justified the distinction in the past\; to raise examples of foundational epistemological questions that can motivate the need to overcome the traditional dichotomy and ground the elaboration of new “mixed” methods. \n\nThe first aim is pursued by presenting th e results of the historical debate on the methods in terms of its ontologi cal and epistemological pillars: the view of reality that justified the di stinction between natural and human realms\; the “normative or idiograph ic” nature of the laws studying a phenomenon. Then\, we will show how th e different possible ontological and epistemological positions influenced the methodological principles\, chosen by the different traditions to make knowledge reliable\, general\, robust\, effective\, objective and “true ”. \n\nThe second aim is pursued by discussing some examples taken from research in the way of science education of how sometimes this dualism doe sn’t seem to hold\, or at least doesn’t differentiate to make sense. T he advent of online learning platforms and extensive use of technologies i n classrooms (e.g. sensors) leads to big data about students’ behaviour being available in educational research. These particular cases raise epis temological and ontological questions about which reality we are looking a t and how we could stand toward this reality (made of real and virtual one s). Different ontological attitudes are emerging\, for example in the EDM \, and LA communities\, and in Informed ML. Moreover\, the use of big data \, extended and rapid by nature\, in the context of STEM education researc h\, which is specific and detailed\, makes qualitative or quantitative app roaches not completely suitable.\n\nFinally\, questions are raised to stim ulate reflections to re-think about the epistemological approaches to meth ods in the field of STEM education such as “what new methodological para digms are needed in order to deal with these research changes and meaningf ully characterise them?" and “Can we still talk about natural settings i n modern learning environments?”.\n\nhttps://indico.unina.it/event/60/co ntributions/638/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/638/ END:VEVENT BEGIN:VEVENT SUMMARY:Why does Integrated STEM education not fit: Pinpointing the breaki ng points. DTSTART;VALUE=DATE-TIME:20230130T100000Z DTEND;VALUE=DATE-TIME:20230130T102000Z DTSTAMP;VALUE=DATE-TIME:20260519T003925Z UID:indico-contribution-60-631@cern.ch DESCRIPTION:Speakers: Jessica Niewint-Gori (Indire)\nDespite of countless financial resources at the European and Italian national level\, the avail ability of training and many best practices as teaching resources\, STEM e ducation struggles to be integrated in the school system in holistic way. Starting from a reflection on the key points and actors involved with STE M practices in schools this contribution intends to highlight the critical points in the educational system that may hinder the process of their int egration. A minimalist version of STEM integration would require the combi nation of at least two STEM subject areas STEM integration can also exte nd beyond STEM disciplines to incorporate art or any other subject and the interest and development on interdisciplinary STEM education is increasin g. STEM integration could be represented as a gradient\, which at one extr eme is defined by a multidisciplinary approach of two or more disciplines taught and learned separately but within a unifying theme\, and at the oth er end a transdisciplinary approach\, the learning goals of the different disciplines are designed through a learning scenario based on real-world p roblems. Yoder and colleagues define six key elements of a student-centre d and evidence-based educational STEM ecosystem including: effective teach ers\, high standards and quality curricula\, formal and informal learning of STEM subjects\, integrated use of technology. Aside for teaching resour ces\, teachers need to be trained in STEM pedagogical design\, implementat ion\, and evaluation. To improve teachers' self-efficacy in teaching STEM professional development programs also need to address teachers' needs for instructional practices and provide opportunities for peer collaboration. The Scientix report on educational practices in Europe disposes a differ ent picture. The responses show that for STEM subjects\, the frontal and t ransmissive lecture remains the most reported pedagogical approach and mos t STEM teachers interviewed have not undergone any professional training o f STEM subject-related methodologies or technologies. In daily practice al so the lack of time required for instructional design and pedagogical impl ementation and a formative-oriented integrated STEM approach which does no t always fit easily into the curriculum. To overcome these barriers a scho ol curriculum ready to promote an integrated STEM approach and a shift to distributed leadership to support and promote STEM practices and teaching at the whole-school level could enable teacher collaboration and stakehold er engagement in STEM education. The school leadership should support and promote a professional development with collective and active participatio n\, consistent with the guidelines and curriculum of an extended duration to ensure accompaniment in the application of new skills in teaching pract ice\; as the active collaboration with the territory\, like industry or mu seums\, in classroom activities\, to contextualise the learning content to a real-world application. Policy makers should encourage professional dev elopment of teachers and school leaders\, as foster the introduction of al so student-centred methodologies and formative assessment in learning.\n\n https://indico.unina.it/event/60/contributions/631/ LOCATION:Complesso S. Marcellino e Festo G4 URL:https://indico.unina.it/event/60/contributions/631/ END:VEVENT END:VCALENDAR