As Soon As Possible - The Effectiveness Of A Financial Education Program In Italian School
Tommaso Agasisti1, Alessio D'Ignazio2, Gabriele Iannotta1, Angela Romagnoli2, Marco Tonello3
1Politecnico di Milano, Department of Management, Economics and Industrial Engineering; 2Bank of Italy, Directorate General for Consumer Protection and Financial Education; 3Bank of Italy, DG for Economics and Statistics, Venice Branch; GLO and CRILDA-Catholic University Milan
This paper evaluates the effects of “Financial Education in School” (FEiS), a large financial education program offered by the Bank of Italy from primary to secondary schools nationwide, which consists of teachers’ training, dedicated materials and classroom activities.
The rigorous evaluation of classroom financial education programs is challenged both by heterogeneity across different schools, school grades and students characteristics and by self-selection into treatment. To tackle such potential sources of endogeneity, we ground our empirical strategy on two RCTs, concerning primary school and lower secondary school students, respectively.
For conducting the empirical analysis, we exploit a rich dataset taking from two main sources: (i) student questionnaires on financial literacy, administered both before and after the FEiS program; (ii) student characteristics, including demographic variables and scores in both maths and reading tests, drawn from the National Institute for the Evaluation of the Education and Training System (INVALSI). In addition, we also exploit some information on teachers-level characteristics, gathered by means of non-mandatory questionnaires; the latter were available for a subset of classes only.
By means of two Randomized Controlled Trials, which involved more than 1,500 grade 5 and grade 8 students. We find that the program improves students’ financial knowledge by between 0.5 and 0.6 SD for both school grades, a result in line with previous research. On the other hand, no impact is detected, on average, for students that were invited to study the volumes on their own, at home. Investigating heterogeneous effects, student’s family socio-economic status (ESCS) matters; in particular, students with higher-than-the-median ESCS experience larger benefits from the Bank of Italy program and improve their financial literacy also when studying on their own.
The Lasting Heritage of Improved Teacher-Student Relations in Middle School
Gianluca Argentin1, Giulia Assirelli2, Tiziano Gerosa3, Matteo Moscatelli2
1University of Milan-Bicocca, Italy; 2Catholic University of Milan, Italy; 3SUPSI, Switzerland
Existing research on school effectiveness shows that teachers play a pivotal role in determining student achievement, significantly influencing their performance (Wayne and Youngs, 2003; Rivkin et al. 2005; Creemers et al. 2010). However, prior studies failed in explaining the sources of this effect as they were not able to identify specific teachers’ characteristics that determine their effectiveness. Consequently, teacher quality remains a black box, posing challenges for the development of policies aimed at increasing it (Rice, 2003; Kini and Podolski, 2016).
With the aim of addressing this gap, the OpenTeQ project (“Opening the black box of teacher quality”) adopts a novel approach – that integrates economical, sociological and pedagogical perspectives – to test in a rigorous way whether teachers’ relational skills contribute to their effectiveness and whether these skills can be enhanced through large-scale interventions.
To reach this goal, during the school year 2016/2017 we implemented a light-touch professional development intervention focused on the improvement of relational skills. This intervention was targeted to lower secondary school teachers in Italian schools. To evaluate the impact of the intervention we designed a cluster randomized controlled trial. Teachers of the treatment group were exposed to the self-training programme, which was based on a brief booklet and short videos offering practical tips to better managing relationships with colleagues, students and parents.
In the short run, the effectiveness of the intervention on teachers’ and students’ attitudes and on pupils’ achievement was assessed through a) data collected ad hoc at the end of the school year (7th grade), b) school administrative data (marks registered at the end of the following school year and competences measured through standardized tests). Results showed that the intervention was effective in producing a statistically significant impact on teachers’ self-efficacy, that is a relevant predictor of students’ achievement. Moreover, our treatment also improved students’ interest and achievement in the main subjects (Italian and mathematics) (Argentin et al., 2020).
Now, thanks to the availability of administrative data on students’ educational career, we have the opportunity to assess the long-term effect of the intervention. Students are indeed followed until the school year 2022/2023, that for regular students is the end of the upper secondary school cycle. The collected data are upper secondary track, marks and failures at the end of each school year.
Preliminary results show that the light-touch intervention developed during the lower secondary cycle still exerts its effect in the long run: treated students are found to be more often enrolled in the academic track (i.e. licei) and to better perform in the main subjects, as they register higher marks in Italian and Mathematics. These results not only confirm that teachers’ relational skills are a relevant leverage for teachers’ effectiveness, but also that light touch interventions can produce relevant long-lasting impacts.
Experimental Effects of an Education Savings Program to Tackle School Dropout
Davide Azzolini1, Loris Vergolini2
1Fondazione Bruno Kessler, Italy; 2Università di Bologna, Italy
We conduct a randomized controlled trial to test the effectiveness of a savings-based financial aid program for high school education in Italy. The program (WILL) was implemented by a consortium of philanthropic organizations as part of a nation-wide demonstration in four Italian areas (Torino, Firenze, Teramo, South Sardinia) from 2019 to 2023. The program targeted first-year students of lower secondary schools from low-income families and offered them and their families the opportunity to save small amounts of money (6 euros per week up to a maximum of 1,000 euros over a 4-year period) in a digital wallet. Household deposits were multiplied by four if the money was spent on proven educational expenses (e.g., computers/internet; culture, book purchases; various school expenses, language or computer courses, sports, transportation). In addition to the savings account, beneficiaries were offered a financial education program, educational support and guidance.
The study involved 576 students (including 293 treated and 283 control). Data were collected approximately 20 and 36 months after randomization. Administrative data from the program indicate that families saved regularly (on average, 5.2 euros per week, totaling 598 euros) and spent an average of 2,320 euros.
We find that the program has had a positive effect on family savings (for their children's education) and that the increased family savings did not come at the expense of the families' own ability to meet subsistence expenses (e.g., food, rent, medical expenses, etc.). However, the data also show that households with lower incomes (ISEE index) show markedly lower levels of saving and spending than households with higher ISEE.
The increased savings and matched moneys provided by the program had substantial effects on the computer equipment (i.e., greater likelihood of having a dedicated computer and better internet connections) that students were able to use to cope with distance education during Covid.
On average, there is no impact on middle school final grades, but a large positive effect is observed among low-income individuals. A positive effect is evident in terms of school attendance, including reduced instances of skipping entire school days or missing some classes and arriving late to school.
No effects of program participation are found on educational aspirations and expectations. However, among families with lower ISEE, that is, families with lower educational aspirations and expectations for their children, the program had a strong positive effect, raising by a substantial extent parents' desire and concrete belief that their son/daughter will successfully complete secondary school.
A third wave of data collection is scheduled for Fall 2024, which will enable us to estimate the program’s effect on dropout during the first two years of high school.
Based on our and previous findings (Martini et al 2021), we conclude that matched savings accounts show potential for combating educational inequality in Italy. The program is replicable and scalable--a similar program is currently being run in Torino. Regressivity in the financial mechanism, due to limited savings capacity of families, could be redressed by implementing marginal changes in program design.
Evidence-Based Reform of Educational System in French-speaking Belgium: Reduction of Grade Retention Rate
Dylan Dachet, Ariane Baye
University of Liège, Belgium
In 2015, urged on by the worrying results of international surveys (Lafontaine et al., 2019 a & b; Quittre et al., 2017), the French-speaking Community of Belgium initiated a structural reform of its educational system (Le Pacte pour un enseignement d’Excellence). One of the original features of this reform is that it is the first (Dachet & Baye, 2020 & 2024) in the French-speaking countries to explicitly refer to the Evidence-Based Education paradigm in one of its founding texts (FW-B, 2015).
Among the many aspects of this structural reform, reducing the use of grade retention is undoubtedly the most “evidence-based” one (Allen et al., 2009; Bright, 2011; Goos et al., 2021; Valbuena et al., 2021). In Belgium, however, grade retention is often viewed positively by teachers and, more generally, in the public opinion. Besides, debates remain lively in the scientific community, with some researchers defending the value of grade repetition because of its perceived usefulness to teachers (Draelants, 2018 & 2019), while others point out the negative or null effects highlighted in the scientific literature (Galand et al., 2019).
At the international level, the debate on the effects of grade retention that might have thought to be closed has revived, since educational economists have shown, on the basis of discontinuity regression designs, some positive effects of grade retention in specific contexts (Özek & Mariano, 2023). At the same time, an analysis of the intrinsic quality of quasi-experimental studies on grade repetition has revealed some worrying findings regarding their risk of bias (Dachet, 2024): existence of confounding factors, low baseline equivalence, post-hoc assignment of students, use of over-aligned measuring instruments, high attrition rate, high differential attrition rate, low statistical power…
To date, previous work aimed at measuring the quality of research dealing with the effects of grade retention has not analyzed the quality of studies following alternative causal designs such as discontinuity regression designs. However, these are precisely the study designs that currently find the most positive effects of grade retention (Goos et al., 2021). Moreover, these are also research designs whose validity is widely debated in the scientific community (Cook, 2002; Huntington-Klein, 2021; Lee, 2016).
In this contribution, we propose an analysis of the quality of all available causal research on the effects of grade retention. We aim to provide an answer to the following question: based on the analysis of the quality of research on repetition, what can be concluded about its effectiveness?
Stimulating Creativity and Grit of High School Students with Creative STEM Activities: an RCT with Noncompliance
Martina Francesca Ferracane1,3, Veronica Ballerini2, Fiammetta Menchetti2, Silvia Noirjean2, Alice Dominici1
1European University Institute, Italy; 2University of Florence; 3Teesside University
Creativity and grit are widely recognized as vital components for the success and prosperity of individuals. Therefore, it is of paramount importance to set up interventions to foster them, especially among children and teens. We conduct a randomized controlled trial where students from five Italian high schools are randomly assigned to creative STEM courses. The courses are delivered by FabLabs, small-scale workshops that offer access to tools for digital fabrication and employa hands-on pedagogical approach expected to impact students’ creativity and grit positively. Assignment to the courses is at the class level: only students in classes randomly selected to join the Fablab activities can participate, and they may decide whether to enrol or not on a voluntary basis. We address noncompliance by adopting an instrumental variable approach. We use two modalities to measure creativity: a self-assessment through the Short Scale of Creative Self and an assessment made by an independent expert, who used an index of creativity developed by the authors. While the effect on self-assessed creativity is not significant, results show that the external, independent assessment of students’ creativity can capture a significant effect of Fablabs’ courses. It also emerges that FabLab activities have a positive significant effect on students’ grit, measures through the Duckworth scale.
|