Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).
Convenor(s): Leonardo Piromalli (Iref – Istituto di Ricerche Educative e Formative, Italy)
Presentation 2
The Influence of Algorithmic Bias and Autocomplete Predictions of AI Systems in Education (AIED) on Critical Thinking and Students’ Epistemic Autonomy
Agnese Addone1,2, Pietro Boccadoro3,4
1: Lab. Informatica&Scuola CINI, Rome, Italy; 2: Institute for Globally Distributed Open Research and Education (IGDORE), Rome, Italy; 3: IEEE member, Bari, Italy; 4: Nextome srl, Bari, Italy
This study investigates the relationship between human agency and Artificial In- telligence (AI) systems, focusing on Artificial Intelligence in Education (AIED) dynamics and the growing crisis of technological integrity. As generative tools become part of everyday life, a significant threat to independent thinking arises from the hidden biases within autocomplete predictions ([8, 9]). These suggestions are the real-time word or phrase hints provided by an AI as a user types. Far from being neutral aids, they are weighted by the biases of their underlying training data. In education, AI may systematically steer students toward specific cultural perspectives, gender stereotypes, or Western- centric worldviews ([1]). When interacting with these systems, learners encounter a subtle but persis- tent form of algorithmic nudging, leading to subconscious adoption of both the logic and lexicon of these biased suggestions. This gradually reshapes a student’s original intent, presenting a critical challenge for educators: ensuring that the AI does not lead to standardized thinking, nudging the next generation towards biased, polarized or ethically compromised worldviews ([4, 5, 6]). Given the recent geopolitical landscape, the integration of Large Language Model (LLM)s into national defense infrastructures and the resulting regulatory pressures on private AI labs, (e.g., the ”supply chain risk” designation of entities resisting military de-alignment). this study analyses the socio-pedagogical impact of autocomplete bias. As training datasets may move toward a ”war-centric” ori- entation, the predictive nature of AI may cease to be a neutral utility, and may be a vehicle for cognitive heteronomy ([2, 7]). The analysis identifies three vectors of technological misuse and their impact on the educational landscape: • Dangerous Data Polarization and the Martial Shift: With given opti- mizations, a martial bias turns creative tools into devices for normalized conflict. These algorithms trap students into narrow, reactive thought echo-chambers. • From Surveillance to Cognitive Homogenization: The black-box implemen- tation of predictive analytics for student grading and the deployment of invasive biometric surveillance in schools are often framed as optimization. Learners are considered as data points to be managed rather than subjects to be nurtured. • Restoring the Ethical Covenant in Pedagogy: The educational stakehold- ers cannot effectively engage with tools appearing clandestine vehicles for state propaganda or militarized logic. The paper posits that trustworthy and ethical AI is actually a pedagogical requirement for democracy. In conclusion, this work advocates for a Sovereign Ethical AI in Education, denouncing the risk of tools that may transform into silent devices for martial indoctrination ([3]). It also calls for schools to become sites of Critical AI Literacy, where polarized biases of modern technology are deconstructed. The transparency and ethical non-negotiability demand can ensure that technology remains an instrument for human flourishing, rather than a silent and polarized architect.
