Overview and details of the sessions of this conference. Please select a date or location to show only sessions at that day or location. Please select a single session for detailed view (with abstracts and downloads if available).
Convenor(s): Nadia Sansone (UnitelmaSapienza University of Rome, Italy)
Presentation 5
Metacognitive AI Literacy Across Contexts: Empirical Evidence from the rAIse Framework
Nadia Sansone
UnitelmaSapienza University of Rome, Italy
The widespread adoption of generative AI in educational and professional contexts has outpaced the development of frameworks supporting critical, metacognitive engagement. Most users approach conversational AI as a digital oracle — seeking speed and immediate output — rather than as a dialogical partner for meaning-making. This pattern mirrors a structural problem: the very affordances that make generative AI powerful (fluency, responsiveness, apparent authority) also make it prone to fostering cognitive delegation rather than reflective thinking. This paper presents rAIse (Reflect-Ask-Interpret-Source-Evolve), a five-phase framework for conversational AI literacy developed within the AI4E Laboratory at UnitelmaSapienza, and reports empirical evidence of its effectiveness across multiple learning contexts. rAIse is grounded in five foundational theoretical traditions: Flavell's metacognition, Vygotsky's Zone of Proximal Development, Bakhtin's dialogism, Schön's reflection-in-action, and Freire's generative pedagogy (Evolve). Each phase is operationalized through stimulus questions that slow down and structure human-AI interaction, shifting engagement from mechanical to strategic and from passive to reflective. The framework explicitly repositions AI not as an authoritative source but as a co-constructor of knowledge requiring active critical participation from the user — a dialogical partner rather than an oracular authority. A pre-post single-group design was conducted across multiple professional and educational contexts — university students, in-service teachers, private sector professionals, and public administration officers — through the AI4E Laboratory at UnitelmaSapienza. A total of 657 participants completed the rAIse process module and an accompanying validated questionnaire (24 items; Cronbach's α = 0.826 pre, 0.827 post). Two conditions were compared: facilitated workshops (2–4 hours, with theoretical introduction and guided application) versus fully autonomous self-application without prior training or facilitation. Four AI literacy competencies were measured via five-point Likert scales: use of AI for professional support, use of AI for information research, critical analysis of AI outputs, and ethical awareness. Wilcoxon signed-rank tests revealed large effect sizes across all four competencies (r = 0.619–0.838, p < .001). A counterintuitive and theoretically significant finding emerged: participants who applied rAIse autonomously — without any prior facilitated training — showed systematically greater improvement than workshop participants, with the most pronounced difference in ethical awareness (+0.123 effect size). This finding suggests that the framework possesses intrinsic self-scaffolding properties: learning occurs through structured doing rather than through listening to explanations, positioning the AI conversation itself as the formative environment. Moreover, participants who completed the EVOLVE phase reflective item demonstrated significantly higher gains in critical analysis (U = 47566, p = .044), confirming that metacognitive writing is constitutive to the learning process. Reduced post-intervention standard deviations across all competencies further indicate that rAIse reduces initial variability, functioning as an equalizing scaffold within heterogeneous groups. Three conclusions emerge. First, rAIse is effective transversally across diverse professional and educational profiles, from primary teachers to public sector officers. Second, traditional didactic mediation is not a prerequisite for its effectiveness: the framework's structure is sufficient to generate meaningful learning independently. Third, the challenge of AI literacy is ultimately cultural rather than technical — it requires reclaiming metacognition, slowness, and democratic agency as non-negotiable dimensions of human-AI interaction in educational settings.