Conference Program

The integration of generative AI in higher education demands a fundamental shift in focus: from what AI produces to how students interact with it. Without deliberate metacognitive scaffolding, conversational AI risks reinforcing cognitive delegation rather than developing critical judgment. This exploratory mixed-methods study examines whether a structured e-tivity embedding the rAIse framework (Reflect-Ask-Interpret-Source-Evolve) as a procedural scaffold can transform AI conversation into a formative and self-regulatory practice aligned with Assessment as Learning principles (Earl, 2003). Two research questions guided the study: RQ1 — To what extent do metacognitive regulation strategies emerge, particularly in self-assessment and revision phases? RQ2 — What interaction profiles emerge from conversational logs, and how do these associate with the degree of alignment between perceived and enacted competence, as revealed by peer feedback?

17 university students (online degree programme in Psychological Sciences, UnitelmaSapienza) participated in an e-tivity titled "Learning to learn with AI," conducted across three synchronous webinars between October and November 2025. In the first webinar, students were introduced to rAIse through live practice and submitted a baseline evaluative prompt formulated without any structured approach, serving as the analytical reference point for measuring subsequent changes in prompt quality and interaction strategy. In the second webinar, students applied rAIse to co-construct an assessment rubric with AI and use it to evaluate their own work through motivated, iterative revision — directly reformulating their baseline prompt using rAIse phases and introducing explicit criteria, meta-prompting, and regulation practices. In the third webinar, each student analyzed an anonymized peer's conversational log using a rAIse-derived coding grid, providing written feedback on prompt quality, cognitive function attributed to AI, and regulation strategies observed; collective debriefing enabled students to confront the gap between perceived and enacted competence. The dataset comprises 68 chat logs, rAIse pre/post questionnaires (four AI literacy dimensions, Likert 1–5), and structured peer feedback forms.

Logs were analyzed using ordinal indicators of prompt quality (0–3), cognitive function attributed to AI (0–3, from substitutive to critical-regulative), transformative iteration level (0–3), and regulation markers including meta-prompting, motivated output critique, and validation requests. Perceived-enacted alignment was estimated by triangulating pre/post questionnaire delta scores with peer feedback evaluations.

Addressing RQ1, metacognitive regulation emerged in three recurring strategies: regulation for clarity (contesting vague AI-generated criteria), regulation for control (meta-prompting, inverting the standard dialogic flow), and regulation for refinement (negotiating content through motivated rejection of AI proposals). Addressing RQ2, five interaction profiles emerged along a delegating-to-regulative continuum: Reflective Learner (29%), Strategic Architect (18%), Task-Oriented (18%), Pragmatic (23%), and Surface Learner (12%). Surface profiles systematically overestimated their competence while reflective profiles showed close alignment between questionnaire delta scores and log-observed strategies. The baseline-to-rAIse comparison further confirmed that structured scaffolding produces qualitative shifts in prompt construction and interaction governance.

These findings yield three design implications: explicit criteria must be embedded structurally in tasks; verification and triangulation must be required and assessed; peer analysis of conversational logs creates a metacognitive mirror unavailable through individual reflection alone. Epistemic responsibility remains with the learner — and the design must structurally demand it.