Online searching can be viewed as a step in solving an information problem and includes query formulation, result examination, webpage selection, and the interpretation of retrieved information. Today's students use online search engines almost every day for schoolwork, projects, and personal curiosity, making online searching a central skill in both their learning and everyday life. However, many young learners still struggle with key parts of online searching. Students often have difficulty choosing effective search terms, revising or improving their queries, evaluating the reliability of sources, and managing the time they spend on webpages.

These challenges highlight the importance of studying students' online search behavior to identify which search behavior patterns support their search effectiveness. This focus is particularly relevant from an educational perspective. First, instructors need to know how students search before teaching or supporting the development of online searching skills. Second, little is known about which online search behaviors are truly effective for different kinds of information tasks, making it important to examine these behaviors.

To address these challenges, our study focuses on understanding how young students, aged 16 to 20 years, conduct online searches and examines their behavior using data collected through our Reflective Online Search Education (ROSE) platform (Botturi et al., in press). In the study, 305 students were assigned three online search tasks on different topics: Basil, Internet, and Vegan Diet. ROSE recorded their search stories, i.e., their navigation actions. After conducting a task, students complete a multiple-choice question with a single correct option, and their performance is scored using a four-level scheme that considers both the correctness of the answer and the reasoning behind it. By capturing each student's online search actions, the dataset enables us to describe their online search behavior. Based on this dataset, our study addresses four research questions (RQs):

RQ1: What distinct online search behavior can be identified in students' search stories?

To address this RQ, we analyzed students' online search behavior by distinguishing between their search engine behavior (actions within the search engine) and their document behavior (actions on results and webpages). We identified distinct patterns within each dimension and then combined them to derive an overall online search behavior for each student. Based on these patterns, we classified participants into distinct types of online search behavior.

RQ2: How do students' query behaviors (new, revised, and repeated queries) reflect their approaches to information seeking during the task?

To investigate RQ2, we computed for each student and task the proportions of new, revised, and repeated queries and averaged across tasks to characterize individual query behavior. A dominant query behavior was then identified for each student by determining which query behavior they used most often. This allowed us to describe how their query choices reflected different approaches to information seeking.

RQ3: To what extent do students' performance levels vary across tasks, and how are their online search behaviors and query behaviors associated with higher or lower performance?

To address RQ3, we analyzed students’ performance scores to examine how performance varied across tasks.

Online search behavior and query behavior were then examined in relation to performance. This allowed us to assess whether specific behaviors or query approaches were associated with higher or lower performance.

Results show that students do not rely on a consistent search behavior; instead, their search behavior varies depending on the task. This indicates that task characteristics strongly shape search behavior. No clear relationship was found between behavior and performance or between query and performance, suggesting that no single online search behavior or query behavior consistently leads to effective online searching.

Theoretical background & Research Questions

Information Literacy (IL) can be defined as the ability to think critically and make judgements about any information we find and use. Higher education students are the major beneficiaries of IL educational interventions, but most interventions are based on one-size-fits-all models (e.g., the Big 6), adapted to different subjects and contexts. The ROSE (Reflective Online Searching Education) approach was developed to foster students’ self-reflection on their own Online Searching (OS) skills. This is implemented through the ROSE platform (Botturi et al., in press), a digital tool which records the students’ actions on the browser during online search on given tasks, and represents them in visual interactive timelines, together with overall statistics of that search and suggestions designed to foster exploration of different search strategies.

In this study, we tested whether the ROSE approach and platform are effective for improving university students’ IL self-efficacy and self-reported IL skills comparing it with a direct instruction approach.

Methods

The sample (N=91) consisted of 49 first-year Bachelor students recruited from a Swiss Teacher Education University, and 42 first-year students from a Bachelor’s degree in Communication at a Swiss University. This was a convenience sample, and the interventions had to comply with different academic settings.

For the analysis, we created 4 groups:

- EGK: Experimental Group of pre-service Kindergarten teachers; NK = 19

- EGP: Experimental Group of pre-service Primary school teachers; NP=30

- EGS: Experimental Group of students in Communication; NS=19

- CGC: Control Group of students in Communication; NC=23

EGK and EGP students were split in 5 separate training sessions with around 10 participants each, while EGS and CGC attended larger sessions of about 50 students.

All sessions started with a pre-intervention test, which included demographic questions and self-report scales of IL self-efficacy and IL skills. Then, all students completed two online-searching tasks on the ROSE platform.

Thanks to a set of self- and peer-reflection exercises supported by the ROSE interactive timelines, the experimental groups (EGK, EGP, EGS; NT=68) followed a debriefing session focused on their own search queries, reading search engine results, use of time, and sources choice. The debriefing for the control group (CGC; NC = 23), instead, included group activities on the same topics but without the support of the ROSE platform, so with a direct instruction approach.

A post-intervention test in both conditions assessed changes in IL self-efficacy and IL skills using the same scales.

Results & Discussion

Descriptive statistics show self-efficacy scores to be overall higher in the post-test compared to the pre-test, with significantly higher scores in the post-test within EGS, EGP, and CGC, suggesting IL-self efficacy may increase regardless of the type of intervention.

However, descriptive statistics show very little increase in IL skills after the intervention in the pre-service teachers’ groups (EGP, EGK), and post-intervention scores seem to be even lower in the university students’ groups, suggesting rather a potential role of academic profile and of both interventions in altering university students’ overestimation of their own OS skills.

Such results suggest that the reflective approach can be at least as effective as a direct instruction approach, and that the academic profile of learners and the size of the class play a relevant role in the interventions’ effectiveness. Results should be considered with care, due to the small sample and the short (90’ to 120’) one-shot interventions. Further research about the impact of a reflective approach for OS education is currently underway.

1 Introduction

Children are searching the web by the time they reach elementary school. Although Search Engines (SE) play a central role in school-related search activities, they are often designed with adults in mind and do not necessarily account for children’s distinct search behaviors. Based on the concept of Search-as-Learning (SAL) [1], project SOL (Scaffolding to foster independence when children search Online for Learning, www.solandchildren.wordpress.com/) explores whether SE themselves can provide dynamic, developmentally appropriate guidance, particularly in the classroom. SAL conceptualizes information seeking as “a process, in which individuals purposefully engage to change their state of knowledge" [2], highlighting two inter-twined dimensions: learning to search while searching to learn. Although early SAL literature mentions non-typical user groups, such as children, most research works target or refer to studies involving adults. To better understand what children need for an effective SAL experience, SOL draws on the educational theory of scaffolding [3], enabling adaptive assistance that fades as independence grows. Therefore, we pose the following Research Questions:

1. What are the functionalities needed to scaffold children’s learning while conducting curriculum-related online inquiry tasks?

2. What kind of interfaces are needed to scaffold children’s learning while conducting such tasks?

With children’s abilities and expectations greatly varying as they grow, to control scope, we use four pillars, or scope compass, that guide the design and evaluation of Information Retrieval (IR) systems for children: user group, task, environment, and search strategy. In our case, children ages 9 to 12, conducting online information discovery tasks related to the learning environment, supported by scaffolding.

2 Research Objectives and Methodologies

SOL pursues three objectives. First, to understand children’s existing search practices in the classroom, a learning environment where children routinely search for and use information Through observations, query log analysis, and participatory design, we identify navigation strategies children adopt with different Information Access Systems (IAS) including SE and agents based on generative AI; the systems’ emotional reaction to children’s inquiries; and the risks that information disorder poses to children’s online learning Second, we translate these needs into design concepts that extend or complement existing IAS. These may include implicit supports integrated into SE or explicit scaffolds visible to users. Third, we iteratively prototype and evaluate SOL through usability studies, measuring whether scaffolds improve relevance judgment, source reliability, and knowledge articulation. Our methodology combines methods from Human-Computer Interaction, IR, and Education, accounting for the role of educators and classroom dynamics.

3 Conclusion

SOL revisits scaffolding for digital inquiry by embedding guidance directly into the search experience. It advances educational research by clarifying how children learn to search while searching to learn. Additionally, it contributes to design research by introducing developmentally aligned search technologies, fostering digital literacy, and critical thinking. Finally, SOL positions children as design partners, ensuring that future IAS empower them to develop search autonomy over time.

Bibliography

[1] Gwizdka, J., Hansen, P., Hauff, C., He, J., Kando, N.: Search as learning (sal) workshop 2016. In: Proceedings of the 39th International ACM SI-GIR Conference on Research and Development in Information Retrieval. p. 1249–1250. SIGIR ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2911451.2917766, https:

//doi.org/10.1145/2911451.2917766

[2] Marchionini, G.: Information Seeking in Electronic Environments. Cam-bridge Series on Human-Computer Interaction, Cambridge University Press (1995)

[3] Walqui, A.: Scaffolding instruction for english language learners: A concep-tual framework. International Journal of Bilingual Education and Bilingual-ism 9(2), 159–180 (2006). https://doi.org/10.1080/13670050608668639,

https://doi.org/10.1080/13670050608668639

With the emergence of generative machine learning systems capable of producing text (e.g., ChatGPT), the processes of searching for and evaluating information have changed substantially. This already affects primary school students, who from around the age of 10 begin to search for information on the internet on a regular basis. Such searches take place predominantly via Google, where an “AI overview” is presented on the results page for a large portion of search queries. In addition, platforms that provide AI-based applications for schools increasingly enable information searches via AI chatbots. At the intersection of 1) the actual capabilities of these systems, 2) children’s conceptions of such systems, and 3) the systems’ designs, new challenges become apparent.

Texts provided by generative machine learning systems during information searches are based on the systems’ training data. Because these models are trained on large datasets, the quality of the data cannot be fully controlled. As a result, such systems may reproduce biased information or misinformation. Removing unwanted information is difficult, and the major providers do not disclose how potentially problematic content is filtered. Beyond reproducing inaccurate information, the models may also modify information or generate additional inaccurate content (hallucinations).

Children tend to anthropomorphize AI systems by attributing to them the capacity to think and act autonomously. They overestimate the actual capabilities of such systems and show limited awareness of the role training data plays in their development (Mertala and Fagerlund 2024). Consequently, children overestimate the quality and credibility of AI-generated outputs. Children’s interactions with information systems are strongly guided by system design and by the tasks they are given. Dialogic interaction formats and anthropomorphic representation through avatars may further reinforce the misconception that these systems are thinking and acting entities. On the other hand, children use the dialogic interaction format to ask follow-up questions and when prompted accordingly, these systems are capable of generating responses adapted to children's reading skills.

Solving information problems can be described in simplified terms as a five-step process: 1) defining the information problem, 2) selecting information sources, 3) scanning information, 4) processing information, and 5) organizing and presenting information (Brand-Gruwel et al. 2005). When interacting with chatbots, the latter four steps are taken over by the system. Students define the information problem and are then presented with a solution in the form of a generated text. As a result, AI systems offer a high level of convenience; correspondingly, when AI-generated answers are displayed in search engines, users do not consult the underlying websites in 90% of cases (Pew Research Center 2025). Given the challenges related to children’s conceptions of such systems, the problem of misinformation, and the convenience these systems provide, two key questions arise for the teaching of information literacy in the classroom: 1) How can students be guided to interact appropriately with such systems? 2) How can students be motivated to go beyond chatbot outputs by visiting websites to verify information and by carrying out the final four steps involved in solving information problems? Suggestions for addressing these challenges are presented and serve as the starting point for the symposium’s final discussion.

References

Brand-Gruwel, Saskia, Iwan Wopereis, and Yvonne Vermetten. 2005. ‘Information Problem Solving by Experts and Novices: Analysis of a Complex Cognitive Skill’. Computers in Human Behavior, 21 (3): 487–508.

Mertala, Pekka, and Janne Fagerlund. 2024. ‘Finnish 5th and 6th Graders’ Misconceptions about Artificial Intelligence’. International Journal of Child-Computer Interaction 39 (March).

Pew Research Center. 2025. ‘Google Users Are Less Likely to Click on Links When an AI Summary Appears in the Results’. https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/.