Symposium
An interdisciplinary view on learners’ argumentation competences: empirical studies from Economics Education, Political Education and English.
Chair(s): Stefan Daniel Keller (PH Zürich, Schweiz)
Discussant(s): Dorothee Gronostay (TU Dortmund)
Argumentative competences are an important goal of modern education. In the competence model for democratic culture and intercultural dialogue (European Council, 2021), argumentation skills are described as basic for “analytical and critical thinking skills”, “conflict resolution skills” and “knowledge and critical understanding of the world (politics, human rights, cultures, etc.)”. Argumentative competences play an important role in the curricula of different subjects. For that reason, research on argumentative competences is conducted in different subjects.
Previous research on argumentation in the classroom is mostly devoted to science subjects and language subjects in compulsory education (e.g., Budke et al., 2015; Duschl & Osborne, 2002; Erduran & Jiménez-Aleixandre, 2007; Osborne et al., 2004). Little is known so far about argumentation in social science subjects in compulsory and post-compulsory school. Especially for the domain "politics and economics", only few theoretical and empirical works are available (Ackermann & Kavadarli, 2022; Gronostay, 2019; Siegfried, 2021). Furthermore, these subject specific research traditions have been largely unconnected.
The main goal of this symposium is to bring together empirical work on argumentation skills from various school subjects and domains, and to analyse similarities and differences between them in empirical studies. There are two guiding questions: First, whether the argumentative competences are similar or different between subjects; and second, what the relationship is between argumentative competences in spoken or written form is. This symposium addresses both these questions in an interdisciplinary format by presenting 3 empirical studies from different school subjects: English, Economic Education, and Political Education.
In paper 1, Lohmann, Lötscher and Keller analyse argumentative competences in the context of argumentative essays in English at upper secondary level (“Gymnasium”). They show that in English, written English argumentative skills and receptive competences (reading and listening) are highly correlated, suggesting there is a common underlying language competence from which all of these abilities draw. Also, students need most support in language competence while elements such as structuring a text or finding content can be transferred from other domains (e.g. first language writing).
In paper 2, Ackermann and Siegfried focus on written and oral argumentation in a problem-oriented, cooperative teaching/learning setting in the subject "Politics and Economics" at upper secondary level. They analyse the structure and content of written arguments in short essays on a socio-economic problem, the change in written argument quality during the teaching/learning setting as well as the connection between oral contributions from the group discussion and written argument quality in short essays.
In paper 3, Aydin, Waldis & Wenger present a program for promoting oral argumentation in citizenship education. Their paper provides empirical insights into a strategy training course and classroom debates. They show that despite standardized argumentation training, debates in the observed class turned out very heterogeneous. This seems to be partly due to the diversity of thematic inputs and partly due to students’ linguistic competencies and the debate culture.
Based on these presentations, consequences and transfer for teacher education and practical teaching will be discussed.
Presentations of the Symposium
Argumentation in English essays at upper-secondary level: linguistic and argumentative competences of foreign language learners
Stefan Keller1, Flavio Lötscher1, Julian Lohmann2
1PH Zürich, 2Uni Kiel
Background
Foreign language skills are vital for learners’ active participation in modern knowledge societies and integration in an international democratic system (Keller, 2023). Argumentative competences in English as a foreign language (EFL) are particularly important at secondary level. Yet they presuppose advanced skills of language proficiency, text structuring and argumentation which need to be explicitly instructed for students to master them (Keller et al., 2020).
This paper presents insights on the internal structure and development of EFL argumentative competences at upper secondary level. In particular, it has the following two research questions:
1) What competences of argumentative writing do learners at upper-secondary level reach?
2) What is the composite structure of argumentative writing competence in English in terms of use of language, structure and content / argumentation?
Data
This study was carried out as a repeated measurement design in upper secondary schools in Germany and Switzerland with an interval of eight to nine months between the two measurement points. Students completed computer-based tests on writing (two TOEFL argumentative writing tasks at each time point). Overall, data were collected from n=1882 students in Switzerland (58% female; age: x¯T1=17.56, SDT1=.91; x¯T2=18.27, SDT2=.91) and n=965 students in Germany (58.6% female; age: x¯T1=16.91, SDT1=.56; x¯T2=17.61, SDT2=.56).
Method
Two types of analyses were performed to analyse leaners’ argumentative writing competences. In a first study (Keller et al., 2020), holistic ratings were produced by two trained raters on the operational TOEFL iBT scoring rubrics. Inter-rater agreement, as measured by quadratic weighted kappa (QWK), was satisfying for all prompts and time points (QWK=.865/.775). In a second study, learners’ texts from the same dataset were analytically analysed by a new set of trained raters (both for T1 and T2). The analytic assessment of learner texts included the most important elements of an argumentative essay, based on relevant linguistic and pedagogical analyses of the genre. Furthermore, the rubric captured a range of criteria mentioned in upper-secondary EFL curricula in Germany and Switzerland, mostly control of grammatical and lexical language features and the ability to state one’s opinion in a large range of contexts (Fleckenstein et al. 2019). In terms of ICC, the weighted Kappa for language quality was .74, .70 for structure and .61 for content.
Results
For RQ1, results show that learners at upper secondary level mostly reach level B2 according to the CEFR in argumentative writing in terms of internal structure of their competences. For RQ2, analytic ratings show that students achieve highest scores for structure, lower ones for content and argumentation, and the lowest scores for language proficiency (i.e., spelling, grammar and lexical quality). Our study, however, also found considerable differences in development rates between the three aspects of writing quality: on average, learners gained half of a standard deviation in structure, a quarter of a standard deviation in content, and less than one fifth of a standard deviation in language quality, i.e., hardly anything at all.
Discussion and Relevance
In terms of fostering argumentative writing, the fact that the linguistic quality (in the sense of general language proficiency) of argumentative EFL essays hardly budges over one year would suggest learners need specific support in this area, and that linguistic structures should be taught in the specific context of EFL argumentative writing. By contrast it seems that the ability of finding content for argumentation and especially structuring of texts can be transferred from other languages or domains of learning.
Quality of written arguments on “energy policy” in a classroom intervention with upper secondary students: the role of individual learning characteristics and cooperative learning activities
Nicole Ackermann1, Christin Siegfried2
1PH Zürich, 2Uni Göttingen
Problem statement
An overarching educational goal of upper secondary education is that students acquire competences for present and future participation in society and economy (Eberle, 2015; Ross, 2021). To achieve this goal, content knowledge and argumentation skills on societal problems are, amongst other things, required (Ackermann & Siegfried, 2022). Whereas most empirical studies at upper secondary level in German-speaking countries focus on content knowledge of students in different school types and professional fields (e.g, Beck, 1993; Müller et al., 2007; Schumann & Eberle, 2014; Schumann et al., 2017) and the effect of different teaching/learning settings (e.g., traditional, action-oriented, self-organised) on performance and motivation (e.g., Seifried & Sembill, 2010; Sembill et al., 2007), a few recent studies examine argumentative competences in the domain “politics and economics” (e.g., Ackermann & Kavadarli, 2022; Gronostay, 2019; Siegfried, 2021).
Research goal and questions
Our study aims to investigate students’ written arguments before and after a problem-oriented, cooperative classroom intervention on a societal problem. We raise three research questions:
• (RQ1) What is the quality of written arguments regarding structural and content criteria before and after the intervention?
• (RQ2) How many profiles of written argument quality can be identified and characterised before and after the intervention?
• (RQ3) To what extent is written argument quality effected by individual learning requirements and group-based learning activities?
Methods
For the data collection, a problem-oriented, cooperative teaching/learning setting on the topic "energy policy" was developed and used in the subject "politics and economics". The sample was randomly drawn from high school students in the German state of Hesse (N=190, grade 10+11, age 16-18, female 46 %). Within the classroom intervention, students were confronted with the societal problem, then had to search for further information on the problem in the material given and discuss problem solutions in small groups (Siegfried, 2021). Before and after the classroom intervention an achievement test was used for domain-specific content knowledge (Ackermann, 2019) and a writing task on "energy policy measures" for domain-specific reasoning skills.
The textual data of the writing task were analysed qualitatively (Cohens κ = .84) using an analytical framework for domain-specific argumentative writing (Ackermann & Kavadarli, 2022). The video data of the group discussion were transcribed and coded (Cohens κ = .72 to .83) using an analytical framework for domain-specific argumentative discussing (Siegfried, 2021). With the quantified data (codings), frequency analyses, cluster analyses and regression analyses were conducted.
Results
Regarding RQ1, at the level of variables, the quality of the students’ arguments is rather low and did not significantly change by the intervention. Of all the arguments, about 80% were supportive justifications, 60% were specific, 12% were completely correct, 20% had scientific references and 35% contained multiple perspectives.
Regarding RQ2, at the case level, there are three distinct argument profiles before and after the invention. The profiles in the post-test are characterised as follows: Profile-1 (N1≈48 %) has a moderate argument quality, Profile-2 (N2≈43 %) shows a moderate to high argument quality, Profile-3 (N3≈9 %) has a very low argument quality.
Regarding RQ3, there is an effect of some individual learning requirements and some group-based learning activities on written argument quality. In the pre-test, argumentation quality is predicated by content knowledge and partly by the grade in German as first language. In the post-test, argumentation quality is partly predicted by content knowledge and by contributions in the group discussion.
Discussion
From these results, implications can be derived for both further subject specific research on argumentation in social sciences subjects and for teacher training to support argumentative competences.
Promoting Oral Argumentation in Citizenship Education. Empirical Insights into a Strategy Training Course and Classroom Debates
Açelya Aydin, Manuel Hubacher, Monika Waldis, Liliane Wenger
PH FHNW
Theories of deliberative democracy call for citizens to be communicatively activated in forming opinions on issues since the 1990s. Therefore, civic education is expected to introduce adolescents to deliberative processes and to promote their argumentative competence–understood as a linguistic and a political competence (Rapanta & Felton, 2022; Zohar & Nemet, 2002).
Empirical findings show that instructional interventions can promote oral reasoning and critical thinking in a variety of subjects. However, intervention effects are often small and inconsistent. Authors suggest moderately controversial topics for civic education goals in middle school (Gronostay, 2019). Empirical findings show that students in deliberative settings are more willing to engage opposing arguments and revise their own than in competitive settings (Felton et al., 2015). In addition, student dialogues in small groups are conducive to argumentation skills (Rapanta & Felton, 2022; Schuitema et al., 2011). There is limited information to date on the effectiveness of strategy training. In an intervention study (grade 8/9, Gymnasium), Gronostay (2019) combined thematic input with argumentation training that included analytical tasks and oral practice. This led to more transactive speech acts (counterarguments, objections) in the experimental group in the subsequent fishbowl discussion. However, revision and adaptation of own arguments remained largely absent. In this study, we investigate the extent to which an adapted version of this strategy training with additional dialogue exercises contributes to more transactive dialogues. The following research questions will be investigated:
RQ1: What it the distribution of different argumentative moves (e.g., counterarguments, integration) in processes of argumentative reappraisal?
RQ2: What is the complexity of argumentations (number of reply moves per argument) and the distribution of argumentative sequences (one-sided/critical/responsive)?
RQ3: To what extent are different patterns regarding argumentative moves and complexity visible in the observed classes?
RQ4: Are "political" arguments visible and how are they negotiated?
Method
Six teachers (4-27 years of teaching experience) participated with their classes in the testing of an argumentation training in the school year 2022/23. The students (aged 13-17, lower secondary (grade 8/9, different educational levels) exhibited heterogeneous linguistic competencies. The instructional setting consisted of a) the standardized argumentation training (90′), b) a thematic input on a controversial topic selected by the teachers (90′), c) a fishbowl debate about it in class (45′), and d) a written judgment task (45′). As a controversial topic, teachers chose either a topic close to everyday life (sustainable school) or a topic of social relevance (introduction of a cashless society).
Data collection included video recording of the fishbowl debates and written documentation of all lessons and teaching materials used. The analysis of the transcribed student debates was conducted in four steps: 1) analysis of argumentative structure (Toulmin, 1958/2003), 2) analysis of argumentative moves (Gronostay, 2019; Felton et al. 2015), 3) complexity per argumentative sequence (one-sided, critical, or responsive) (Gronostay, 2019), and 4) identification of political issues per sequence (table 1). All debates were coded in teams of two and then discursively cleaned up by the research team. Code definitions, anchor examples, and collaborative discussion of special cases and their documentation ensured standardization.
Results
The argumentative moves practiced in the argumentation training were visible in the fishbowl debates. Contradiction and integration of arguments occurred frequently. A larger proportion of the argumentations were responsive. Class-specific patterns emerged at both code levels (argumentative moves, complexity). Political issue were often pronounced in one-sided argumentation.
Conclusion
Despite standardized argumentation training, debates in the observed class turned out very heterogeneous. This seems to be partly due to the diversity of thematic inputs and partly due to students’ linguistic competencies and the debate culture (on/off-topic, meta-talk, commitment to topic) in the classes.
|
