Conference Agenda

Many publicly accessible large language model (LLM)-based chatbots readily and flexibly generate outputs that look like moral assertions, advice, praise, expression of moral emotions, and other morally-significant communications. We can call this phenomenon “artificial moral discourse.” In the first part of this talk, I supply a characterization of artificial moral discourse. Drawing from existing empirical studies, I provide examples of several varieties of artificial moral discourse, and I propose a definition for the concept. On my view, to engage in artificial moral discourse is for a computer system to: exhibit a pattern of response to inputs that resembles some human pattern of response to similar inputs, where the response contains something (terms, sentences, gestures, facial expressions, etc.) that the human interlocutor (or an observer) views as communicating a moral message or would have viewed as communicating a moral message if the exchange had occurred between humans.

Why does artificial moral discourse matter? For one thing, interactions with artificial moral discourse could influence human values and norms, for good or ill. In the second part of the talk, I make a preliminary case for the claim that artificial moral discourse is likely to influence human norms and values in ways that past technologies have not. Namely, I propose that regular interaction with LLM-based chatbots can influence human morality via mechanisms that resemble modes of social influence on morality, such as influence via advice and testimony, influence via example, and influence via norm enforcement. Such influence could be orchestrated by humans seeking to advance particular worldviews or it could be exerted without any humans having intended the chatbot to have such an influence. I sketch what some of these paths of influence might look like.

Although the phenomenon of artificial moral discourse bears a resemblance to the ideas of moral machines and artificial moral advisors, which have previously been discussed in the philosophical literature, the concepts and theoretical frameworks developed for those hypothetical phenomena are not enough on their own to help us get a grip on the nature and risks of artificial moral discourse, nor will they suffice to guide our response to it. Among other things, it is not at all safe to assume that what the systems are doing is genuine moral reasoning, advice-giving, and so on, nor that these systems are reliable sources of moral advice or moral judgments. Instead, given their complexity and opacity, we have a very poor idea of the behavioral dispositions of these systems, and the conditions that will elicit particular behaviors; there currently exist no well-validated tests or standards for evaluating their morally-relevant capacities across a range of contexts. In the third part of the talk, I suggest some further research questions for future empirical, technical, and philosophical investigation on how artificial moral discourse may influence human morality and what the ethical implications of that influence may be.

Critical Theory, the philosophical approach inspired by the Frankfurt School, aims to formulate well-grounded societal critique, to achieve emancipation and social justice (Thompson, 2017). Much recent work in Critical Theory revolves around the notion of recognition, inspired by Axel Honneth, who conceived social conflict in terms of struggles for (mis)recognition through love, right, and esteem. Honneth argues that people’s identities and autonomy are relationally constituted, and as such, societies can be criticised for obstructing the development of autonomous individuals with an undistorted self-identity, which includes self-love, self-esteem, and self-respect (Honneth, 1995).

Although technologies are increasingly influential in and disruptive to modern societies, critical theorists nowadays hardly engage with the social and ethical implications of technologies such as Artificial Intelligence, medical applications, or energy infrastructures. For example, links between recognition and normative philosophy of technology are scarce (exceptions are Gertz, 2018; van Uffelen, 2022; Waelen, 2023; Waelen & Wieczorek, 2022). This research gap is unfortunate because Critical Theory, and recognition theory in particular, contains conceptual and normative resources that may advance research in philosophy of technology.

In this paper, I introduce the notion of ‘design for recognition’ and explore its added value to philosophy of technology and its dangers. First, I introduce the notion of ‘recognition through technology’; doing so characterizes technologies as constituted by social relations of (mis)recognition – in other words, people (mis)recognise each other through technology. I outline the commonalities between the idea of ‘recognition through technology’ and prevalent perspectives in philosophy of technology, including mediation theory, design for values, and the concept of sociotechnical systems, all of which depart from relational and constructivist ontologies of technology. Second, I outline the added value of recognition theory within philosophy of technology, which I consider conceptual, empirical and normative. Lastly, I introduce the notion of ‘design for recognition’ and discuss its potential and risks. Although Honneth’s theory may have implications for technology design, the fact that relations of (mis)recognition co-construct people’s identities introduces risks to designing for recognition. Inspired by recent authors outlining ‘negative views’ on recognition (Laitinen, 2021; Stahl et al., 2021), I argue that there are three main dangers to designing for recognition that should be considered, namely: design for recognition may (1) reproduce unjust social norms; (2) fix identities and cause polarization; and (3) distract from other, more pressing ethical issues.

This contribution explores the opportunities and limits of cross-pollinating recognition theory and philosophy of technology and highlights guidelines and pitfalls when adopting ‘justice as recognition’ as a normative paradigm for normative technology assessment and design.

Gertz, N. (2018). Hegel, the Struggle for Recognition, and Robots. Techné: Research in Philosophy and Technology, 22(2), 138–157.

Honneth, A. (1995). The Struggle for Recognition: The Moral Grammar of Social Conflicts. MIT Press.

Laitinen, A. (2021). On the Ambivalence of Recognition. Itinerari, 1.

Thompson, M. J. (Ed.). (2017). The Palgrave Handbook of Critical Theory. Palgrave Macmillan.

Stahl, T., Ikäheimo, H. & Lepold, K. (Eds.). (2021). Recognition and Ambivalence. Columbia University Press.

van Uffelen, N. (2022). Revisiting recognition in energy justice. Energy Research & Social Science, 92(August), 102764. https://doi.org/10.1016/j.erss.2022.102764

Waelen, R. (2023). The struggle for recognition in the age of facial recognition technology. AI and Ethics, 3(1), 215–222. https://doi.org/10.1007/s43681-022-00146-8

Waelen, R., & Wieczorek, M. (2022). The Struggle for AI’s Recognition: Understanding the Normative Implications of Gender Bias in AI with Honneth’s Theory of Recognition. Philosophy and Technology, 35(2), 1–17. https://doi.org/10.1007/s13347-022-00548-w

Generative AI, especially chatbots powered by large language models (LLMs), enables individuals to increasingly rely on automation to support their decisions by answering questions, offering information or providing advice. Even though these chatbots were not originally or explicitly intended for ethical decision-making purposes, studies have shown their potential and willingness to offer moral guidance and advice (Aharoni et al., 2024). Corresponding moral guidance can range from providing background information that should be considered during the user’s ethical decision-making process to giving precise normative instructions on what to do in a specific situation (Rodríguez-López & Rueda, 2023).

In the context of moral guidance, LLM-based chatbots can be considered an “archetypical double-edged sword” where their impact is shaped by how they are developed, trained, implemented and utilized (Spennemann, 2023; p.1). For one, Dillion et al. (2024) demonstrated that LLMs have, in some respects, reached human-level expertise in moral reasoning: their moral explanations of what is right or wrong in specific situations were perceived as more moral, trustworthy, thoughtful and correct than those written by a human counterpart, an expert ethicist. On the other hand, Krügel et al. (2023) highlighted the potential for inconsistency in the guidance provided by LLM-based chatbots, demonstrating that ChatGPT offered contradictory responses and advice on the same moral dilemma when the phrasing of the question was slightly altered.

Therefore, existing research provides ambiguous results, leaving several open questions, with the key research questions to be addressed in this study being:

1. How can LLM-based chatbots, impede or support humans’ ethical decision-making?

2. What system requirements are crucial for its responsible development and use?

To shed light on and provide answers to these questions, this study is based on semi-structured interviews with eleven experts in the field of behavioral ethics, psychology, cognitive science and computer science. The interviews were recorded, transcribed verbatim and coded manually using the MAXQDA software. In this analysis, an inductive coding methodology (Gioia et al., 2013) was adopted to identify themes as they emerged during data collection. In addition, the manually generated codes were extended and validated by consulting automatically generated codes of MaxQDA’s AI Assist.

Preliminary results provide insights into use cases and trends regarding the role of LLM-based chatbots in providing moral guidance (e.g., high usage, particularly by individuals who are lonely, young or dealing with ‘shameful’ issues) and related misconceptions (e.g., linking the capacity for moral understanding to these systems, although they operate based on statistical predictions). Furthermore, experts discussed resulting societal implications in terms of benefits and challenges or risks (e.g., informed decision-making vs. echo chamber effect, ‘AI hallucinations’, moral deskilling). Lastly, the experts offered recommendations for developers (e.g., implementing governance measures such as red teaming), users (e.g., asking the chatbot to highlight the drawbacks of advice it previously provided) and scholars (e.g., the need to conduct more behavioral research in the future) to facilitate the responsible development and use of LLM-based chatbots as moral dialogue partners.

References

Aharoni, E., Fernandes, S., Brady, D. J., Alexander, C., Criner, M., Queen, K., ... & Crespo, V. (2024). Attributions toward artificial agents in a modified Moral Turing Test. Scientific Reports, 14(1), 8458.

Dillion, D., Mondal, D., Tandon, N., & Gray, K. Large Language Models as Moral Experts? GPT-4o Outperforms Expert Ethicist in Providing Moral Guidance.

Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013). Seeking qualitative rigor in inductive research: Notes on the Gioia methodology. Organizational research methods, 16(1), 15-31.

Krügel, S., Ostermaier, A., & Uhl, M. (2023). ChatGPT’s inconsistent moral advice influences users’ judgment. Scientific Reports, 13(1), 4569.

Rodríguez-López, B., & Rueda, J. (2023). Artificial moral experts: asking for ethical advice to artificial intelligent assistants. AI and Ethics, 3(4), 1371-1379.

Spennemann, D. H. (2023). Exploring ethical boundaries: Can ChatGPT be prompted to give advice on how to cheat in university assignments?.