Despite the recent success of large language models (LLMs), philosophers and AI researchers widely reject the claim that LLMs can understand, speak, or act, arguing that it is both false and excessively anthropomorphic. My presentation aims to challenge key aspects of this skepticism. To that end, I frame questions about the ability to understand, speak, and act as subsets of a broader question, the question of rule-following. For that reason, my paper examines the extent to which LLMs can be said to follow rules. I show that the predominant attempts to deny LLMs the ability to understand, speak, and act stem from misguided conceptions about rule-following, which I refer to as “mentalism” and “neurophysiologicalism”. In opposition to these misconceptions, I develop a new non-reductionist and deflationary approach to rule-following. On this basis, I argue that today’s LLMs are in interesting, but substantive respects capable of rule-following. I also demonstrate how my deflationary account of rule-following opens up novel ways to assess the capacity of LLMs to understand, speak, and act. Lastly, I reject the charge of anthropomorphism: while my approach exhibits a philosophically unproblematic, epistemically beneficial form of anthropocentrism, it manages to mitigate anthropomorphism.