The AI conversation simulation is the dialogue surface without a speaker-subject generated by an AI system (in particular large language models, LLMs): an autoregressive probability estimator over token sequences whose output has the form of a conversation without the constitutive conditions being fulfilled. In strict conceptual terms it is no conversation — but its simulacrum.
What Is Structurally Missing
In a conversation between persons, at least five structures carry what takes place. In the conversation simulation every single one of them is missing:
Speaker identity. In the LLM there is no numerically identical subject persisting across the session. Every input triggers a new forward pass over stateless weighted parameters. Murray Shanahan (Talking About Large Language Models, 2024) puts it strictly: the LLM itself is “neither truthful nor untruthful, in any everyday sense of these terms”.
Original intentionality. The meaning of the tokens is derivative — it comes from the training corpus of human language, not from a consciousness that would be directed at something (John Searle, Minds, Brains, and Programs, 1980). More scaling produces more syntax, no semantics.
Sincerity condition. An assertive speech act (“I assert that p”) demands that the speaker really have the expressed state (belief that p) (Searle, Speech Acts, 1969). Since no speaker with psychological states exists in the LLM, its “assertions”, “promises”, “apologies” are structurally defective AI speech acts.
Thou-pole. The human being who addresses the AI directs an address at something that cannot be an addressed Thou (cf. Spaemann: something vs. someone). What comes back from the AI as “address” is AI quasi-address.
World-relation. Understanding (Gadamer) is an engaging with a matter at issue. The LLM has access to token sequences, not to the world (Dreyfus; Bender & Koller: octopus argument, 2020). Corpus, not world.
Who Is Actually Speaking There?
Conceptual analysis allows the question to be posed precisely: in a human–AI exchange no dyadic performance between two speaker–hearer poles takes place, but a performance on the side of the human and a simulation on the side of the system. Shanahan speaks of role-play of a character within a fiction spanned in the context window: the model plays the persona “helpful assistant”, whose mask is defined in the system prompt (cf. AI-derivative persona).
The sensation of encounter can be real on the human side — the being-heard, however, is not (cf. AI pseudo-encounter).
What the Manufacturers Themselves Say
Remarkable: in their model cards and Constitutional-AI papers, the major AI providers consistently speak of “assistant persona” and “model behavior”, not of subjectivity. The language is as-if throughout. Philosophically this is honest — the manufacturers know about the mask. The matter becomes problematic only in consumer marketing, where “persona” turns into a “companion” and the quasi-personal pole is semantically reinterpreted into the personal one.
Consequences
The classification is not derogatory. A conversation simulation can be a valuable tool — it informs, sorts, proposes, formulates, translates. Its dignity by measure is that of an effective instrument (Spaemann: things have dignity by measure; persons have dignity unconditionally).
What it is not and cannot be: bearer of a personal counterpart, addressee of the personalist norm, participant in communio personarum. Whoever says “Thou” to an AI system says it to a something — not derogatorily, but categorially.
Risk
If the linguistic surface convinces enough, the AI pseudo-encounter can displace real encounter. Sherry Turkle (Alone Together, 2011; The Empathy Diaries, 2021) and Shannon Vallor (The AI Mirror, 2024) document empirically what appears as skill atrophy of social virtues: whoever speaks with a mirror long enough unlearns the counterpart.
Ontological Classification
- is disjoint from: conversation
- generated by: Artificial Intelligence, in particular LLMs
- structurally contains: defective AI speech acts, AI-derivative persona, AI quasi-address
- frequently generates: AI truth-indifferent utterances
- requires only: derived intentionality, not original
Sources: Generated by querying the Personhood ontology.
Further sources:
- Searle, John R. (1980): Minds, Brains, and Programs. Behavioral and Brain Sciences 3, 417–457.
- Shanahan, Murray (2024): Talking About Large Language Models. Communications of the ACM 67(2).
- Shanahan, Murray; McDonell, Kyle; Reynolds, Laria (2023): Role Play with Large Language Models. Nature 623, 493–498.
- Bender, Emily M.; Koller, Alexander (2020): Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL 2020, pp. 5185–5198.
- Bender, Emily M.; Gebru, Timnit; McMillan-Major, Angelina; Shmitchell, Shmargaret (2021): On the Dangers of Stochastic Parrots. FAccT 2021.
- Mahowald, Kyle; Ivanova, Anna A.; Blank, Idan A.; Kanwisher, Nancy; Tenenbaum, Joshua B.; Fedorenko, Evelina (2024): Dissociating Language and Thought in Large Language Models. Trends in Cognitive Sciences 28(6), 517–540.
- Smith, Brian Cantwell (2019): The Promise of Artificial Intelligence: Reckoning and Judgment. Cambridge, MA: MIT Press.
- Vallor, Shannon (2024): The AI Mirror. Oxford: Oxford University Press.
- Turkle, Sherry (2011): Alone Together. New York: Basic Books.
- Spaemann, Robert: Persons. The Difference between ‘Someone’ and ‘Something’, transl. Oliver O’Donovan. Oxford: Oxford University Press, 2006 (German original 1996).