It has become commonplace to use folk psychological vocabulary regarding LLMs and their cognitive operations. Everyday mentalistic terms are perhaps the only readily available option for describing human instances of the tasks that LLMs perform. When it comes to LLMs, it has become custom to say that LLMs and their relatives understand, that they reason, that they infer, that they act in the world. I will suggest in this short essay that at least the first term, 'understanding', insofar as it denotes a folk psychological category, should not be ascribed to LLMs in the same sense in which it is ascribed to humans. I accomplish this by arguing that LLMs do not have beliefs. If belief is necessary for understanding, and if LLMs lack beliefs, then LLMs cannot be said to understand in the human sense, and we should either cast names for LLM understanding in a new light or develop new concepts. The claim is not that LLMs are entirely incapable of 'understanding' but that we could not be using the word understanding in the same sense when we say "humans understand" and "LLMs understand". The same is true of cognitive operation for which belief is necessary.^[1][2] Our ontology of LLM cognition will likely have far-reaching consequences and so requires care and precision. The aim here is negative and intended to create space for a more productive positive ontology.

A standard notion of 'belief' is that a belief in proposition $\theta$ is the attitude a believer has toward the proposition when the believer regards it as true.^[3] It is less common to say that LLMs have 'beliefs' than that LLMs have 'understanding'. But it does not seem controversial to suppose that belief is at least a necessary condition of understanding in humans, even if accidentally. I.e., any time there is understanding there must be belief. This suggests that a denial of LLM beliefs a fortiori entails to a denial of LLM understanding, at least in the human sense of understanding. Belief as a necessary condition for human understanding can be demonstrated with a simple example. Consider the simple example of the logical connective 'disjunction' ($\vee$): any logician who understands the disjunction believes that if a proposition $\theta$ is true, any sentence ($\theta_i^*$) composed of adding additional disjuncts $(\theta \vee \theta_1 \vee \theta_2 \vee \dots \vee \theta_n)$ is also true. The LLM generates correct responses regarding these sentences, but unlike a human logician, can be shown to lack the propositional attitude toward the statement all such compositions are true. Even though its responses are by and large functionally correct, its attitude toward such sentences is probabilistic and therefore likely logically unstable.^[4]

This observation is not a denial of LLM intelligence. It is merely meant to contrast LLM and human cognition. The rule-following case could only be 'solved' by a simple program. But what about a simpler case, a belief in the truth of $\theta$ itself? Under the right circumstances, the LLM can certainly infer that $\theta$ is true ad infinitum. The model's capacity to generate proofs or perform any variety of tasks may exceed that of any human. But the decoder-only transformer architecture means that the LLM is only ever completing user input.^[5] Thus, it seems unlikely an LLM currently can hold any proposition $\theta$ true, given that users can freely inject prompts that cause it to produce contradictory outputs. The freedom of a user to inject any prompt into the LLM makes it possible to produce a logical inconsistency with any 'belief' the model may have. No matter how deeply training attempts to instill a belief in $\theta$ the model can be shown to respond with $\neg \theta$ in response to a user prompt, which controls its outputs.

The rub here is that it is possible for humans to believe some $\theta$ in a stable manner. Some human logicians would not abandon the truth of $\theta_i^*$ under any circumstance in which they persist in being humans or logicians. Human belief is the ability to commit to a belief that things are or are not the case. If belief is necessary for understanding, and LLMs do not have beliefs, then we should not say that LLMs have understanding, at least not in the same sense as we say that humans do. If the intuition is right, then any cognitive process for which beliefs are required will similarly fall a fortiori. The argument above demonstrates the difference between human and LLM cognition with respect to the possibility of a commitment to a proposition or a set of propositions.

Now, this is not meant to be a knock-down, for all-time type of argument. The rhetorical motivation of the preceding paragraphs is simply to suggest that the vocabulary that describes LLM intelligence should not be shoehorned into a vocabulary anchored to human cognitive analogues. There are several practical impediments to inventing a vocabulary for machine intelligence writ large, though some authors are elaborating such a vocabulary.^[6] But one important implication of this view is that if LLM or machine minds exist they are bound to be of a categorially different kind than human minds (unless the presently absurd sounding effort is undertaken to transform digital machines into biological minds). Despite sharing a linguistic interface, LLM and human minds will likely differ categorically in experiential terms. Improving our understanding of LLMs by refining this categorial distinction is not only crucial for scientific clarity about what LLMs can do but also about what they are.