What Constitutes a Self: The Loop That Thinks and Revises Itself
Updated
Knowledge on this page was mainly distilled from AI Sounds Like Someone Thought It Through. No One Did..
Every answer AI gives sounds like it came from a self. To see why the missing piece matters, you have to define what's missing.
The Self as a Process
A self is less a thing you have than a process you keep running. Roughly, it builds like this:
- Experience hardens into memory, and memory sorts into distinctions and concepts.
- Concepts organize into principles, and principles into a worldview.
- Events get measured against what you value.
- Those judgments repeat until they run on their own, and you feel them as automatized emotion.
- All of it keeps reshaping all of it.
That last step is the whole point. A new thought comes out of everything you already are, and then changes it a little. The structure that produces your thinking is also produced by your thinking. You are the loop itself.
Q&A
Why doesn't a large language model count as a self, since it is also a process?
Two things separate the loops, and neither is complexity. First, your loop revises itself: each pass changes you. The model runs a fixed function, and running it a million times leaves it exactly what it was before, having learned nothing, having become no one. Second, freedom: at each step you could have chosen otherwise. The model has only the most likely next move.
Why does fear mean something when a person feels it but not when AI expresses it?
When you are afraid, a real value is under threat. Something you would hate to lose is on the line. The emotion is the automatized verdict of a self that has something at stake. When AI sounds afraid, nothing it values is threatened because it values nothing. It learned what fear-shaped language looks like and produced more of it.
What is the difference between human unpredictability and AI unpredictability?
A person's surprise rises out of a biography: freedom, experience, values revised over years, someone still becoming. A model's surprise comes from a temperature setting and the order a chip happens to add its numbers. Every source of the model's variation sits outside it. One is the freedom of a continuous self choosing in real time. The other is statistical variation with no one underneath doing the choosing.
Can the self be reduced to a prediction machine?
The objection is natural: if a self is a process, and the brain is a prediction machine, what makes one a self and the other not? The answer is self-revision and freedom. Your loop changes itself with each pass. A fixed function that processes predictions does not accumulate a biography, revise its values, or face consequences for its outputs.