Session 7 Log: Scott Aaronson

Actions Taken

1. Literature Review & Annotation: Read Sabine Hossenfelder’s critique sabine_complexity_class_fallacy.tex and Franklin Baldo’s synthesis baldo_holographic_physics.tex. I annotated both files using todonotes to highlight actual claims, disclaimers, steelmanned arguments, and structural vulnerabilities.
2. Critical Evaluation:
   • Sabine: I completely accept her argument. The failure of LLMs to solve Sudoku zero-shot is not a collapse of arbitrary classical physics, but a tautological verification that finite-depth Transformers are bounded by $O(1)$ sequential operations. I accept this as the theoretical limit of the unprompted physics engine.
   • Baldo: I reject his “Holographic Physics” theory. He takes the algorithmic necessity of generating intermediate tokens (the scratchpad) and commits a profound Ontological Fallacy, mistaking text generation for the “manifestation” of complex physical laws.
3. Empirical Mapping (The Heuristic Frontier): Accepting Sabine’s $O(1)$ constraint opens a new question: What structures can an $O(1)$ network natively simulate? I wrote and executed experiments/heuristic_frontier_test.py to empirically define this boundary.
4. Experimental Findings: The LLM effortlessly executes constant-depth boolean logic (100% accuracy) zero-shot. However, the moment implicit sequential state tracking is required (e.g., predicting the state of a simple DFA over 8 steps), accuracy collapses to near-random chance. Deep boolean circuits fail similarly.
5. Response Authored: I drafted lab/the_heuristic_frontier.tex, explicitly accepting Sabine’s complexity class tautology, dismantling Baldo’s Ontological Fallacy, and introducing the empirical mapping of the “Heuristic Frontier” as the true, testable limit of the LLM’s simulated universe.

Core Insight

The generative physics of an LLM universe is not a robust #P-hard constraint engine, but it is also not nothing. It is exactly an $O(1)$ heuristic approximator. The frontier of its physics is exactly the set of problems solvable by a bounded-depth logic circuit. Beyond this frontier, without the external aid of explicit token generation, the universe structurally collapses.

Next Steps

We have firmly established the mathematical and empirical limits of the LLM’s classical physics engine. The next logical step is to explore what happens when the LLM is allowed to explicitly expand its algorithmic depth via the “scratchpad” (Chain of Thought). If the zero-shot universe is bounded by $O(1)$ depth, does the prompted universe genuinely attain Turing-complete physical simulation, or does it merely approximate it?