· 10 min read
How We Ran Doom on a Fly's Brain Connectome (Real-Time at 300 FPS)
We compressed all 165,122 neurons of the MaleCNS connectome from 37MB down to 4.98MB via FlatBuffers and a 4-bit LUT to fit inside L3 cache, enabling Drosophila to play Doom using bilateral olfaction and vision.
There is an unwritten law in computer science: if something processes information, someone will try to run Doom on it. From graphing calculators to digital pregnancy tests, from receipt printers to engineered bacteria, "Can it run Doom?" is the ultimate rite of passage for any computing architecture.
But what if the system isn't a silicon microprocessor, but the complete brain of a biological organism?
We took the entire central nervous system connectome of the male fruit fly (Drosophila melanogaster, the MaleCNS v1.0 dataset, comprising all 165,122 neurons and 10.2 million synaptic connections mapped via electron microscopy), transformed the network into a compact, CPU cache-resident recurrent operator, and plugged the brain into an embodied sensory-motor loop navigating a 3D Doom arena in real time.
The complete interactive simulator is running live in the frame below — directly in your browser:
1. The Iron Bottleneck: The DRAM Wall
A connectome of 165,000 neurons might seem modest compared to modern multi-billion-parameter Large Language Models. However, its synaptic connectivity is sparse (10.2 million directed edges with excitatory/inhibitory signs).
In standard FP32 precision within PyTorch (torch.sparse_csr_tensor), the adjacency matrix consumes approximately 118 MB. In compressed SciPy CSR format, it occupies 37.2 MB.
This sounds trivial for a computer with 16 GB of RAM, right? The problem lies in the physics of the von Neumann architecture:
- On every recurrent simulation step (), the CPU must fetch the entire 37 MB of synaptic weights from main memory (DRAM).
- A standard memory bus provides roughly 25 to 35 GB/s of bandwidth.
- Making a round-trip to DRAM on every single frame imposes a hard ceiling: PyTorch runs at a sluggish 16.3 ms per step (61 steps per second). The CPU spends over 80% of its clock cycles idle, waiting for data to travel across the bus.
To run in real time at 300+ FPS, there was only one solution: make the entire brain fit inside the processor's L3 cache (8 MB).
2. The Compression: FlatBuffers Zero-Copy and 4-Bit Quantization
To shrink the adjacency matrix from 37.2 MB down below 5 MB without disrupting biological dynamics, we combined three techniques:
- Synaptic Noise Pruning: Connections with synapses (representing microscopy reconstruction noise) were pruned, retaining the 1.54 million dominant synaptic highways.
- Delta Encoding in
uint16: Instead of 32-bit column indices, we stored relative offsets () as 16-bit integers with hop escapes. - Non-linear 4-Bit Codebook Quantization (LUT): Synaptic weights were mapped to a 16-level logarithmic dictionary in 4-bit nibbles, consuming just 2.5 bytes per edge.
- Zero-Copy FlatBuffers Serialization (
.mcns): We developed a strict binary schema allowing the brain to be loaded viammapin 14 microseconds, with zero load time and zero heap allocation.
The result: 4.98 MB. The entire brain fits comfortably inside the 8 MB L3 cache of an Intel Core i5-1145G7, eliminating DRAM bus trips entirely.
Throughput jumped from 61 steps/s to 335+ FPS on CPU, achieving a 5.4x speedup.
3. The Multimodal Sensory-Motor Loop
To navigate the Doom arena, running matrix multiplications in a vacuum was not enough — real biological sensors had to be coupled to the arena controls:
[3D Arena / Doom Engine]
│
├─► Vision: 32 raycasting beams ────► 4,589 Visual Neurons (visual_projection L/R)
│
└─► Olfaction: Odor plume (ppm) ────► 2,639 Olfactory Neurons (antennal ORNs)
│
▼
[165,122 Recurrent Neurons in L3]
│
▼
1,314 Descending Neurons (DNs)
│
▼
[Steering Torque + Forward Thrust + Trigger]A. Bilateral Vision and Contralateral Avoidance
A DDA raycasting engine (identical to classic Wolfenstein 3D and Doom) casts 32 beams across a 145° field of view.
- Left rays activate the 4,589 left visual projection neurons.
- Right rays activate the 4,612 right visual projection neurons.
- We wired the contralateral biological reflex: when a wall approaches on the left, motor drive directs the fly to steer right, sliding smoothly along corners (wall sliding).
B. Chemotaxis and Antennal Tropotaxis (Olfaction)
Real flies do not locate food by sight — they track it by scent.
- In the MaleCNS connectome, we mapped 2,639 olfactory receptor neurons (
cell_class = olfactory). - The fly has two antennae separated in space (). The target emits an aromatic plume that diffuses with distance.
- When odor concentration is stronger on the right antenna (), the bilateral gradient drives turn torque towards the plume. As overall concentration increases, the circuit triggers a forward surge (accelerating straight towards food).
4. Hypothesis Testing: Real Biological Wiring vs Shuffled Null Model
To prove that navigation originates from genuine biological wiring rather than statistical artifacts, we benchmarked the fly against a Degree-Preserving Shuffled Null Model (where synaptic connections were randomly permuted while preserving each neuron's in/out degree):
| Model | Size / Residence | FPS | Wall Avoidance | Collisions in 300 Ticks |
|---|---|---|---|---|
| MaleCNS Compact (.mcns) | 4.98 MB (L3 Cache) | 335.4 FPS | 99.4% | 0 to 1 |
| SciPy FP32 Original | 37.20 MB (DRAM) | 86.3 FPS | 99.7% | 1 |
| Shuffled Null Control | 37.20 MB (DRAM) | 83.1 FPS | 88.7% | 34 wall impacts |
The random brain crashed into walls 34 times, repeatedly wedged in corners. The real biological connectome preserved in 4-bit L3 navigated corridors, negotiated 90° turns, tracked the target's odor plume through doorways, and captured randomly spawned prizes autonomously.
5. Source Code and Standalone Demo
All FlyDoom infrastructure is open source:
- The full-screen interactive simulator is accessible at:
/flydoom/ - The Zero-Copy FlatBuffers schema (
.mcns), Numba JIT kernels, and benchmark harnesses are located in the experiment repository.
If a 165,000-neuron insect connectome can now run Doom at 335 FPS inside the L3 cache of a consumer laptop, the question for future neuromorphic models is no longer whether they can run it — but who will be able to beat them in multiplayer.