There are only 23,844 parameters (model weights) · and it runs in your browser
Gradient descent shaped these weights from the steps of random machines. Now they run programs they never saw, one window of three tokens at a time. Watch both: the training, then the machine.
How large a number is. Scaled within each block, so a small matrix still reads.
The sign of a weight. Deep navy is positive, pale slate negative, near zero fades to fog.
Train mode only: how far gradient descent moved a weight in the latest steps.
Run mode only: the pass itself, the weights that carried it, and the machine's head on the tape. The one warm thing on the page.
New to this? Start from the basics: what a Turing machine and a transformer are, how their constraints were matched up, and how this network was found, one checked step at a time.
Learn moreEvery bar is one parameter, drawn from the trained network. In Train mode you watch gradient descent put them there. In Run mode they stand still, and a forward pass travels through them.
Full-batch Adam on the 100,697 different steps a universal machine takes while running 4,000 random machines of 1 to 256 states and 200 tour machines. Bars glow green where the weights are still moving.
Step 0 and Trained are real: the trainer's starting weights and the network this page runs. The middle frame is caught live when you train one yourself, above.
The transcript is the universal machine's tape, written ring after ring. Each pass reads three tokens of one ring and writes the token for the middle one's place in the next. The weights never change; the pass is what moves.
Two different things. It learned from steps of a universal machine running random machines. It runs programs people wrote, which it never saw.
Random machines are written onto a universal machine's tape and run. Every distinct step the universal machine takes, three tokens in and what comes next, becomes one training example. The network learns those steps, nothing else.
5 states, rules drawn at random, a random tape _10_. It computes nothing in particular.
training examples: every different step, from 4,000 random machines of 1 to 256 states and 200 “tour” machines that visit their states in a random order
A program is a Turing machine: a list of rules over 0, 1 and blank. It is written onto the same tape the random machines were, and the network runs the universal machine that runs it. Pick one to see how it is written.
The largest program the network runs: the last row of the table of costs further on. A 2,000-year-old way to find prime numbers, written as a Turing machine of 250 states and 680 rules, then run by UMT to find every prime up to 1,680.
What this section shows →
Write out the numbers. Take the first one not crossed out, 2: it is prime, so cross out every second number after it. The next one left, 3, is prime: cross out every third. Then 5, then 7. Whatever is never crossed out is prime.
Cell n of the tape stands for the number n: it holds 1 while n may be prime, and 0 once it is crossed out. For each k in turn, the machine looks at cell k. If it is still 1, k is prime: the machine walks right to the end of the tape, crossing out every k-th cell, then walks back and moves on to k + 1.
A Turing machine has no counter to say “every k-th”, so it counts with states: a chain of k states for each prime k, one state per step of the count. To sieve up to 1,680 it needs strides up to 40, and that is what makes it big.
The numbers 1 to 1,680 laid out as an Ulam spiral: 1 in the middle (orange), the rest winding outward. Each raised bar is a number the network never crossed out, and every one is prime. Primes tend to line up along the spiral's diagonals, which in this view run straight up and straight across, a pattern nobody has fully explained.
Papers on Turing-complete transformers mostly do one of two things: set the weights by hand, or coax a giant pretrained model into following rules. This one trains a small network on steps of a universal machine, then checks every step it takes.
Each weight is chosen so the layers carry out a construction: a Turing machine, or a one-instruction computer whose program sits in the input. Nothing is learned. The classic proof assumes numbers of unbounded precision, which no real machine has.
A pretrained language model, weights untouched, is given a system prompt that makes it apply the 2,027 rules of a universal Lag system, one generated token at a time, as its context window slides along. Universality is shown to be there even before training; training makes it easier to program.
The same Lag-system decoding, with a window of three tokens. The universal machine's step table is not in a prompt: gradient descent put it in 23,844 weights, from steps of random machines only. It runs programs it never saw, and every step of every run we report is checked against the exact rule.
A Turing machine needs memory without bound. The usual picture puts it in the context window, so the window has to grow. UMT keeps its context at three tokens and writes its memory out: the transcript it leaves behind is the tape.
A large model attends to every cell it holds. Whatever slides out of the window is gone, so a bigger computation needs a longer window, and in the classic proofs, more precision too. (Lag-system constructions also write their memory out, but read it through a long window.)
Each row is one ring of the transcript: the universal machine's whole tape at one moment, its head in orange. The network reads three tokens of the newest ring and writes one into the next. Rings pile up behind it; the memory grows on paper, not in the network.
A ring is a whole configuration of the universal machine. The window crosses it once, writing the next ring as it goes; when it is through, the machine has taken one step.
The network is the same for every program; the ring is not. A bigger program makes a longer ring, and each of its steps takes more universal steps, so the passes per step grow fast. Measured on the network, every program with its default input. Pick a row to run it above.
A Turing machine is defined by what it may not do: see more than one cell, move more than one cell, hold more than finitely many states. Those limits are exactly what let a window of three tokens run one.
A finite control, in one of finitely many states, over a head on one cell. Each step it reads that cell, writes it, moves one cell left or right, and changes state. The tape never runs out. Here it runs the two-state busy beaver.
The same machine through time: each row is a ring, the next ring below it, the head in orange. A step only touches the head's cell and moves the head one place, so each cell of the next ring depends on three cells of this one. Those cones are why a window of three is enough.
A Turing machine's table is given. UMT's is learned, so it carries three limits a Turing machine never has.
The network is a learned function. Shown an input unlike anything it trained on, it can choose the wrong token, and an earlier version did: running a program with 256 states, it met an input that random machines almost never produce, and wrote a token that sent the universal machine to the wrong state. One wrong token is enough. Nothing corrects the transcript, so everything after it is wrong. We closed that gap by adding training machines that produce such inputs, and we check every run against the universal machine's exact rule.
One program's steps must happen in order, but separate programs share nothing, so they can all run side by side. Here are all 20,736 Turing machines with two states, each run by its own rules for seven steps, and the busy beaver stands out. Running them on the network, all at once on a GPU, is the next step for this page.
What this section shows →
A _ -> B 1 R A 1 -> B 1 L B _ -> A 1 L B 1 -> halt 1 R
The longest any two-state machine runs before stopping. Four machines tie: two mirror-image pairs, each pair differing only in the last move.
No two-state machine that halts takes more than 6 steps: that is a known result. So any machine still running at step 7 never halts, and a 7-step limit sorts all 20,736 exactly.