Universal Machine Transformer (UMT)

Run any machine

One network, any machine

A small transformer was trained on the steps of a universal Turing machine, while it ran random machines on short random tapes. Describe any machine below, give it an input, and watch that one trained network run it, live in your browser. Nothing is precomputed.

This network has 211,270 parameters, and gradient descent finds one of just 8,170 that is just as exact: as far as we can find, the smallest trained transformer shown to act as a universal computer, about 7,300 times smaller than the nearest comparable result. How small it is

Machine
1

The machine you described

Its tape, its head, and its state, as the universal machine runs it.

Machine steps0
state q0
Its rules
2

The universal machine

Its whole tape: your machine written as data, then your machine's tape. It is now finding the described machine's head.

Universal steps0
* the current state one state's three rules the rule being used your machine's tape the universal machine's head
3

The network's step

Every step above is one forward pass: read three tokens of the transcript, append what comes next, and move the window forward by one.

Forward passes0
The window it read
Where each attention head looked
What it predicted

How small it is

Trained transformers that run programs they were never trained on, by parameter count. Log scale: each gridline is ten times the last (K thousand, M million).

Width 8, trainedplain gradient descent; every seed exact (train it in your browser) 8,170
lag3, trainedthe network these pages run 211,270
Xu, Yang, Li & McAllester (2026)trained; interprets MicroPy, a small universal language 59,500,000

This project's hand-built network, lag1, realizes the same universal machine with 21,331 integer weights, none learned; gradient descent finds one 2.6 times smaller. At width 6 (6,092 parameters) two training seeds of three are exact. The comparison is between networks that learned to be universal.

Turing completeness belongs to the system: the network is the step of a loop that reads the old end of an unbounded transcript. Its exactness is measured on all 14,849 steps the universal machine has been seen to take, not proved for every step it could take. Details and sources in the README.

How this works

The tempting match for a language model pairs the Turing machine's tape with the context window. Here the match is different:

Turing machinethis transformer
the tapethe transcript: every token emitted, never overwritten
the head's viewthe context window: three tokens
the control statewritten into the transcript, read back through the window
the rule tablethe trained weights; one forward pass is one step

The window reads the old end of the transcript while each new token is appended at the far end. That turns the transcript into a queue, which is what makes the loop able to compute anything. A window that only sees the newest tokens, as in ordinary decoding, cannot.

The network never saw any of the named machines. It was trained only on the universal machine's steps while it ran 4,000 random machines on tapes of up to six cells. Every machine is written on the tape as data, so learning the universal machine's few local steps was enough to run them all. If your own machine needs a step the network never saw, the checks in panel 3 will say so.