PREDICTION MEETS INFORMATION THEORY
Autoregressive
Compression.
GPT-2 reads the preceding tokens to predict the next. Explore each probability distribution, watch the arithmetic interval narrow, and follow the code that identifies the sequence.
Quantized ONNX · browser CPU (WASM)
Up to 2,048 Unicode code points. Your text stays in this browser.
The count comes first; the remaining hexadecimal digits are the fraction.
First run downloads GPT-2 weights (approximately 130 MB) from Hugging Face and caches them locally. No inference server or text uploads.
Claude Shannon estimated about one bit per letter for printed English. On your input, GPT-2 got .
About the Huffman comparison
Ordinary Huffman coding over GPT-2 tokens, using one frequency table from the entire input. The count covers the same preceding tokens as our encoding. Payload bits only; codebook, length metadata, and final byte padding excluded.
0.| Token | Probability |
|---|
Inspect current and new bounds
NEW BOUNDS