Image: Mohsenvand / MIT Media Lab — Figure 1 from the paper "Sketchpad: certified exact arithmetic with a small vision-language model" (arXiv:2611.04434). It shows one step of 74,829 × 6,153, the margins of all 2,446 views, and the 220-problem comparison.
TL;DR — Sketchpad MIT exact arithmetic explained: A researcher at MIT Media Lab has built Sketchpad, a 0.8-billion-parameter vision-language model that does arithmetic with a mathematical guarantee: it answered all 220 problems in a pre-registered test, including sums of up to 1,000 digits, where the strongest frontier model managed 217. The trick is a "sheet" the model reads three cells at a time, so its exactness reduces to 2,446 checkable cases — and a Lean proof assistant has verified the argument step by step.
📌 KEY HIGHLIGHTS
- 220 out of 220: Sketchpad answered every problem in a pre-registered comparison — sums of 10 to 1,000 digits and products of 2- to 20-digit numbers — while GPT-6.1 Sol scored 217, Claude Opus 5.5 scored 216 and Gemini 4 Argon 207.
- A proof, not a benchmark score: the paper proves in Lean that if the model is exact on 2,446 finite "views" of its working sheet, it is exact on inputs of any length. All 2,446 views pass on the laptop GPU that runs it.
- Tiny by design: the base is Qwen3.5-0.8B with a rank-16 LoRA adapter of just 10.2 million parameters — 1.2% of the model. The same base model without the adapter scores 23 out of 220.
- It runs in a browser: the model, adapter and certificate are public on Hugging Face and the code is on GitHub under Apache 2.0. It runs on a laptop GPU through WebGPU — no server needed.
- Honest limits: the guarantee covers the model's arithmetic steps, not the runtime around them, and the chat version still loses 5.8 points on fresh grade-school word problems. This is a research preview, not a finished product.
What happened?
On 3 November 2026, Mostafa 'Neo' Mohsenvand of the MIT Media Lab posted a paper to arXiv titled "Sketchpad: certified exact arithmetic with a small vision-language model" (arXiv:2611.04434). The paper was picked up by the press on 9 November. It tackles a familiar frustration: large language models are usually right at long arithmetic, but their accuracy is only ever measured on samples — there is no guarantee for the next unseen input, and the errors that remain are silent. Calculator tool use is exact, but it moves the computation outside the network.
Sketchpad takes a different route. Instead of asking the model to do a whole sum in one go, or to write out chain-of-thought reasoning, it gives the model a rendered "sheet" — the kind of column layout you used for long multiplication at school — and shows it only three cells at a time. Each forward pass reads that three-cell view and outputs one step: a digit, a carry or borrow, and the next operation. A small fixed runtime moves the view along and writes the outputs down. The model never writes a digit of a result itself, and nothing but the current view passes between steps.
Because every step depends only on its three-cell view, the number of possible situations is finite — and small. The paper lists them as tables: column addition (242 views), the five operations of long multiplication (1,553), subtraction (242), quotient-digit estimation (400) and estimate decrement (9). That is 2,446 views in total. A low-rank adapter trained on exactly those tables teaches the procedures to Qwen3.5-0.8B. Without the adapter, the base model passes only 12 of the 2,446 views.
Sketchpad MIT exact arithmetic explained: the 220-problem comparison
The head-to-head test was fixed before it was run: 220 problems, 20 at each length — sums of 10, 30, 100, 300 and 1,000 digits, and products of 2-, 4-, 6-, 10-, 15- and 20-digit numbers — with the same prompt and a budget of 32,768 output tokens for every model. Sketchpad answered all 220. Here is how the rest did:
| Model | Score (out of 220) | Where it lost points |
|---|---|---|
| Sketchpad (Qwen3.5-0.8B + adapter) | 220 | None |
| GPT-6.1 Sol | 217 | 300- and 1,000-digit sums; all three errors were completed answers, wrong by 1 to 637 digits |
| Claude Opus 5.5 | 216 | 300- and 1,000-digit sums |
| Gemini 4 Argon | 207 | 300- and 1,000-digit sums; 9 of its 13 errors ran out of tokens |
| DeepSeek-V4-Pro | 199 | Mostly long sums, by running out of tokens |
| Mistral-Large-4 | 168 | Long sums and long products |
| Gemma 4 31B | 156 | Long sums; 46 of its 64 errors were wrong completed answers |
| GLM-5.3 | 152 | Long sums and long products |
| gpt-oss-120b | 149 | Long sums and long products |
| Qwen3.5-27B (chain of thought) | 121 | Long sums; 93 of 99 failures ran out of tokens |
| Qwen3.5-0.8B (base model, no adapter) | 23 | Almost everything; 3 with chain of thought |
Two details matter. First, every model in the comparison — including the frontier ones — was exact on all the products. The gap opened only on the longest sums, where the big models produced confident, finished, wrong answers. Second, Sketchpad also passed a separate 200-problem test running live in a browser on a laptop GPU: all 200 exact, 184 of them within a pre-registered 12-second target, the slowest taking 16.5 seconds. Each step takes about 0.1 seconds on a laptop GPU.
Image: Mohsenvand / MIT Media Lab — Figure 2 from the paper: the sheet machine. One view and its outputs for each calculator table, the division 4,100 ÷ 59, and four procedures batched into 32 forward passes instead of 73.
Why the Lean proof matters — and what it does not cover
The headline is not the score; it is the guarantee behind it. The paper's Theorem 1, proved in Lean 4 with Mathlib 4.35.0 and no placeholder steps ("sorry") or extra axioms, says exactness on the 2,446 views is equivalent to exactness at every length: if the model gets every view right, every run of every length is exact, and vice versa. The certificate was measured on the actual deployment path — an Apple M4 laptop GPU running the model in a browser at 16-bit precision — where the smallest margin over all 2,446 views was 3.02 logits, so the result tolerates any change to the logits smaller than 1.51.
There is a second theorem with a warning in it. Theorem 2 proves that if a step is allowed to read the steps before it — as chain-of-thought reasoning does — then no finite test can certify the model, however long the inputs tested. That is the point of the pinned-sheet design: by construction, each step sees only its three cells, so a finite check is enough.
But the guarantee has a boundary, and the paper states it plainly. The certificate covers the model's steps. The runtime that sequences those steps, the layouts of the sheet, and the correctness of the classical procedures are conventional code — they are not formalised in Lean. The same applies to turning a word problem into a computation: a dialogue adapter built on top of the arithmetic one invokes the sheet with 0.994 precision and 0.989 recall on held-out phrasings, and writes the correct computation for 92.8% of word problems — but it mistranslates about 7% of them, and on fresh grade-school problems it reasons 5.8 points worse than the base model on GSM8K. A scalar "gate" on the adapter restores the base model exactly (verified bit for bit) when the sheet is not in use, so the arithmetic skill does not have to damage the model's other abilities.
The same certified procedures also power a small symbolic algebra runtime that solves, expands and factors polynomials, with every number computed by the model. Under fault injection it rejected 200 of 200 incorrect roots. It is limited to polynomials of low degree in one variable and linear systems of up to three equations.
Why it matters for India readers
There is no India launch or price here — the work is free to use. The paper is on arXiv, the code is Apache 2.0 on GitHub, and the model runs in a web browser on an ordinary laptop GPU, so students and researchers in India can try the demo without a server or a paid API key. That accessibility is part of the story: a 0.8-billion-parameter model with a 10-million-parameter adapter, running locally, now carries a checked guarantee on arithmetic where the largest frontier models still make silent mistakes on 1,000-digit sums.
The wider lesson is about how to trust AI at calculation-heavy work — exam preparation, accounting checks, engineering estimates — where a wrong digit is worse than a refusal. Sketchpad does not claim language models are now reliable calculators in general. It shows one disciplined design in which reliability can be proved rather than sampled, and it also shows where that proof stops: the moment a problem has to be translated from words into sums, the guarantee no longer applies. For anyone building AI tools that handle numbers, that boundary is the useful part.
It is also a useful companion to another Lean-checked result from the past few days: the AI-assisted proof of the Aaronson-Ambainis conjecture, where a machine-checked proof settled an 18-year-old question about quantum speedups. And it follows MIT's WorldParticle model, one system that simulates sand, cloth, fluids and proteins. The same trend showed up earlier in the KLS conjecture proof with AI agents at Microsoft Research — 2026 is turning into the year machine-checked mathematics went mainstream.
Should you care?
✅ Worth your attention if you care about trustworthy AI arithmetic, you teach or study maths or computing, or you build tools where a wrong number is costly. Try the browser demo and read the paper — both are free.
⏳ Keep in perspective if you want a general-purpose maths assistant: the guarantee covers the arithmetic steps only, the chat version still mistranslates some word problems, and a step takes about 0.1 seconds, so very long calculations are slow. This is a research preview, not a product.
FAQs
What is Sketchpad MIT?
Sketchpad is a research system from Mostafa 'Neo' Mohsenvand at the MIT Media Lab, published on arXiv on 3 November 2026. It is a Qwen3.5-0.8B vision-language model with a small trained adapter that performs arithmetic on a rendered "sheet", reading three cells at a time, with its exactness proved in the Lean proof assistant.
How can a small 0.8B model be exact at any length?
Because each step reads only a three-cell view of the sheet, there are only 2,446 possible views in total. The paper proves in Lean that getting all 2,446 views right is equivalent to being exact on inputs of any length. The model was checked on all 2,446 views on the laptop GPU that runs it, with the smallest margin 3.02 logits.
Did Sketchpad really beat GPT-6.1 Sol?
On this specific pre-registered test of 220 problems, yes: Sketchpad scored 220 out of 220 and GPT-6.1 Sol scored 217, with all three of its errors on sums of 300 and 1,000 digits. But the scope is narrow — sums up to 1,000 digits and products up to 20 digits — and every model tested was exact on the products. It is a win on certified long arithmetic, not a claim that Sketchpad is better at maths in general.
Can I try Sketchpad myself?
Yes. The model, adapter and certificate are on Hugging Face as mohsenvand/sketchpad-0.8b-preview, and the runtime, algebra, baselines and paper sources are on GitHub under Apache 2.0. The model runs in a web browser on a laptop GPU. The Hugging Face page labels it a research preview, not a general assistant, and notes it is not safety-tuned.
Takeaway: A 0.8B model with a 10-million-parameter adapter now does long arithmetic with a Lean-checked guarantee — 220 out of 220 where the strongest frontier model scored 217. The proof covers the arithmetic steps, not the words around them, and that boundary is exactly what makes it worth understanding.
Try the demo in your browser and read the paper on arXiv — the link is in the Sources below.
Sources
- Sketchpad: certified exact arithmetic with a small vision-language model — Mohsenvand, MIT Media Lab, arXiv:2611.04434 (submitted 3 November 2026)
- Full text of the Sketchpad paper (arXiv HTML)
- Sketchpad-0.8B model, adapter and certificate on Hugging Face
- Sketchpad source code on GitHub (Apache 2.0)
- Sketchpad paper PDF on the project page
- AI Weekly — Sketchpad: MIT formally proves 0.8B VLM exact on arbitrary-length arithmetic, 220/220 (9 November 2026)
Comments
Post a Comment