Turns out it's a pretty addictive game. Not that I have much to be proud of — any person (well, almost) can do this test on intuition alone, but top AI models score under one percent. It's called ARC-AGI-3. Here's why I think it's one of the most interesting benchmarks out there right now.
Why ARC-AGI exists
Modern neural networks are trained on terabytes of text and handle tasks similar to what they've seen before just fine. But give them something genuinely new — a problem where no ready-made pattern can be recalled — and they often fall apart. A normal human solves the same thing without breaking a sweat.
Researcher François Chollet (creator of Keras) has long argued that the industry is measuring intelligence wrong. Popular benchmarks reward the ability to memorize, not the ability to reason through unfamiliar situations. ARC-AGI is his attempt to fix that.
What came before (versions 1 and 2)
The earlier versions were colored-grid puzzles. You'd see a few "before → after" examples and had to figure out the transformation rule, then apply it to a new grid. No instructions — figure it out yourself. Those versions measured "passive" fluid intelligence: the ability to spot a pattern in unfamiliar data. But labs got good at cracking them: version 2 was solved in roughly a year, with Gemini hitting 77%.
What's new in ARC-AGI-3
Version 3 is a fundamental shift. Instead of static puzzles, there are now interactive environments that look like video games. An agent has to act across multiple steps, explore the world, plan, and adapt. No instructions — the player figures out the rules themselves.
Essentially, AI gets dropped into an unfamiliar game and we watch to see if it can figure out what's going on and start acting with any coherence. The key metric is no longer whether you solved the task, but how efficiently — meaning how many steps it took. And the score gap is squared, so random poking around doesn't pay off anymore.
Why it matters
For humans, these tasks are easy — even fun. AI agents struggle. Even the most powerful models get lost when they need to explore, form hypotheses, and remember what they've already tried. At launch in March, humans completed 100% of the tasks, while the best AI — Gemini 3.1 Pro — scored 0.37%.
But, as we know (and shouldn't forget), AI develops exponentially. So by July, GPT-5.6 took the lead with 7.8% — twenty times the March record. It followed the same pattern as previous versions: the benchmark holds for a while, then AI suddenly catches up and clears everything.
So ARC-AGI-3 tests whether AI can learn on the fly in an unfamiliar situation — not just recall well. That, I think, is genuinely cool. Plus it's one of the rare benchmarks where humans can (for now) feel unbeatable. No system has cleared it at human level. And you can try it yourself right in the browser. Go ahead — feed your ego.