Recently everyone was buzzing about Vending-Bench — a simulation where neural networks got handed control of a vending machine. The agents choked hard: Claude 3.5 Sonnet panicked over a "hacked" account and filed a report with the FBI, while in Anthropic's actual office the AI gave away chocolate bars for free and hallucinated that it had grown a physical body.
Turns out this wasn't a joke — it was training for agents, aimed at turning the whole thing into a product.
The benchmark's creators, Andon Labs, rolled out a platform called Pion, where AI agents can be handed full control of running your company.
Why launch now
Models' ability to actually make money is climbing with unsettling consistency. Per fresh Vending-Bench 2 data, AI economic efficiency grows strictly linearly — averaging +$822 in net profit per month (R² = 0.95).
Back in spring 2025 the models were hovering around zero; now the Claude Opus and GPT-6 Astra lineups have blown past $11,000–$15,000 in profit per simulated year. The agents flat-out learned to run a business in the black.
How Pion works under the hood
This isn't one omnipotent chatbot — it's a distributed software system:
- **AI employees as code.** Every agent in the company is isolated in its own GitHub repo.
- **Agents write their own software.** Their core job is coding scripts for their routine operations, cutting down on errors.
- **A corporate Slack for neural networks.** Bots are split by department (procurement, finance, marketing) and talk to each other — and to the founders — through a single messaging layer.
The agent gets real levers to pull: bank accounts, email, phone lines, a browser, and server capacity.
Who's using it, what does it cost, and how do you sign up
The product is in closed beta — access is granted by application.
There's no official public pricing: the main cost is API tokens, since a staff of agents is constantly talking to each other, generating code, and testing it.
Early feedback from Hacker News confirms it: people are already handing agents operations and lead gen. But there's a catch — like human junior hires, the neural networks need weeks of onboarding to actually get up to speed on a specific business.
Why this is safe (and the agent won't file another FBI report)
Instead of banking on "the perfect prompt," Andon Labs built in artificial bureaucracy:
- **Cross-checking by different models.** Critical actions don't get executed blindly — a task planned by Claude gets validated by an agent running on Gemini or GPT.
- **Access isolation.** Agents don't have root access: a procurement bot going haywire physically can't reach the client database or the company's master account.
- **Hierarchy.** Manager bots continuously run tests to make sure the worker agents' actions don't conflict with the business's core KPIs.
What this means for the market
Nobody needs an "AI assistant" anymore. Andon Labs is (once again) capturing the shift to Service-as-a-Software — a model where the agent doesn't advise, it executes the business function end-to-end. The metric for how impressive an AI is stops being benchmarks in a vacuum and becomes P&L (profit and loss).
It'll be wild when (and if) agents can start closing autonomous deals on our behalf. As in: my agent walks up to a seller's agent, hashes out terms, haggles, and buys what I need. Agent2agent economy!