The Eval Stack: Proving Agents Are Right, Not Just Claiming It

Most AI research tools simply point a large language model at the internet and hope the output is accurate. Saarth Shah, the founder of Sixtyfour, took the opposite approach: grade everything, and only ship what actually improves the score. He keeps a visible scoreboard. Every new version of Sixtyfour’s research agents is tested against a carefully curated set of questions, assembled by hand by a team of experts. Only when the score goes up does the feature go live.

This method, which Shah calls the Eval Stack, flips the usual AI development script. Instead of claiming an agent is smarter or more reliable, the system proves it with data. The evaluation set isn’t static either. It grows as the product matures, incorporating edge cases and real-world queries that users have submitted. That means the benchmark is always relevant, not just a dusty academic test.

Shah argues that most AI companies skip this step because it is slow and expensive. Hiring experts to write and verify hundreds of nuanced questions takes time and money. But he believes the trade-off is worth it. Without a rigorous evaluation layer, users have no way to know if an agent is actually improving or just getting better at sounding confident.

Sixtyfour’s agents are designed for deep research tasks, where accuracy matters more than speed. The company targets professionals who need verified answers, not just plausible summaries. By tying every product change to a measurable improvement on the eval set, Shah ensures that the agent’s next answer is more likely to be right than the last one. That is the difference between claiming an agent works and proving it does.