AI’s Big Problem: Labs Are Missing What Users Actually Want
▼ Summary
– Tim O’Reilly measures success by creating more value than captured, applying this to AI by advocating for open-source systems that give designers and users control, not just open-weight models.
– He criticizes big labs for building an “architecture of control” that tracks users, arguing that frontier models optimize for narrow use cases and may be worse for ordinary needs, unlike widely diffused lower-level models.
– O’Reilly claims frontier AI models pose greater security risks, like cybersecurity threats, than open-weight models, so risks justify slowing frontier development rather than restricting open source.
– He suggests frontier models may become like mainframes—useful for hard problems but not widely adopted—while open-source efforts, like the Pi agentic harness, enable broader innovation.
– His nonprofit, the AI Disclosures Project, promotes an open-memory consortium to counter Meta’s lock-in strategy, ensuring users can switch models and providers while keeping their context.
Tim O’Reilly has long measured the value of a company, a person, or a society by a simple rule: create more value than you capture. So it makes sense that the publisher, internet pioneer, VC, conference organizer, and tech sage applies that same standard to how people build and use AI. He’s now championing a future where open-source AI acts as a democratizing force for the public. His concern is that today’s cloud giants are repeating the playbook of Microsoft in the 1990s, trying to trap users inside their own ecosystems. To counter that, he’s pushing for open-sourcing AI technology, not just making technical specs like neural-net weights publicly available, but opening up the entire AI stack so designers and users have full control.
For O’Reilly, AI is a fresh creative medium, one he uses heavily and even documents in a blog dedicated to his conversations with it. During a recent discussion, it became clear we hold opposing views on whether AI can truly generate original work. It’s not hard to guess where each of us landed.
STEVEN LEVY: You’re fully committed to open-source AI. Make your argument.
TIM O’REILLY: First, we need to agree on what we mean. Most people use “open-source AI” to mean open-weight models, but it’s a much bigger concept. Back in the ’90s, while everyone focused on open-source licenses, I was saying, “No, no, it’s about the system’s architecture. Does it allow participation?”
Why does that matter?
The major labs are misreading the future. They’ve convinced themselves that having the largest, most advanced model is the path to victory. Big models like Claude are tuned for specific use cases, but those aren’t necessarily what people actually want. I want to add my own unique touch. The key is a clear separation between the model, the harness, and the application. Right now, we don’t have that. They’ve built an infrastructure of control instead of one built on freedom and participation, which means they can track your every move.
Isn’t it against big companies’ best interests to give up that control?
Absolutely, it’s against their interests. But that doesn’t mean their strategic choice is correct. For a while, the newest and most powerful models were clearly better for everything. Now, they’re superior in some areas but worse in others. People are increasingly noting that models like Fable and Sol are weaker writers than their lower-tier counterparts. [Anthropic and OpenAI would likely disagree.] The advances we’re seeing in so-called frontier AI are actually moving us away from what everyday people need. The US could win the frontier AI race, but China might still outpace us because they have smaller, more widely distributed models across society. The real goal is to let people innovate freely and think outside the box.
Some worry that open-source software poses security risks, since bad actors could bypass the guardrails of frontier models.
Every cybersecurity incident we’ve seen so far has come from frontier models. So risks like cyber threats or the potential to create pathogens are actually stronger arguments for slowing down frontier development than for restricting open-weight models.
So you believe open source will strip the major AI powers of their dominance?
I don’t make predictions. But I’ll say the world is moving in the direction I hoped. What if the big frontier models end up like mainframes or supercomputers, tackling only the toughest problems that never really spread across society? People are already building things like Pi, an open-source agentic harness. At my nonprofit, the AI Disclosures Project, we’re exploring the concept of an open-memory consortium. Mark Zuckerberg’s strategy is to lock you in by offering the AI that knows you best. The open-source vision must reject that. Open source can let you switch models and providers while keeping all the context you need.
(Source: Wired)
