Ex-Deepmind, David Silver – founder of Ineffable Intelligence, published (along with Richard Sutton), in 2025, a paper titled “Welcome to the Era of Experience”. In it he outlines a compelling thesis: that the next frontier of AI won’t be built primarily on static human data, but on agents that learn continuously from experience i.e. self-improving systems that refine themselves through interaction with environments.
If you read the paper carefully, it’s not hard to see why markets might anchor significant valuation to this vision. The prospect of self-learning, self-improving agents, systems that compound capability over time, represents a structural shift, not an incremental one.
Conceptually, the idea is plausible. Reinforcement learning, self-play, and autonomous feedback loops already exist in constrained domains. Scaling that paradigm into more general environments is an engineering challenge, but not a science fiction one.
Below are some of the questions that came to mind as I read through David and Richard’s paper – questions, I hope investors would be asking, too:
1. Who sets the objectives?
Even the most autonomous agent optimises something. Reward functions, utility functions, evaluation metrics – these do not emerge in a vacuum. Humans define them. If objectives shape behavior, then whoever defines objectives shapes outcomes.
2. Who validates learning and outputs?
Self-improvement does not eliminate the need for verification. Performance benchmarks, safety audits, vulnerability assessment, and alignment evaluation still require human oversight. Trust requires external validation.
3. Architecture: one super-agent or many?
Do we converge toward a single vertically integrated super-agent? Or a network of specialized, interoperable agents? If the latter, access control, inter-agent communication protocols, and permissioning layers become critical. Who governs these interfaces? Who authorises coordination? Human institutions, presumably.
4. What defines AGI?
Is AGI a technical milestone (e.g. general problem-solving across domains)?
An economic threshold (outperforming humans at most cognitive tasks)?
A capability benchmark?
A regulatory classification?
There is no universally accepted operational definition. Until there is, “AGI achieved” will remain partly a narrative construct – shaped by benchmarks, incentives, and interpretation. We are already seeing this with LLMs.
Lastly,
Are these valid questions? I believe so.
Is the idea worth pursuing? Yes, absolutely.
Should the British government back this? Yes, if it is going to remain in Britain and British.
Download Welcome to the Era of Experience from original source.
First dropped: | Last modified: February 19, 2026