The scaffolding layer that developers once needed to ship LLM applications – indexing layers, query engines, retrieval pipelines, carefully orchestrated agent loops – is collapsing. And according to Jerry Liu, co-founder and CEO of LlamaIndex, that’s not a problem. This is the thing.
“As a result, there’s less need for a framework to actually help users build these deterministic workflows in a lighter and shallower way,” Jerry Liu, co-founder and CEO of Laminex, explains in a new VentureBeat Beyond the Pilot podcast.
The context is becoming an abyss
Liu’s Lindex is one of the leading retrieval-augmented generation (RAG) frameworks connecting private, custom, and domain-specific data to LLM. But he also acknowledges that such frameworks are becoming less relevant.
He notes that with each new release, the models demonstrate an increasing ability to reason on “huge amounts” of unstructured data, and they are getting better at it than humans. They can be relied upon to reason extensively, self-correct, and make multi-step plans; Modern Context Protocol (MCP) and Cloud Agent Skills plug-ins allow models to find and use tools without requiring independent integration for each.
The agent pattern has consolidated toward what Liu calls "Managed Agent Diagram" – A harness layer combined with tools, MCP connectors, and skill plug-ins instead of custom-built orchestration for each workflow.
Additionally, coding agents excel at writing code, meaning developers don’t need to rely on extensive libraries. In fact, about 95% of LlamaIndex code is generated by AI. “Engineers aren’t actually writing real code,” Liu said. “They’re all typing in natural language.” This means that the layers between programmers and non-programmers are collapsing, because “the new programming language is essentially English.”
Instead of manual coding or struggling to understand API and documentation integration, developers can simply point to cloud code. “These types of things were either extremely ineffective or would break the agent just three years ago,” Liu said. “It’s very easy for people to create relatively advanced retrievals with extremely simple primitives.”
So what is the main differentiator when the stack collapses?
Context, says Liu. Agents must be able to understand file formats to extract the correct information. Providing high accuracy and cheap parsing becomes important, and due to its development with agentive document processing through optical character recognition (OCR), Laminedex is well positioned here.
“We’ve really identified that there’s a core set of data that’s locked away in all these file format containers,” he said. Ultimately, “Whether you use OpenAI codecs or cloud code, it doesn’t really matter. The thing they all need is context.”
keep the stack modular
There is growing concern about anthropic locking in session data such as builders; In light of this, Liu emphasizes the importance of modularity and agnosticism. Builders should not bet on any one frontier model, or overbuild in a way that complicates the components of the stack.
Recovery has evolved into an “agent-plus-sandbox,” as he describes it, and enterprises must ensure their code bases are free of technical debt and adaptable to changing patterns. They also have to accept that some parts of the pile will eventually have to be thrown away.
“Because with every new model release, there’s always a different model that is kind of the winner,” Liu said. “You want to make sure you have some flexibility to really take advantage of it.”
Listen to the podcast to learn more about it:
- LlamaIndex started as a ‘toy project’, initially with only 40% accuracy;
-
How can SaaS companies take advantage of complex workflows that must be standardized and repeatable for workers with average knowledge;
-
Why vertical AI companies are moving forward and why ‘build vs buy’ is still a very valid question in the agent era.
You can also listen and subscribe beyond the pilot But spotify, Apple Or wherever you get your podcasts.
<a href