One of Big Tech’s most prominent AI scientists wants to dismantle the current approach to building human-level AI. Yann LeCun has indicated that what we need is not a large language model, but a “world model”.
LeCun, the chief AI scientist for “fundamental AI research” at Meta, is expected to resign from Meta soon, according to multiple reports from reliable outlets. Lekan is a 65-year-old senior statesman in the world of AI science, and he has unlimited resources to serve as the big AI brain at one of the world’s largest tech companies.
Why is he leaving a company that is spending wildly, poaching the most skilled AI experts from other companies, and, according to CEO Mark Zuckerberg’s July blog post, making such astonishing leaps in its own that reportedly “the evolution of superintelligence is now visible”?
He’s been pointing north for a really long time. When it comes to human-level intelligence, LeCun has recently become notorious for saying LLMs as we currently understand them are useless – no longer worth pursuing, no matter how big Big Tech makes them. He said in April last year that “LLM is basically an off-ramp, a distraction, a dead end.” (Prominent AI critic Gary Marcus has criticized LeCun for “belligerently” defending LLM and then flip-flopping on Marcus’s own criticisms.)
A Wall Street Journal analysis of Laken’s career published Friday points to some other possibilities about the reasons for his departure in light of this belief. Last summer, a 28-year-old man named Alexander Wang – who was the co-creator of the LLM-based sensation ChatGPT – became head of AI at Meta, making a budding LLM fanatic Lecan’s boss. And Meta brought on another relatively young chief scientist, Shengjia Zhao, to work on LeCun this year. Meta’s announcement about Zhao’s new role clearly indicates a huge “breakthrough” he has provided. Lacan says he has lost faith in scalping.
If you’re wondering how LeCun can be a chief scientist while Zhao is also a chief scientist, that’s because Meta’s AI operation seems like it has a singular org chart, divided into multiple, distinct groups. Apparently in an effort to fix all this, hundreds of people were laid off last month.
A Financial Times report on LeCun earlier this week suggests that LeCun will now found a startup focused on the “world model.”
Again, Lacan is not shy about why he thinks world models have the answers to what AI needs. He gave a detailed speech about this at the AI Action Summit in Paris in February, but the US representative, Vice President JD Vance, gave an aggressive speech about how it would be better to get out of the US’s way on AI.
Why is Yann Lecan attracted to world models?
As LeCun, who worked on Meta AI smart glasses, said in his speech, but No Meta’s Llama LLM is a big believer in wearables – to a large extent.
It’s amazing how Ray-Ban Meta glasses can help visually impaired people. https://t.co/w3ZxCFtTlE
– Yann Lecun (@ylecun) 30 September 2024
We will need to interact with wearables in the future As if they were people, He thinks, and LLMs don’t understand the world like people do. With an LLM, he says, “We can’t even reproduce cat intelligence or mouse intelligence, let alone dog intelligence. They can perform amazing feats. They understand the physical world. Any domestic cat can plan very complex tasks. And they have causal models of the world.”
LeCun offers a thought experiment to illustrate what he thinks is – if you like – a world model, and it’s something he thinks any human can easily do that LLM simply cannot:
“If I say to you ‘Imagine a cube floating in the air in front of you. Okay now rotate this cube 90 degrees around a vertical axis. What does that look like?’ It’s very easy for you to create this mental model of a rotating cube.”
With very little effort, an LLM can write a nasty limerick about a spinning, rotating cube, sure, but that’s not really going to help you have a conversation with anyone. Lacan believes this is due to the difference between textual data and data obtained from processing many parts of the world that are not textual. While LLMs are trained on a volume of text that would take 450,000 years to read, LeCun says, a four-year-old child who is awake for 16,000 hours has processed 1.4 x 10^14 bytes of sensory data about the world, either with his eyes or by touch, which he says is more than LLMs.
By the way, these are just the estimates that LeCun gave in his speech, and it should be noted that he has given others as well. However, the abstraction the numbers are pointing to is that LLMs are limited in ways that Lacan thinks world models would not be.
What model does LeCun want to make and how will he make it?
LeCun has already started working on world models in the meta – including creating an introductory video that leads you to imagine a rotating cube.
LeCun’s dream model as described in his AI Action Summit speech involves including a current “estimate of the state of the world” as some kind of abstract representation. EverythingOr at least everything that is relevant in the current context, and instead of sequential, symbolic prediction, it “predicts the resulting state of the world that will occur after you take a sequence of actions.”
World models will allow future computer scientists to build, he says, “systems that can plan actions – possibly hierarchically – so as to accomplish a purpose, and systems that can reason.” LeCun also emphasizes that such systems will have more robust security features, because the methods by which we control them will be built into them, rather than being mysterious black boxes that spit out text, and which have to be refined by fine tuning.
What LeCun says is that classical AI – such as the software used in search engines – can all be reduced to optimization problems. He suggests that his world model will look at the current state of the world and find efficient solutions that are compatible with a few different states. “You want an energy function that measures the incompatibility, and given an
Again, these are credible reports of leaked information about LeCun’s plans, and he hasn’t even confirmed that he’s exploring anything new. If what we can glean from Lacan’s public statements seems tentative and a bit vague at the present stage, it should be so. It seems LeCun has a moon in mind, and he’s pushing for another ChatGPT-like burst of superhuman abilities. Anything truly noteworthy could take years – or indeed forever – to materialize, not to mention billions of dollars from investors.
Gizmodo contacted Meta for comment on how LeCun’s work fits into the company’s AI mission, and will update if we hear back.