Over the past few years, many of us have gotten a crash course in what we now call artificial intelligence—but really, it has mostly been a crash course in large language models. Increasingly, however, LLMs are no longer the only category of AI drawing high expectations, massive funding rounds, and significant research and product development.
Over the past year, we’ve seen a plethora of new announcements in a category labeled “world models,” and you’ll likely see more movement there in the coming months and years.
Instead of or in addition to working with language, world models aim to lay the groundwork for AI systems that are capable of simulating the physical world, or at least a useful approximation of it.
To examine what’s different and important about this idea, Ars spoke with three expert practitioners working on world models and related technologies: Vincent Sitzmann from MIT, Anastasis Germanidis from Runway, and Ben Mildenhall from World Labs.
From these conversations, we learned that while LLMs-as-a-product started with an interface (chat) and then sought a use case, the big players in world models right now are arguably working in the other direction: They’re starting with specific use cases and applications in robotics, research, and asset generation, but it’s unclear exactly how the interfaces, systems, and tools will ultimately look.
Table of Contents
The off-ramp from LLM disillusionment
As you’ll soon see, there are many parallels between LLMs and world models in terms of architecture and how people expect them to improve over time. For some, though, they’re seen as a potential answer to the limitations of LLMs, even though work on them predates that contemporary narrative.
“The idea that you’re going to extend the capabilities of LLMs to the point that they’re going to have human-level intelligence is complete nonsense,” former Meta chief AI scientist Yann LeCun told Wired earlier this year. LeCun has made waves with an opinion that some working in AI and LLMs see as contrarian, but he’s actually speaking for a sizable segment of the field.
