Andrej Karpathy is back, this time explaining how LLMs are rewriting software.
At YC AI Startup School, the former head of AI at Tesla gave a talk titled “Software Is Changing (Again),” during which he discussed with students and developers how the concepts of code, computation, and programming are being rethought at a fundamental level.
He defined three kinds of Software. The first, Software 1.0, consists of traditional programming, in which humans write explicit instructions for computers to execute.
Karpathy said that in Software 2.0, instead of writing code manually (as in Software 1.0), developers work with neural networks, specifically by tuning datasets and using an optimiser (like gradient descent) to learn the weights or parameters of the neural network automatically.
While the third is Software 3.0, where LLMs have made neural networks programmable in a new way. Instead of writing traditional code, users now write prompts in natural language like English, which effectively serve as programs that instruct the model.
“I think a quite fundamental change is that neural networks have become programmable with large language models. I see this as something new and unique, it’s a new kind of computer. In my mind, it’s worth giving it a new designation, Software 3.0,” Karpathy said. He also discussed the rise of vibe coding in recent months and how its growing popularity among kids gives him hope for an exciting future.
Karpathy shared a few apps he built while vibe-coding, like MenuGen (menugen.app), which turns menu text into visuals to help users make sense of it.
Human in the Loop
While LLMs may eventually be able to browse, click, and navigate the web more like humans do, Karpathy believes it’s still valuable to meet them halfway. He said humans should generate content in a format that can be easily understood by LLMs. Karpathy gave the example of Gitingest, which turns any Git repository into a simple text digest of its codebase. This is useful for feeding a codebase into any LLM.
He referred to the next wave of software as partial autonomy apps built on LLMs, where humans continue to play a key role in oversight and control rather than handing over full autonomy. “We have to keep the AI on the leash. A lot of people are getting way overexcited with AI agents,” said Karpathy.
“When I see things like ‘2025 is the year of agents,’ I get very concerned… this is the decade of agents,” he added. He urged the developers to build augmented systems, like Iron Man suits, not Iron Man robots that accelerate human productivity without removing human oversight, as LLMs are still fallible.
Referencing his work on Tesla’s Autopilot, Karpathy pointed out that despite years of development, full autonomy has not yet been achieved, even in vehicles that appear driverless. “There’s still a lot of teleoperation. We haven’t declared success.”
Karpathy referred to LLMs as “people spirits”—superhuman in some ways (like memory or general knowledge) but deeply flawed in others (like hallucinations, logical inconsistencies, or context retention). He said they simulate intelligence but don’t develop knowledge over time like a human would. Instead, they rely on fixed weights and short-term context windows, which he compares to working memory.
He cited tools like Perplexity AI and Cursor as examples of intelligent orchestration of multiple LLM components behind the scenes and mechanisms for human-in-the-loop verification. Crucially, these apps also offered what Karpathy called an “autonomy slider,” allowing users to control how much freedom the AI had depending on the complexity and risk of the task.
Build for the Agents
Karpathy said we need a new interface built specifically for agents. He explained that a new kind of software user has arrived—neither a person clicking through a GUI nor a backend system making API calls.
Instead, LLMs represent something in between. Karpathy described them as the third major consumer and manipulator of digital information, urging developers to start designing with them in mind.
Traditionally, software has served two users. Humans through graphical interfaces and computers through APIs. But LLMs occupy a new space. “There’s a new category of consumer,” he said. “Agents. They’re computers, but they are humanlike. People’s spirits on the internet.”
Karpathy said that whenever he uses ChatGPT, he feels like he is talking to an operating system through the terminal. He believes that it should have a new GUI, other than just a text bubble.
LLMs Resemble Fabs and Utilities
He further compared the development of LLMs to semiconductor manufacturing. Building advanced LLMs, he said, involves massive capital investment, proprietary methods, and tightly integrated R&D, similar to running a chip fabrication facility.
“The capex required for building LLMs is actually quite large,” he said. “We have deep tech trees, R&D secrets, centralised in LLM labs.”
Beyond hardware analogies, Karpathy’s central argument is that LLMs are evolving into full-fledged operating systems. These models coordinate memory, computation, and interaction much like a traditional OS. “The LLM is a new kind of computer—it’s like the CPU. Context windows are like memory. And the LLM orchestrates memory and compute.”
He pointed to applications like Cursor that can run on any major foundation model like GPT-4, Claude, and Gemini as examples of this platform-agnostic future. “You can take an LLM app like Cursor and run it on GPT or Claude, or Gemini. That’s kind of like downloading an app and running it on Windows, Linux, or Mac.”
We’re Back in the 1960s of Computing
At present, LLMs remain centralised and expensive to run, which Karpathy compared to the mainframe era of the 1960s. Instead of personal computers, we’re using interfaces like ChatGPT that tap into vast cloud-based models. “LLM compute is still very expensive, so they’re centralised in the cloud and we are all just thin clients interacting with it.”
He noted early signs of a shift. Some developers are already experimenting with running smaller models locally on consumer hardware like Mac Minis, but a true personal computing revolution for LLMs is still far off.
Karpathy likened LLMs to electricity: centralised, metered, and essential. Labs like OpenAI and Anthropic invest heavily in training their models, then serve intelligence over APIs, much like utilities deliver power.
When these services go offline, the impact is immediate. “It’s like an intelligence brownout. The planet just gets dumber for a while.”
But unlike electricity, LLMs are not bound by physical laws. They are shaped by data, architecture, and training methods. This flexibility changes how we build, share, and improve them, turning LLMs into more than just a utility. They’re becoming a programmable layer of intelligence for the internet.
