This story is available exclusively to Business Insider subscribers. Become an Insider and start reading now.
Andrej Karpathy thinks we’re getting way too excited about AI, especially when it comes to deploying agents that act without supervision.
In a keynote at an event hosted by Y Combinator earlier this week, the computer scientist said people need to “keep AI on the leash.” The OpenAI cofounder said current large language models still make mistakes no human ever would.
Karpathy likened LLMs to “people spirits” — uncanny simulations of human intelligence that hallucinate facts, lack self-knowledge, and suffer from “amnesia.”
“They will insist that 9.11 is greater than 9.9 or that there are two R’s in ‘strawberry,'” Karpathy said in a talk published on Y Combinator’s YouTube channel on Thursday. “They’re going to be superhuman in some problem-solving domains and then they’re going to make mistakes that basically no human will make.”
Even though LLMs can churn out 10,000 lines of code in seconds, he said, that doesn’t mean developers should sit back and let them run wild. “I’m still the bottleneck,” he said. “I have to make sure this thing isn’t introducing bugs.”
“It gets way too overreactive,” he added.
Karparthy urged developers to slow down and write more concrete prompts.
“I always go in small incremental chunks. I want to make sure that everything is good,” he said.
“It makes a lot more sense to spend a bit more time to be more concrete in your prompts, which increases the probability of successful verification, and you can move forward,” he added.
Karparthy did not respond to a request for comment from Business Insider.
The OpenAI cofounder coined the term “vibe coding” in February to describe the process of prompting AI to write code. The idea, he said, is that developers can “fully give in to the vibes” and “forget the code even exists.”
Related stories
AI still needs supervision
Karpathy isn’t the only one urging caution.
Bob McGrew, OpenAI’s former head of research, said on an episode of Sequoia Capital’s “Training Data” podcast earlier this week that human engineers are still essential — not just to guide AI, but to step in when things get messy.
When something goes wrong or if a project “becomes too complicated for AI to understand,” a human engineer can help break the problem down into parts for an AI to solve.
AI agents are like “genies,” said Kent Beck, one of the authors of the seminal “Agile Manifesto” — they’ll often grant your wish, but not always in the way you’d like them to.
“They will not do what you mean. They have their own agenda,” Beck said on a recent episode of “The Pragmatic Engineer” podcast. “And the best analogy I could find is a genie. It grants you wishes, and then you wish for something, and then you get it, but it’s not what you actually wanted.”
Beck also said results are so inconsistent that using AI to code can sometimes feel like gambling.
Despite the nascent tech’s limitations, even the biggest tech companies are betting on AI for the future of coding. AI writes more than 30% of Alphabet’s new code, up from 25% last year, said CEO Sundar Pichai on the company’s most recent earnings call.