Ivan Zhang says AI firms have their work cut out winning back clients “burned” by failed pilots.
At Web Summit Vancouver, Cohere co-founder Ivan Zhang addressed the proof-of-concept fatigue that has taken hold among enterprises racing to adopt the technology without a resulting payoff.
During a panel discussion yesterday with Tailscale co-founder and CEO Avery Pennarun, Zhang acknowledged the frustration. “Everyone is tired of [proofs of concept],” he said.
He noted many of the clients of his Toronto-based enterprise large language model (LLM) company have built initial applications, but have not moved them into production. The issues range from cost and governance to data security and privacy requirements, which Cohere hopes to help address with its new workspace platform product, North.
“The next phase of go-to-market for this technology is, ‘where is the ROI?’”
Ivan Zhang, Cohere
In a follow-up interview with BetaKit, Zhang acknowledged that many firms have not seen the return on investment (ROI) necessary to justify their AI spending.
“Sometimes the systems they end up building, the cost of the model itself is more expensive than the humans that are actually running it,” he said.
Zhang added that Cohere has seen cases where customers have sought to augment existing workforces with AI but did not ultimately see the productivity of those teams increase. “In those cases, the human just did less work with the same amount of output.”
Zhang anticipates that AI startups will now be tasked with winning back companies “burned” by projects that didn’t pan out. “The next phase of go-to-market for this technology is, ‘where is the ROI?’”
A recent National Bureau of Economic Research working paper surveyed 7,000 workplaces to determine if AI chatbots impacted their bottom line and found “no significant impact on earnings or recorded hours in any occupation.” Another study from Boston Consulting Group found only a quarter of the 1,800 executives surveyed have seen significant value from AI so far, and determined companies have typically extracted greater value by focusing their AI investments than diluting efforts across multiple pilots.
RELATED: Web Summit Vancouver’s opening night offers thoughtful critiques and alternatives to current tech
Zhang agrees with the data. On stage, he advised firms considering leveraging LLMs to focus first on business problems rather than spending large amounts of time and money building flashy solutions without clear use cases.
“Don’t get lost in building something and searching for a problem,” he said.
Zhang argued that AI and AI agent adoption may just be one part of the solution for companies seeking ROI. “Like any other … era of software, it is just a tool in the toolbox to ultimately solve a business problem [and] create value for your customers.”
Zhang’s remarks—and New York University professor emeritus Gary Marcus’ pointed criticisms of generative AI during Web Summit Vancouver’s opening night—represent a counterpoint to some of the continued hype from across the industry about the tech’s potential to solve all of the world’s problems.
RELATED: “The reality check is happening now”: AI leaders on the state of the market at Collision
On stage, Marcus pointed to the “little progress” made in reducing the propensity of LLMs to generate false or fabricated information, describing them as “auto-complete on steroids.” He went on to add that the assumption that generative AI’s hallucination problem can be solved by having models ingest more training data has been proven false.
LLM hallucination rates have remained stubbornly high: the latest models from companies like OpenAI, Google, and DeepSeek are generating more errors, not fewer.
Despite some progress on this front, Cohere still appears to rank behind other players in the space. According to Hugging Face’s LLM hallucination leaderboard, Cohere’s latest Command A offering generates incorrect and made-up answers at a lower rate than its previous models, but ranks 66 overall globally in this respect, behind products from a range of competitors.
Zhang acknowledged to BetaKit that hallucination remains a problem in generative AI. He said the company has tried to help address this through transparency, including showing users “the raw thinking” of its LLMs, what tools its systems use and how, and citations to derived answers.
RELATED: Outcome of copyright case against Cohere uncertain but likely “precedent-setting”: expert
Cohere’s most recent funding round, a $500-million USD Series D at a $5.5-billion valuation ($687 million CAD at $7.6-billion at the time), brings its total funding to approximately $1 billion. But as the startup faces off against better-funded competitors with even deeper pockets, Zhang and Cohere—like Hugging Face’s Sasha Luccioni—believe that bigger is not always better when it comes to building more cost-effective and relatively less energy-intensive AI models.
Zhang argued that a model is “only as good as the data and systems it can access.” He added that Cohere’s products are built to be run completely in their customers’ environment. “That forces us to build even more efficient models.”
Zhang touted Cohere’s “intense growth” and said the “relatively nascent” nature of the space leaves plenty of room for the company to expand.
“It is just a tool in the toolbox to ultimately solve a business problem [and] create value for your customers.”
Ivan Zhang, Cohere
The state of Cohere’s growth has been a recent topic of focus for tech media. Cohere reached $100 million USD ($138 million CAD) in annualized revenue this month after more than doubling its sales since the start of 2025, and CEO Aidan Gomez recently told Bloomberg the company was “not far away” from profitability. But The Information has reported this is still $350-million USD behind what Cohere told investors in 2023 it expected to be making annually by now.
Revenue targets and stiff competition are not the only challenges Cohere must contend with: the AI startup also has what one expert called a potentially “precedent-setting” copyright-infringement lawsuit from major media companies on its plate. A group of media organizations including the Toronto Star, Condé Nast, and Vox have alleged Cohere scraped media content without consent and used it to train AI models, accessed content in real time without permission, and generated infringing outputs. Cohere is just one of many AI startups facing similar lawsuits.
Cohere has denied these claims, arguing that the suing publishers had gone out of their way to “manufacture” a case and disputed the notion that any practical copyright infringement had occurred.
Zhang declined to offer much comment on the matter, pointing BetaKit to a blog post detailing Cohere’s thinking. “We’re confident in that,” he said.
Feature image by Vaughn Ridley/Web Summit via Flickr.
