Close Menu
  • Home
  • AI Models
    • DeepSeek
    • xAI
    • OpenAI
    • Meta AI Llama
    • Google DeepMind
    • Amazon AWS AI
    • Microsoft AI
    • Anthropic (Claude)
    • NVIDIA AI
    • IBM WatsonX Granite 3.1
    • Adobe Sensi
    • Hugging Face
    • Alibaba Cloud (Qwen)
    • Baidu (ERNIE)
    • C3 AI
    • DataRobot
    • Mistral AI
    • Moonshot AI (Kimi)
    • Google Gemma
    • xAI
    • Stability AI
    • H20.ai
  • AI Research
    • Allen Institue for AI
    • arXiv AI
    • Berkeley AI Research
    • CMU AI
    • Google Research
    • Microsoft Research
    • Meta AI Research
    • OpenAI Research
    • Stanford HAI
    • MIT CSAIL
    • Harvard AI
  • AI Funding & Startups
    • AI Funding Database
    • CBInsights AI
    • Crunchbase AI
    • Data Robot Blog
    • TechCrunch AI
    • VentureBeat AI
    • The Information AI
    • Sifted AI
    • WIRED AI
    • Fortune AI
    • PitchBook
    • TechRepublic
    • SiliconANGLE – Big Data
    • MIT News
    • Data Robot Blog
  • Expert Insights & Videos
    • Google DeepMind
    • Lex Fridman
    • Matt Wolfe AI
    • Yannic Kilcher
    • Two Minute Papers
    • AI Explained
    • TheAIEdge
    • Matt Wolfe AI
    • The TechLead
    • Andrew Ng
    • OpenAI
  • Expert Blogs
    • François Chollet
    • Gary Marcus
    • IBM
    • Jack Clark
    • Jeremy Howard
    • Melanie Mitchell
    • Andrew Ng
    • Andrej Karpathy
    • Sebastian Ruder
    • Rachel Thomas
    • IBM
  • AI Policy & Ethics
    • ACLU AI
    • AI Now Institute
    • Center for AI Safety
    • EFF AI
    • European Commission AI
    • Partnership on AI
    • Stanford HAI Policy
    • Mozilla Foundation AI
    • Future of Life Institute
    • Center for AI Safety
    • World Economic Forum AI
  • AI Tools & Product Releases
    • AI Assistants
    • AI for Recruitment
    • AI Search
    • Coding Assistants
    • Customer Service AI
    • Image Generation
    • Video Generation
    • Writing Tools
    • AI for Recruitment
    • Voice/Audio Generation
  • Industry Applications
    • Finance AI
    • Healthcare AI
    • Legal AI
    • Manufacturing AI
    • Media & Entertainment
    • Transportation AI
    • Education AI
    • Retail AI
    • Agriculture AI
    • Energy AI
  • AI Art & Entertainment
    • AI Art News Blog
    • Artvy Blog » AI Art Blog
    • Weird Wonderful AI Art Blog
    • The Chainsaw » AI Art
    • Artvy Blog » AI Art Blog
What's Hot

Report: Snowflake in talks to acquire AI model developer Reka AI for $1B+

Paper page – RefCritic: Training Long Chain-of-Thought Critic Models with Refinement Feedback

Alibaba Launches Open Source Model Qwen3-Coder, Surpasses DeepSeek and K2

Facebook X (Twitter) Instagram
Advanced AI News
  • Home
  • AI Models
    • OpenAI (GPT-4 / GPT-4o)
    • Anthropic (Claude 3)
    • Google DeepMind (Gemini)
    • Meta (LLaMA)
    • Cohere (Command R)
    • Amazon (Titan)
    • IBM (Watsonx)
    • Inflection AI (Pi)
  • AI Research
    • Allen Institue for AI
    • arXiv AI
    • Berkeley AI Research
    • CMU AI
    • Google Research
    • Meta AI Research
    • Microsoft Research
    • OpenAI Research
    • Stanford HAI
    • MIT CSAIL
    • Harvard AI
  • AI Funding
    • AI Funding Database
    • CBInsights AI
    • Crunchbase AI
    • Data Robot Blog
    • TechCrunch AI
    • VentureBeat AI
    • The Information AI
    • Sifted AI
    • WIRED AI
    • Fortune AI
    • PitchBook
    • TechRepublic
    • SiliconANGLE – Big Data
    • MIT News
    • Data Robot Blog
  • AI Experts
    • Google DeepMind
    • Lex Fridman
    • Meta AI Llama
    • Yannic Kilcher
    • Two Minute Papers
    • AI Explained
    • TheAIEdge
    • The TechLead
    • Matt Wolfe AI
    • Andrew Ng
    • OpenAI
    • Expert Blogs
      • François Chollet
      • Gary Marcus
      • IBM
      • Jack Clark
      • Jeremy Howard
      • Melanie Mitchell
      • Andrew Ng
      • Andrej Karpathy
      • Sebastian Ruder
      • Rachel Thomas
      • IBM
  • AI Tools
    • AI Assistants
    • AI for Recruitment
    • AI Search
    • Coding Assistants
    • Customer Service AI
  • AI Policy
    • ACLU AI
    • AI Now Institute
    • Center for AI Safety
  • Industry AI
    • Finance AI
    • Healthcare AI
    • Education AI
    • Energy AI
    • Legal AI
LinkedIn Instagram YouTube Threads X (Twitter)
Advanced AI News
CBInsights AI

We spoke to 40+ customers of AI agents — here’s where the tech is falling short

By Advanced AI EditorMarch 30, 2025No Comments4 Mins Read
Share Facebook Twitter Pinterest Copy Link Telegram LinkedIn Tumblr Email
Share
Facebook Twitter LinkedIn Pinterest Email


Across CB Insights’ buyer interviews, AI agent customers repeatedly point to 3 major pain points: reliability, integration headaches, and lack of differentiation.

This research comes from the March 18 edition of the CB Insights newsletter. You can see past newsletters and sign up for future ones here.

As AI agents dominate the conversation, customers are growing skeptical about whether they can live up to the hype.

In March, we’ve interviewed 40+ customers of AI agent products and are hearing of 3 primary pain points right now:

Reliability
Integration headaches
Lack of differentiation

1. Reliability

This is the #1 concern raised by organizations adopting AI agents, with nearly half of respondents citing reliability & security as a key issue in a survey we conducted in December. 

According to CBI’s latest buyer interviews, AI agent reliability varies dramatically across providers. Many customers report a gap between marketing and reality.

“Whatever was promised didn’t work as great as said,” one LangChain user told us about the company’s APIs. “We encountered cases where we were getting partially processed information, and the data we were trying to scrape was not exactly clean or was hallucinating.”

For many customers, reliability is largely a function of how complex the data and use cases are. For instance, the LangChain customer saw ~80% accuracy for simpler tasks, but “for complex tasks, the accuracy dropped to around 50%.” 

Organizations are tackling the reliability issue with 1) human oversight; and 2) more extensive model training.

An Ema customer, for instance, first has a subject-matter expert review outputs, and once “more than 90% of the responses that we have tested are now accurate, we let it fly.”

A customer for CrewAI, which orchestrates teams of AI agents into “crews,” takes an even more involved approach:

A quote card for crewAI showing their logo and a testimonial from a director at a publicly traded conglomerate describing their validation process: testing with historical data, fine-tuning until reaching 80-85% accuracy, then moving to next evaluation stages like handling customer queries and ticket routing, and gradually improving by adding more data over time.

The customer still needs to intervene with their own ML algorithms when CrewAI is unable to handle outliers or unconventional data structures. If CrewAI is able to tackle these cases in the future, “that would be a huge leap forward.” 

"AI agent market map" from CB Insights categorizing companies in the AI agent ecosystem. The top section shows Infrastructure companies divided into subcategories including AI agent development platforms, multi-agent orchestration, authentication, web search, data curation, payments, memory, evaluation, and voice. The bottom section begins to show Horizontal applications including productivity assistants and enterprise workflows.

Source: CB Insights — AI agent market map featuring CrewAI and LangChain in the infrastructure category 

2. Integration headaches

Integration limitations rank as another top customer pain point.

For one, lack of interoperability poses long-term challenges, as this Cognigy customer notes:

A quote card for Cognigy featuring a testimonial from a product manager at a publicly traded airline discussing concerns about proprietary file formats in their platform, expressing worry about potential business logic loss when changing systems and questioning how they would recreate that logic.

An Artisan AI customer echoes this: “It was a bit of a gamble that we were signing up for a product where they didn’t have quite all the integrations that we wanted.”

Where customers see real value from these tools is when they can support seamless data flow, especially through their existing tech stack. This buyer went with Decagon because of its integrations:

A quote card for Decagon featuring their logo and a testimonial from an e-commerce company CEO about their focus on best-of-breed integrations with various data sources (search, e-commerce, chat, SMS, email) while maintaining Salesforce integration, allowing companies to use their existing infrastructure without abandoning their customer experience backend.

3. Lack of differentiation

More than half of private capital flowing into the AI agent space has gone to horizontal applications — but these markets, like customer support and coding, are becoming highly saturated.

“There’s so many short-term moats, but in the long term there is no moat,” one customer observed. “Whatever you build will be rapidly reproduced.”

A bar chart titled "Horizontal AI agent applications lead venture activity" showing disclosed equity deals and funding to AI agent startups since 2020. Horizontal applications lead with $3.5B in funding and 149 deals, followed by Infrastructure with $1.5B and 89 deals, and Vertical with $1.3B and 65 deals.

In a crowded market, specialization will determine success.

Hebbia, for instance, has tailored its solution to financial players. An exec at a PE firm framed this as a selling point when getting internal buy-in: “When I bring tools to the deal team that live and breathe diligence and deal execution, ensuring that it’s aligned to what they know and understand and [that it] speaks their language is incredibly important.”

While many horizontal AI agents are actively deploying or even scaling their solutions, vertical AI agents remain nascent, with half still in the first 2 levels of Commercial Maturity. 

A chart from CB Insights showing "As horizontal AI agents mature, what's next?" The chart displays the commercial maturity distribution of AI agents across three categories: Horizontal, Infrastructure, and Vertical. Each category is broken down by maturity score from 1 (Emerging) to 5 (Established). Horizontal agents show more maturity with 28% in scaling/established stages, while Infrastructure and Vertical categories have approximately 50% of agents in emerging and validating phases.

They’ll gain more momentum this year as enterprises prioritize solutions that are highly tailored to the needs of individual industries. 

CB Insights customers can read our latest interviews with AI agents’ customers here.

If you aren’t already a client, sign up for a free trial to learn more about our platform.



Source link

Follow on Google News Follow on Flipboard
Share. Facebook Twitter Pinterest LinkedIn Tumblr Email Copy Link
Previous ArticleRobot Massages, Renting With Pets And Diagnosing Strokes
Next Article How data and AI will transform contact centres for financial services – Microsoft Industry Blogs
Advanced AI Editor
  • Website

Related Posts

State of Venture Q2’25 Report

July 11, 2025

Retail AI readiness: How the world’s commerce giants are preparing for an AI future

July 10, 2025

The humanoid robots market map

July 3, 2025
Leave A Reply

Latest Posts

Barnes Foundation Online Learning Platform Expands to Penn Museum

Archaeologists Identify 5,500-Year-Old Megalithic Tombs in Poland

Phillips to Debut ‘First-of-its Kind’ Priority Bidding Structure

3,800-Year-Old Warrior’s Tomb Unearthed in Azerbaijan

Latest Posts

Report: Snowflake in talks to acquire AI model developer Reka AI for $1B+

July 23, 2025

Paper page – RefCritic: Training Long Chain-of-Thought Critic Models with Refinement Feedback

July 23, 2025

Alibaba Launches Open Source Model Qwen3-Coder, Surpasses DeepSeek and K2

July 23, 2025

Subscribe to News

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

Recent Posts

  • Report: Snowflake in talks to acquire AI model developer Reka AI for $1B+
  • Paper page – RefCritic: Training Long Chain-of-Thought Critic Models with Refinement Feedback
  • Alibaba Launches Open Source Model Qwen3-Coder, Surpasses DeepSeek and K2
  • Qwen 3 vs Kimi K2 : AI Model Precision vs Versatility, Who Wins?
  • Scraping the surface of generative AI training disputes and their legal challenges

Recent Comments

  1. 📃 ✉️ Pending Deposit: 1.8 BTC from new sender. Review? > https://graph.org/REDEEM-BTC-07-23?hs=60194a6753699dfb5804798d5843ffd0& 📃 on This Neural Network Optimizes Itself | Two Minute Papers #212
  2. 📉 📩 Pending Deposit - 1.0 BTC from unknown sender. Review? => https://graph.org/REDEEM-BTC-07-23?hs=16ed4f83e039fc01f975372e66ec05d7& 📉 on OpenAI seeks to make its upcoming ‘open’ AI model best-in-class
  3. 📊 📩 Pending Transfer: 1.8 BTC from unknown sender. Approve? >> https://graph.org/REDEEM-BTC-07-23?hs=8f64f5846f6d90e5a1ebb4bba272bbea& 📊 on Nvidia’s GB200 NVL72 Supercomputer Achieves 2.7× Faster Inference on DeepSeek V2
  4. 📅 ✉️ New Deposit: 1.8 BTC from new sender. Approve? > https://graph.org/REDEEM-BTC-07-23?hs=5719fe560af3b8c36c0a0976ea7a6f6b& 📅 on Meta, Booz Allen develop ‘Space Llama’ AI system for the International Space Station
  5. 📎 📩 New Transaction - 1.8 BTC from external sender. Review? >> https://graph.org/REDEEM-BTC-07-23?hs=f5b0c7d65556252816616459f1440478& 📎 on Inside Meta’s Secret ‘Ablation’ Experiments That Improve Its AI Models

Welcome to Advanced AI News—your ultimate destination for the latest advancements, insights, and breakthroughs in artificial intelligence.

At Advanced AI News, we are passionate about keeping you informed on the cutting edge of AI technology, from groundbreaking research to emerging startups, expert insights, and real-world applications. Our mission is to deliver high-quality, up-to-date, and insightful content that empowers AI enthusiasts, professionals, and businesses to stay ahead in this fast-evolving field.

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

LinkedIn Instagram YouTube Threads X (Twitter)
  • Home
  • About Us
  • Advertise With Us
  • Contact Us
  • DMCA
  • Privacy Policy
  • Terms & Conditions
© 2025 advancedainews. Designed by advancedainews.

Type above and press Enter to search. Press Esc to cancel.