The Gist
AI still trails human expertise. Scale AI is testing against human-level benchmarks—and models aren’t close yet. Collaboration beats replacement. AI may enhance performance in CX roles rather than eliminate them. Customer satisfaction rises with hybrid models. When AI supports—not replaces—agents, satisfaction scores improve.
Scale AI is working to bring artificial intelligence up to human-level proficiency by having deeply skilled humans teach it what they know. The company’s contributor network is made up of 12% PhDs and 40%+ masters and professional degree holders who are training the AI in their areas of expertise.
Table of Contents
AI Isn’t Replacing Humans Anytime Soon
And yet Scale, at least publicly, wants you to know that your job is probably not at risk from artificial intelligence.
Field Chief Technology Officer Vijay Karunamurthy reached out after reading my story about AI being able to do parts of my job and asked to give a counterpoint. He told me, in short, that modern day AI isn’t measuring up to the benchmarks the company set for human-level performance. And even if it approaches that level, the AI will likely be a complement to human workers and not necessarily a replacement.
“Even the most powerful AI models still lack the creativity, emotional intelligence and nuanced decision-making that people bring to their work,” Karunamurthy said.
Scale sets benchmarks for human-level capabilities in certain professions and found a sizable gap between the humans and machines so far, Karunamurthy said. In finance, for instance, the company tested AI’s ability to give retirement advice and found that it lacks the empathy to adeptly handle high stress situations.
“We know from these benchmarks, the models can’t yet replace a really good wealth manager,” he said. When AI prepares investment advisors in advance of a call though, the company found customer satisfaction scores tend to rise meaningfully.
Related Article: Why the Future of Customer Service Depends on Human-AI Collaboration
When AI Supports, Performance Improves
Previously, I argued that AI can replace a number of tasks I do as a journalist, and not the job itself, but Karunamurthy’s point is well-stated. Often, the narrative of AI replacing humans assumes that companies are happy with their workers’ output and would reduce their workforce if AI could do human tasks.
But in reality, AI can often only do some things better than people. And when humans work alongside AI they can do a better job than when working alone, leading to better overall performance.
In medicine, for example, someone might be able to speak with a chatbot about an issue they’re having, but that likely doesn’t measure up to a doctor or nurse who comes to a conversation with a patient’s medical context synthesized and presented by AI.
And in law, we’ve seen ChatGPT try, and fail, to argue legal cases. But lawyers working with legal bots like Harvey can sort through documents more efficiently, leading to better performance.
“The legal domain is a creative field where you learn over time how to leverage your knowledge, your thinking, to solve really challenging problems,” Karunamurthy said, “AI is still not able to replicate that or replace that.”
View all
Customer Experience Will Still Need Humans — for now
Still, as Karunamurthy spoke, he mentioned “really good” lawyers and doctors and not average ones. And as AI capabilities improve — and they are moving fast — it’s unlikely to be a painless transition for everyone. Even as he waxed optimistically about our AI future, Karunamurthy allowed that customer service is one area where average employees could be replaced.
“If you have a customer support call center where people are waiting on the phone for 10 minutes, 20 minutes to try getting an answer, and suddenly replace that with an agent that can get you an answer right away, customer satisfaction scores go through the roof,” he said.
Should a robot takeover of work be en route, it at least doesn’t appear imminent. Scale, for instance, worked with the Center for AI Safety to design Humanity’s Last Exam, a compendium of about 2,700 difficult questions submitted by more than 1,000 professors, researchers and advanced degree holders. The top models today are only able to answer about 20% of the questions, showing a wide gap between AI and experts in their fields.
The 20% result is reassuring, but just a few months ago the AI was barely reaching 10%. With technology moving this fast, predictions about its labor implications are difficult to get right. The one thing that’s certain though, is big change is on the way.
Learn how you can join our contributor community.