‘We believe that if you begin with the right approach, you can take your success rate up double, maybe triple, of that 25 percent,’ Krishna says.
Only 25 percent of CEOs are seeing the return on artificial intelligence they expected, but they are still investing more in AI–presenting an opportunity to IBM as it navigates the engineering challenges with unlocking value with the emerging technology.
This was one of the big takeaways this week from comments made by Arvind Krishna–CEO of the Armonk, N.Y.-based mainframe, cloud and AI vendor–during a virtual press conference and during his keynote address at IBM Think 2025. The conference was held this year in Boston.
“That 25 percent number … the biggest reason (is because AI) often is done as something kind of outside or to the side of people’s existing infrastructure,” Krishna said. “If you use some of our platforms, we don’t restrict you to only AI models and agents that come from IBM. While we provide some, we are also very open to those that partner with us. … We believe that if you begin with the right approach, you can take your success rate up double, maybe triple, of that 25 percent.”
[RELATED: IBM Think 2025: The Biggest News In AI, Linux]
IBM Think 2025
Krishna and other IBM executives used the event to reveal advancements in the vendor’s AI and Linux wares. The CEO shared ways that consulting firms are leveraging AI technology to create their own platforms and free up humans for more complex tasks.
“We believe that agents are really going to help redefine how people build applications,” he said. “We talk about 1 billion new applications that are going to be built over the next four years. Agents are going to form at least a third, if not more, of these applications.”
And Krishna told the Think audience that mainstream adoption of quantum computing technology isn’t as far away as some believe.
“We have built 75 actual quantum computers, not simulators, not just software. You can access 13 of them today on the cloud from IBM,” he said. “I would tell you that it’s not 20 years, but it’s in the four-, maybe five-year time frame that you will begin to see incredible advantages from these computers. We have over 250 industry and academic partners now working with us.”
Here’s more of what Krishna had to say during Think 2025.
1 Billion AI Apps
Over the next few years, we expect there will be over 1 billion new applications constructed using generative AI.
A CEO study that we did (showed) that our clients are expecting to double or even increase the investments on AI beyond that.
However, they are finding that only about 25 percent of the time are they getting the ROI that they expect, driven by a lot of factors–access to enterprise data, the siloed nature of the different applications, together with the fragmentation that is happening in the infrastructure.
That is why you see us focus on both hybrid cloud and artificial intelligence together. And we have made a lot of announcements and a lot of innovation on those combined technologies.
That 25 percent number … the biggest reason (is because AI) often is done as something kind of outside or to the side of people’s existing infrastructure. The logging, the monitoring, the security, all of that is missing.
If you use some of our platforms, we don’t restrict you to only AI models and agents that come from IBM. While we provide some, we are also very open to those that partner with us. Mistral, Llama, just to name some examples. But 150 others who provide agents.
So now it’s one integrated infrastructure. So that’s a lot easier for the enterprise to deal with.
We believe that if you begin with the right approach, you can take your success rate up double, maybe triple, of that 25 percent.
Economic Uncertainty
We are actually seeing people double down on their AI investments. As people are looking for productivity, they’re looking for cost savings, but they’re also looking to scale the revenue of their own companies.
AI is one of the unique technologies that can hit at this intersection of all three of those.
EY (for example) is building a part of their tax consultancy work on top of Watsonx and AI models coming from us. If you look at that, it offers productivity because now the consultants can be more productive.
It actually allows them to scale, also, and go more down market because if the cost of providing this consulting is now cheaper, you can scale to many smaller clients. So that is a scale point. And once the platform is built out, it offers them cost savings.
Everybody is doubling down on AI investments, but they’re now looking for that return on AI. So the only change over the last 12 months is that people are stopping experimentation and focusing very much on–where is the value to the business right now?
AI At The Edge
(An example of) how AI moves from the cloud, not just the center, out all the way to the edge, is work we’re doing with Lumen. Lumen is a massive fiber provider.
Working with them, we’re now looking at bringing AI inferencing out to the edge and much, much closer to where it can be.
Whether it’s for reasons of privacy…but also latency. We have examples in the industry where people who are worrying about pilferage in retail stores want to put cameras coupled with AI technologies, and they need reaction times that are in the milliseconds in order to be able to stop those things in real time.
So there’s a lot of use cases where we think that these are moving out all the way from the center to the edge. And network providers will become a much closer part of these delivery mechanisms.
IBM Using AI Internally
We have a lot of experience in this inside IBM.
We looked at our HR processes, and we began two years ago with the top five most common queries that were being done. We automated those.
Every couple of weeks, after that, we kept adding until at this point we have, I think, over 80 of the common actions, the workflows, that have been fully automated. There is actually about 6 percent of what we do that we don’t think there’s an ROI in automating at this point.
As the technology becomes better over time, it may be possible that the cost to do that comes down, and that is the approach we sort of take across the board in many, many things that are being done.
The human is never going to be out of the loop. The human is always going to be there for the most, either, complex or the edge cases where the AI does not have confidence in its own answer.
AI Agent Adoption
The precursor to agents was assistants. If you look at assistants, I think at last count, we had over 20,000 deployments.
While there are some difficulties, there is also a lot of eagerness, excitement and enthusiasm about adopting these technologies.
If I look at proof of concepts, I think our teams collectively do … 4(000) or 5,000 each year in terms of kicking the tires. And about half of those, by the way, do make their way into some form of production.
I also think that we should understand that everybody–our partners, from SAP with their Joule technology, Salesforce with their Agentforce technologies, ServiceNow with their agentic workflows, Adobe with their agents on Experience, Workday with their agents (making) the HR process more effective, Oracle with some of their agents around both HR as well as payroll. I can go on.
Those are being extremely well adopted in all of these cases, including our own, across many, many clients, because they help. Now, these are not general-purpose agents. Almost all the examples I mentioned, they do a small but very precise set of tasks extremely well.
I do not believe that sensitive financial information, which is often in the context of both the C-suite and the board of directors, is going to be in the purview of something that is completely outside the domain of your CIO (chief information officer) and CISO (chief information security officer). I think we are far, far from those days.
However, could you be using third-party tools to augment your own learning, to maybe figure out what the competition is doing, to maybe figure out what is going on in the public domain.
Absolutely, I believe, and those may be tools that are third-party tools. That would be like asking, ‘Do these people today use Google search to do something?’ Of course, they do. But you normally don’t put your most sensitive information into the Google search bar.
Will the most senior people in the organization start using agentic AI and AI tools to make themselves more productive?
I believe that that answer is a resounding yes.
AI Bias and Hallucinations
Bias actually has been a topic we’ve been working on for many years. It’s one of the reasons that our R&D (research and development) teams worked on something called the AI Fairness toolkit, which allows you to check your models for bias.
It is also a huge part of the tests that we do as we build our own Granite models before we put them out. It has also become part of the guardrails that you can put around different models that you bring in, also, from other parties.
Bias is something that can be handled, and there are known techniques. (They) may not take it down to zero, but it can take it down to really minimal levels.
Hallucination is a little bit different. This is also something that is correlated, not perfectly, but it’s correlated with size of models.
As the amount of data you ingest and as the model sizes tend to go into the hundreds of billions of parameters, we can see that hallucination does increase.
So it now depends on what you’re trying to do. The technique that people use to take care of hallucination is that you use a verifier model.
I’m actually going to go a little bit tongue in cheek and tell you I don’t think hallucination will ever go to zero. At least when I talk to my children, they are often convinced that they said or did something or didn’t, when actually at least 15 percent of the time, my observation is they are dead wrong, but they are convinced that they are correct.
I’m going to take that as a proxy for most humans and sort of extrapolate it there and say, maybe AI is going to be better than that, but probably not at zero.
Smaller, Specialty Models A Value Unlock
AI is the source of productivity for this era, and that going back to the competitive advantage for a business means that that is the heart of your competitive advantage.
But not all AI is built the same. And not all AI is built for the enterprise. Why do I make that claim? Ninety-nine percent of all enterprise data has been untouched by AI.
If you need to go unlock the value from that 99 percent, you need to take an approach to AI that is tailored for the enterprise. And if you can unlock the value from that data, that’s a huge opportunity for the enterprise.
If you think about the massive general-purpose models, those are very useful. But they are not going to help you unlock the value from all of the data inside the enterprise. To win, you are going to need to build special-purpose models, much smaller, that are tailored for a particular use case and that can ingest the enterprise data and then work.
What about accuracy? Go look at the leaderboards. Smaller models are now more accurate than larger models.
What about intellectual property? That is where the open nature of some of the smaller models comes in. And if you think about specific domains, it’s a lot easier to build a smaller model that can go after that.
You can think about these models that are 3 (billion), 8 (billion), 13 (billion), 20 billion parameters, as opposed to 300 (billion), 500 (billion).
They’re incredibly accurate. They are much, much faster. They’re much more cost effective to run. And you can choose to run them where you want. You put all that together, and that is why, for example, we invest in our Granite family of models.
It’s not a substitute for the larger models. It’s an ‘and’ with the larger models. You can now tailor these models specifically to your enterprise needs. At 30 times (reduction in inference costs), you begin to get a huge advantage in cost and in what you can begin to do with them.
As technology comes down the cost curve, it opens up many, many more opportunities.
If you think about how expensive storage and compute were in the ‘90s, and you compare it to today, in that case, I think it’s 100 to 1,000x cheaper.
As that happens, you can throw it at a lot more problems, and that is the advantage.
IBM Addresses AI’s Engineering Challenges
There is no law of computer science that says that AI must remain expensive and must remain large. We can work on it. And I believe that that’s the engineering challenge that we are certainly taking on
Our focus is on enterprise AI, and that is where we are going to be spending our innovation dollars, and bring those to you in terms of the products.
The era of AI experimentation is over. Success is going to be defined by integration and business outcomes.
We believe that agents are really going to help redefine how people build applications. We talk about 1 billion new applications that are going to be built over the next four years. Agents are going to form at least a third, if not more, of these applications. So we believe that this is a powerful technology, and then you can let your agents run around autonomously and help make the enterprise a lot more productive.
A second issue that comes up is always integration.
How do you knit together the APIs you might be calling, the file transfers you might be doing, the B2B workflows that may be there? As you go across all of these knitting them together in one family with one platform of products is what (IBM subsidiary) WebMethods Hybrid integration does.
You’re going to get a lot more speed. You’re going to get a lack of complexity. And that is why we often have these third-party studies that then say, that is what results in the 176 percent ROI, but much faster time to value–30 (percent) to 70 percent depending on the simplicity or the complexity of the task.
And that is why we’re so excited that we now make knitting together the silos a lot easier.
IBM’s AI Chips
(The z17) is the first mainframe that is fully engineered for the AI era. It’s powered by the Telum II processor, which has advanced on-chip inferencing.
This can run 450 billion inference operations per day on a single system. It’s also boosted by the IBM Spyre AI accelerator.
We did one estimate–what happens if you combine the models onto Spyre and Telum and go after fraud in financial transactions.
That one use case could save $190 billion.
Over 250 use cases have been enabled. They go across loan risk mitigation, medical imaging, retail crime prevention and many, many others. Traditionally, people used to maybe sample 1 percent–maybe 10 percent in the best case–of the transactions you send off system, wait for a couple of seconds, get back an answer, or just completely do it offline to improve models and rules for later.
Now you can do them all in line all the time, and I think that those savings then flow back, whether you want to call them productivity or you want to call them revenue.
Quantum Era Coming
Our excitement around the growth opportunities, that is why we leaned in last week and announced our $150 billion investment in the United States across R&D and manufacturing.
We have computing paradigms that are evolving and becoming even more important.
As we move into the next frontier for quantum computing, it’s really important that we have the capabilities here, and that is why the R and D and the manufacturing investments (in the U.S.)
People ask, what’s the difference in AI and quantum? AI learns from data. So by definition … you’re learning from the past.
Quantum is trying to predict the future. It is working literally, in a quantum mechanical way, deep down, it’s trying to predict how molecules interact. It’s trying to predict the properties of new molecules, new materials. It’s trying to help you optimize financial risk. That can come in terms of thinking about what a portfolio might look like. It can come up in thinking about what pricing for complex financial instruments might look like.
It is going to begin to offer up many new opportunities that are beyond what traditional computing can offer. There’s a debate on timing. Some people in the industry go out and say, ‘Oh, it’s 20 years from now.’
We have built 75 actual quantum computers, not simulators, not just software. You can access 13 of them today on the cloud from IBM.
I would tell you that it’s not 20 years, but it’s in the four-, maybe five-year time frame that you will begin to see incredible advantages from these computers. We have over 250 industry and academic partners now working with us.
They’re excited by what these systems can already do, not just into the future. People are already looking to simulate materials that could go into EV (electric vehicle) batteries. They’re worrying about lubricants that can help in oil and gas production. They’re worrying about materials that can help with corrosion. They’re worrying about drugs (that) could help on brain cancer.
Our financial customers are already working on risk algorithms. So this is to tell you that quantum is no longer science fiction. It’s now in the realm of engineering.
We are building these full-stack quantum systems. We launched the Quantum System II earlier this year. It’s a modular quantum computer, modular meaning it’s got building blocks, and you put those building blocks together.
Now we are focused on error correction. And we know how to do it. It’s going to take us a couple of years to go deliver. And we bring the same engineering rigor to quantum that we bring to the mainframe.
We should stand up a quantum computer and it runs for a full year without breaking down, without needing a room full of experts to kind of re-tune it after every application. It can stay and maintain itself. People can access it with just an API. That’s what I mean by the engineering rigor of making it a computer, not just a science experiment.
AI Isn’t Incremental Technology
This moment isn’t just a little bit of an incremental technology. It’s not just a bit more AI, a bit more hybrid cloud.
If you leverage these technologies, you can fundamentally transform your business. You can help to scale your business in a way that was not ever possible.
The rules for business are changing. Of course, we must move fast. But more importantly, we must work smarter.
You can’t just cut costs using technology. You have to leverage technology to grow value, AKA grow revenue in the company. So technology has to be a value unlock.
Our consulting colleagues talk a lot about how digital workers will augment human workers.
Technology should be used not just to make the business grow faster. Don’t automate your workflow and your process as is. Reimagine how the workflow and the process should work by leveraging all of these capabilities that we now have on AI.
(We at IBM had) to reimagine how procurement got done by taking out a lot of the human steps that are there when you have only a human way of doing procurement.
