(Source: Shutterstock)
IBM Monday kicked off its annual Think conference being held in Boston this week. No surprise, generative AI and tools for enabling agentic AI will dominate discussion at the event expected to attract 5000 attendees. IBM, like most everyone, has high expectations.
“Over the next few years, we expect there will be over a billion new applications constructed using generative AI as they make their way in,” said Arvind Krishna, IBM CEO, at a mid-day virtual media/analyst meeting. He was joined by Rob Thomas, chief commercial officer, and Ritika Gunnar, GM for data and AI.
Arvind Krishna, IBM CEO
Citing an IBM CEO Study, Krishna said “our clients are expecting to double or even increase the investments on AI beyond that, however, they are finding that only about 25% of the time are they getting the ROI that they expect, driven by a lot of factors, access to enterprise data, the siloed nature of the different applications, together with the fragmentation that is happening in the infrastructure.”
The relative high failure rate and poor ROI of AI projects has become a growing problem for AI advocates. At Think, IBM is touting its watsonx enterprise AI platform and rolling out new features, many designed to tame the AI’s deployment, execution, and ROI issues. Krishna singled out three:
watsonx Orchestrate – A tool, Krishna, said “lets you build your own agent for enterprise use in five minutes or less.” It has ~80 partner agents out of the box integrated with IBM-built agents for a total of 150.
webMethods Hybrid Integration – A new solution IBM says “that replaces rigid workflows with intelligent and agent-driven automation. It will help users manage the sprawl of integrations across apps, APIs, B2B partners, events, gateways, and file transfers in hybrid cloud environments.”
watsonx.data – This improved tool allows you “bring together an open data lakehouse with data fabric capabilities – like data lineage tracking and governance – to help clients unify, govern, and activate data across silos, formats, and clouds,” says IBM. “[It] allows enterprises to connect their AI apps and agents with their unstructured data, which can lead to 40% more accurate AI than conventional RAG.”
(For a full rundown on the tools being highlighted at Think read the IBM press release here.)
The fourth major announcement from Krishna was introduction of a new mainframe: IBM LinuxONE 5, which IBM says is “the most secure and performant Linux platform to date – with the ability to process up to 450 billion AI inference operations per day.”
LinuxONE 5 features highlighted by IBM include:
“State-of-the-art IBM AI accelerators, including IBM’s Telum II on-chip AI processor and the IBM Spyre Accelerator card (available 4Q 2025 via PCIe card), to enable generative and high-volume AI applications such as transactional workloads.
“Advanced security offerings with confidential containers to help clients protect their data and new integrations with IBM’s pioneering quantum-safe encryption technology to address quantum-enabled cybersecurity attacks.
“Significant reductions in costs and power consumption – moving cloud-native, containerized workloads from a compared x86 solution to an IBM LinuxONE 5 running the same software products can save up to 44% on the total cost of ownership over 5 years.”
IBM clearly has a broad portfolio of enterprise-centric AI offerings. Its focus on the hybrid cloud and AI together, said Krishna is key to taming what he called the infrastructure fragmentation going on. In Q&A, he, Thomas and Gunnar fielded a variety of questions;
Has IBM seen any slowdown in AI spending?
Krishna said, “The short answer is, No, we are actually seeing people double down on their AI investments. As people are looking for productivity, they’re looking for cost savings, but they’re also looking to scale the revenue of their own companies. AI is one of the unique technologies that can hit at the intersection of all three. The CIO surveys done by our IBV group show that everybody is doubling down on AI investments, but they’re now looking for that return on AI. So the only change over the last 12 months is that people are stopping experimentation and focusing very much on where is the value to the business.
Thomas chipped in. “On ROI, I’d say this is the year that value creation has become the focus. So it’s not a disillusionment with AI, as much as it is clients demanding value creation. I think we got through experimentation things like RAG, and to be blunt, some of the results were uncertain or mixed. I think the minute that it turned to automation, being able to automate your core infrastructure with things like TerraForm or vault as an example, automating your financials, moving into assistance and then agents. That’s where we saw, I’d say, the tipping point for value creation.”
Another questioner noted there are often more exceptions than rules in a business process. “How does IBM you create the level of dynamic reasoning required for AI agents to manage a workflow end to end?”
Krishna provided an example from IBM.
“We have a lot of experience in this. I’ll give it a simple example, and then we’ll broaden it from there. We looked at our HR processes, and we began two years ago with the top five most common queries that were being done. We automated those [and] every couple of weeks after we kept adding until at this point, we have over 80 of the common action. So those are workflows that have been fully automated.
“There are actually about 6% of what we do that we don’t think there’s an ROI in automating at this point. As the technology becomes better over time, it may be possible that the cost to do that comes down, and that is the approach we sort of take across the board. In many, many things that are being done, the human is never going to be out of the loop. The human is always going to be there for the most either complex or the edge cases where the AI does not have a confidence in its own answer,” he said.
Another question posed was, “While there is a lot of enthusiasm, companies are facing challenges when it comes to adoption. How many clients have implemented so far? How many POCs, and POCs that are translating to full time projects?”
That’s a good milestone question.
Krishna said, “[To] give a little bit of context on this question, I think the precursor to agents was assistance. If you look at assistance, I think at last count, we had over 20,000 deployments. Sometimes there were three or four at one client, but over 20,000 deployments. So I think it’s actually going back to comment Rob made earlier. While there are some difficulties, there is also a lot of eagerness, excitement and enthusiasm about adopting these technologies. So 20,000 was our number. If I look at proof of concepts, I think out teams do over 4000 or 5000 each year, I think in terms of kicking the tires. And about half of those, by the way, do make their way into some form of production. I wouldn’t say all of them, but about half of them do make their way forward.
“I also think that we should understand that everybody our partners, from SAP with Joule technology, Salesforce with their Agent Force technologies, ServiceNow with their agents on workflows, Adobe, with their agents on on experience, Work day with their agents on how to make the HR process more effective. Oracle, with some of the agents around both HR as well as payroll, can go on. Those are being extremely well adopted in all of these cases, including our own across, across many, many clients, because they help. Now, these are not general purpose agents. Almost all the examples I mentioned, they do a small but very precise set of tasks extremely well.”
Gunnar added, “We have an IBV study that we just released today, or will be released tomorrow, that the majority of enterprises are doing proof of concepts or are leveraging AI agent technology, whether it be from their single application vendors, like in the examples that Arvind mentioned, or whether it be agents that they build on their own. So we know that a majority of organizations, 2025 is the year that we see a lot of organizations experimenting with AI agents. Now we also see some of those in production. And when we see those in production, they’re usually simple, task based execution AI agents.
“We’re seeing a lot of now research and technology maturing in terms of the underlying reasoning capabilities of these models and in terms of what it means to have a full stack observability and traceability. We’re now starting to see, even in the AI agent technologies, more complex use cases such as automations, collaboration, orchestration. And so you can see that as a spectrum where many of them are experimenting simple task based AI agents are being implemented in production. And as the technology continues to mature, we’re going to see even the more complex cases of orchestration, collaboration and automation continue to mature into production, whereby in the next couple years, we believe that over 50% of organizations will have these AI agents embedded in their essential systems.”
Thomas added, “To give you a specific example, about a year and a half ago we partnered with Dunn and Bradstreet to build an assistant called Ask procurement. This was using procurement data and company data and shipping data from Dun and Bradstreet. We constructed this application which leverages orchestrate and Watson x and is able gives a procurement agent the ability to ask questions and to figure out the best place to source something; we’ve now started working on, how do we make that agentic? Because today it’s really just an AI, Q and A, and making it agentic would be actually linking it to if I’m choosing a supplier, what can I get in terms of actual delivery times, as I look at shipping schedules, shipping lanes, whatever may be. So I would say the technology is there. This is about the change management and then applying it to these specific problems. But for most companies, I think often starting with an assistant and then moving towards agentic workflows is sometimes the fastest, lowest risk way to do it.”
Link to IBM press announcement, https://www.hpcwire.com/off-the-wire/ibm-accelerates-enterprise-gen-ai-revolution-with-hybrid-capabilities.
Link to THINK, https://www.ibm.com/events/think.
This article first appeared on HPCwire.