Red Hat AI Inference Server, validated models and integration of Llama Stack and Model Context Protocol help users deliver higher-performing, more consistent AI applications and agents
BOSTON – RED HAT SUMMIT, May 20, 2025–(BUSINESS WIRE)–Red Hat, the world’s leading provider of open source solutions, today continues to deliver customer choice in enterprise AI with the introduction of Red Hat AI Inference Server, Red Hat AI third-party validated models and the integration of Llama Stack and Model Context Protocol (MCP) APIs, along with significant updates across the Red Hat AI portfolio. With these developments, Red Hat intends to further advance the capabilities organizations need to accelerate AI adoption while providing greater customer choice and confidence in generative AI (gen AI) production deployments across the hybrid cloud.
According to Forrester, open source software will be the spark for accelerating enterprise AI efforts.1 As the AI landscape grows more complex and dynamic, Red Hat AI Inference Server and third party validated models provide efficient model inference and a tested collection of AI models optimized for performance on the Red Hat AI platform. Coupled with the integration of new APIs for gen AI agent development, including Llama Stack and MCP, Red Hat is working to tackle deployment complexity, empowering IT leaders, data scientists and developers to accelerate AI initiatives with greater control and efficiency.
Efficient inference across the hybrid cloud with Red Hat AI Inference Server
The Red Hat AI portfolio now includes the new Red Hat AI Inference Server, providing faster, more consistent and cost-effective inference at scale across hybrid cloud environments. This key addition is integrated into the latest releases of Red Hat OpenShift AI and Red Hat Enterprise Linux AI, and is also available as a standalone offering, enabling organizations to deploy intelligent applications with greater efficiency, flexibility and performance.
Tested and optimized models with Red Hat AI third party validated models
Red Hat AI third party validated models, available on Hugging Face, make it easier for enterprises to find the right models for their specific needs. Red Hat AI offers a collection of validated models, as well as deployment guidance to enhance customer confidence in model performance and outcome reproducibility. Select models are also optimized by Red Hat, leveraging model compression techniques to reduce size and increase inference speed, helping to minimize resource consumption and operating costs. Additionally, the ongoing model validation process helps Red Hat AI customers continue to stay at the forefront of optimized gen AI innovation.
Standardized APIs for AI application and agent development with Llama Stack and MCP
Red Hat AI is integrating Llama Stack, initially developed by Meta, along with Anthropic’s MCP, to provide users with standardized APIs for building and deploying AI applications and agents. Currently available in developer preview in Red Hat AI, Llama Stack provides a unified API to access inference with vLLM, retrieval-augmented generation (RAG), model evaluation, guardrails and agents, across any gen AI model. MCP enables models to integrate with external tools by providing a standardized interface for connecting APIs, plugins and data sources in agent workflows.
The latest release of Red Hat OpenShift AI (v2.20) delivers additional enhancements for building, training, deploying and monitoring both gen AI and predictive AI models at scale. These include:
Optimized model catalog (technology preview) provides easy access to validated Red Hat and third party models, enables the deployment of these models on Red Hat OpenShift AI clusters through the web console interface and manages the lifecycle of those models leveraging Red Hat OpenShift AI’s integrated registry.
Distributed training through the Kubeflow Training Operator enables the scheduling and execution of InstructLab model tuning and other PyTorch-based training and tuning workloads, distributed across multiple Red Hat OpenShift nodes and GPUs and includes distributed RDMA networking–acceleration and optimized GPU utilization to reduce costs.
Feature store (technology preview), based on the upstream Kubeflow Feast project, provides a centralized repository for managing and serving data for both model training and inference, streamlining data workflows to improve model accuracy and reusability.
Red Hat Enterprise Linux AI 1.5 brings new updates to Red Hat’s foundation model platform for developing, testing and running large language models (LLMs). Key features in version 1.5 include:
Google Cloud Marketplace availability, expanding the customer choice for running Red Hat Enterprise Linux AI in public cloud environments–along with AWS and Azure–to help simplify the deployment and management of AI workloads on Google Cloud.
Enhanced multi-language capabilities for Spanish, German, French and Italian via InstructLab, allowing for model customization using native scripts and unlocking new possibilities for multilingual AI applications. Users can also bring their own teacher models for greater control over model customization and testing for specific use cases and languages, with future support planned for Japanese, Hindi and Korean.
The Red Hat AI InstructLab on IBM Cloud service is also now generally available. This new cloud service further streamlines the model customization process, improving scalability and user experience, empowering enterprises to make use of their unique data with greater ease and control.
Red Hat’s vision: Any model, any accelerator, any cloud.
The future of AI must be defined by limitless opportunity, not constrained by infrastructure silos. Red Hat sees a horizon where organizations can deploy any model, on any accelerator, across any cloud, delivering an exceptional, more consistent user experience without exorbitant costs. To unlock the true potential of gen AI investments, enterprises require a universal inference platform–a standard for more seamless, high-performance AI innovation, both today and in the years to come.
Red Hat Summit
Join the Red Hat Summit keynotes to hear the latest from Red Hat executives, customers and partners:
Supporting Quotes
Joe Fernandes, vice president and general manager, AI Business Unit, Red Hat
“Faster, more efficient inference is emerging as the newest decision point for gen AI innovation. Red Hat AI, with enhanced inference capabilities through Red Hat AI Inference Server and a new collection of validated third-party models, helps equip organizations to deploy intelligent applications where they need to, how they need to and with the components that best meet their unique needs.”
Michele Rosen, research manager, IDC
“Organizations are moving beyond initial AI explorations and are focused on practical deployments. The key to their continued success lies in the ability to be adaptable with their AI strategies to fit various environments and needs. The future of AI not only demands powerful models, but models that can be deployed with ability and cost-effectiveness. Enterprises seeking to scale their AI initiatives and deliver business value will find this flexibility absolutely essential.”
1Source: Navigate The Open-Source AI Ecosystem In The Cloud, Forrester Research, Inc., February 2025
Additional Resources
Connect with Red Hat
About Red Hat
Red Hat is the open hybrid cloud technology leader, delivering a trusted, consistent and comprehensive foundation for transformative IT innovation and AI applications. Its portfolio of cloud, developer, AI, Linux, automation and application platform technologies enables any application, anywhere—from the datacenter to the edge. As the world’s leading provider of enterprise open source software solutions, Red Hat invests in open ecosystems and communities to solve tomorrow’s IT challenges. Collaborating with partners and customers, Red Hat helps them build, connect, automate, secure and manage their IT environments, supported by consulting services and award-winning training and certification offerings.
Forward-Looking Statements
Except for the historical information and discussions contained herein, statements contained in this press release may constitute forward-looking statements within the meaning of the Private Securities Litigation Reform Act of 1995. Forward-looking statements are based on the company’s current assumptions regarding future business and financial performance. These statements involve a number of risks, uncertainties and other factors that could cause actual results to differ materially. Any forward-looking statement in this press release speaks only as of the date on which it is made. Except as required by law, the company assumes no obligation to update or revise any forward-looking statements.
Red Hat, Red Hat Enterprise Linux, the Red Hat logo and OpenShift are trademarks or registered trademarks of Red Hat, Inc. or its subsidiaries in the U.S. and other countries. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.
View source version on businesswire.com: https://www.businesswire.com/news/home/20250520848054/en/
Contacts
Media Contact: Marian Pierson Red Hat 770-856-6099 mpierson@redhat.com
