Vast Data’s SyncEngine helps AI agents to tap unstructured data from every source

Data storage company Vast Data Inc., which is in the process of transforming itself into an “operating system” for artificial intelligence, today announced a new capability called Vast SyncEngine.

The company says it acts as a “universal data router,” combining a highly performant onboarding system for unstructured data with a global catalog for building AI data pipelines. Available at no additional cost to existing customers, Vast SyncEngine is designed to simplify headaches around discovering and mobilizing distributed, unstructured datasets and software-as-a-service tools, so these data sources can quickly be plugged into their AI applications.

Vast SyncEngine is the latest addition to the Vast AI OS platform.It combines core distributed computing services, including storage, compute, messaging and reasoning into a unified data layer that spans cloud, on-premises and edge environments to power AI applications and agents.

Vast AI OS is based on the company’s original “Disaggregated and Shared-Everything” architecture, which separates storage media from the processors that manage it. That enables the platform to store nearly unlimited amounts of data that can be accessed independently of the customers computing resources.

It has since evolved to encompass every kind of storage, including file, object, block, table and streaming data. Capabilities include vector search and serverless functions, creating a platform that’s capable of powering entire agentic AI workflows.

Now it’s looking to build on that platform with Vast SyncEngine, which was built to solve one of the key bottlenecks in AI – namely, data pipelines. The company explained that data today is scattered across dozens of outdated file and object-based systems, siloed in SaaS applications and invisible to AI pipelines. This fragmented data landscape has created a “last mile” problem, where valuable data remains inaccessible to AI models and applications. Teams try to get around this by stitching together various third-party tools to find, prepare and move data, with mixed results, Vast says.

Cataloging data for AI pipelines

Vast SyncEngine is positioned as the solution to these challenges. It works by collapsing data cataloging, migration and transformation into a single, no-cost capability within the Vast AI OS platform, enabling simpler integration and faster time to insights with lower costs.

The capability aligns with a broader trend that has seen broader trend of storage vendors building data management capabilities into their solutions. It presents a compelling offering for any business looking for a hyperconverged data platform, said Steve McDowell of NAND Research Inc.

“VAST was one of the first storage vendors to deeply integrate a database and data management tools directly into its storage platform, providing a hyper-converged like experience between data management and performance storage,” the analyst said. “SyncEngine takes things further, extending Vast’s data management capabilities beyond what’s stored in its own systems, indexing and cataloging data across storage platforms, and even into SaaS platforms. It turns Vast into an enterprise-wide and storage system-agnostic data plane.”

Constellation Research Inc. analyst Michael Ni said SyncEngine aims to tackle one of the most overlooked barriers to enterprise AI, and benefits immensely from its built-in vector database and event streaming capabilities. “It collapses the problematic toolchain sprawl within a single, high-performance platform,” he said. “So it’s not just speeding up data pipelines, it’s pushing to reshape the very economics of the AI stack.”

The company makes some big claims about the quality of its new capability. Because it has been purpose-built for massive file and object datasets and SaaS platforms, it can ingest data from the source and transform it into insights with “record-breaking speeds,” the company said.

It enables teams to catalog and search across trillions of files and objects leveraging the Vast DataBase, with unlimited ingest throughput that’s bound only by the performance of the source and target systems. Moreover, its massively parallel architecture supports rapid scaling, simply by adding more nodes.

By using Vast SyncEngine, companies can quickly build real-time, searchable data catalogs that span everything from traditional file and object systems to enterprise applications such as Google Drive, Salesforce, Microsoft Office and Sharepoint, and more besides. It supports deep metadata indexing to make data instantly discoverable.

All of this unstructured information, the company says, can be moved without needing to create custom scripts and data transformations, fed into Vast’s InsightEngine and DataEngine platforms, which optimize it for AI applications and agentic workloads. As an added benefit, companies should see much lower costs, as Vast SyncEngine is essentially a replacement for existing data transformation and migration tools.

Ni said the new capability means Vast Data’s AI OS is one of the few platforms that’s able to incorporate everything into a single AI data stack. Few of its rivals can do this, he said.

For instance, though Snowflake Inc. and Databricks Inc. emphasize governance and intelligence atop data, they decouple storage from compute. Moreover, while data catalog vendors such as Collibra Inc. and Informatica LLC excel in terms of metadata, they lack the integrated data mobility and AI prep tools found in Vast’s platform.

“Vast’s consolidation delivers high-performance, low-latency data pipelines for real-time and agentic AI, but it challenges buyers to embrace a single vendor spanning so much ground,” Ni said. “But Vast’s four- to five-times annual revenue growth compares well with the overall market and validates that multiple architectural paths will co-exist, showing there is a clear market appetite for its full-stack approach.”

McDowell agreed, saying there’s lots of goodness in Vast SyncEngine for any enterprise that’s willing to commit to its full platform. But he warned that doing so comes at the cost of flexibility, such as being able to swap in and out tools as the AI stack evolves. “But the challenge with everything Vast is doing is timing, as it’s still very early days for enterprise AI,” he said.

In addition, McDowell said, it’s not easy to judge Vast’s claims that it lowers the cost of AI data pipelines, because SyncEngine is bundled in with its broader offering and not sold separately. “For existing Vast customers, it will be less expensive than buying third-party software, but if you’re not a Vast customer, the cost equation becomes trickier.”

Vast Data co-founder Jeff Denworth said the key consideration here is that winners of the AI race will be those companies that can harness all of their data, rather than just the information that’s easy to access. He explained that data sprawl is like a “silent killer” of AI strategies, blunting the effectiveness of AI applications and agents. In other words, they need a solution like SyncEngine in order to succeed in AI.

“Legacy IT created silos, and we’re tearing them down,” Denworth said. “Whether your data is buried in on-prem systems or hidden in SaaS apps, SyncEngine makes it all accessible, visible and valuable. We’re giving customers a direct path from where their data lives today to where AI transformation begins.”

Image: SiliconANGLE/Dreamina AI