Snowflake platform enhancements focus on performance, governance and interoperability

Snowflake Inc. today is introducing a broad slate of platform updates aimed at improving performance, reducing operational overhead and expanding its range of interoperable tools.

The flagship announcement is that the latest release of the company’s core data warehouse, called Standard Warehouse – Generation 2, is now generally available. Comprising both hardware and software upgrades to Snowflake’s core compute engine, the new release promises more than double the analytics performance without requiring changes to existing workloads or queries and scales computing resources in line with growing data volumes and increasingly complex workloads, Snowflake said.

“For some specific workloads, anything that is write-heavy or updates-heavy it can be two to four times faster,” said Christian Kleinerman (pictured), executive vice president of product.

Complementing the Gen2 rollout is the debut of Snowflake Adaptive Compute, a new compute service in private preview that Snowflake said automates much of the warehouse management process. The new approach, dubbed “adaptive warehouses,” lets users bypass manual configuration tasks like sizing warehouses, setting concurrency levels or dealing with multicluster configurations by dynamically adjusting resources behind the scenes to optimize for cost and performance.

Snowflake’s shift to adaptive infrastructure is consistent with broader cloud platform trends toward greater elasticity and automation. It also aligns with the company’s stated vision of turning data infrastructure into an invisible layer that requires little engineering attention.

AI-powered governance

Snowflake is also introducing several features aimed at making data more discoverable, secure and interoperable. The Snowflake Horizon Catalog has been enhanced with AI-powered data governance tools, including a new Copilot for Horizon Catalog that uses the Snowflake Cortex AI platform to answer security and governance questions via a chat interface.

Horizon Catalog also now supports catalog-linked databases that allow users to synchronize with Apache Iceberg-based metadata catalogs like Apache Polaris, Snowflake Open Catalog and Amazon Web Services Inc.’s Glue. That makes for more seamless governance across hybrid and multicloud environments, particularly when using Iceberg tables, Snowflake said.

In a nod to rising concerns about data resilience, Snowflake is announcing support for the point-in-time, immutable backups called snapshots that safeguard against data loss or ransomware attacks. Once created, snapshots can’t be altered or deleted, which has value in meeting regulatory cyber resilience standards.

Security is further bolstered with new Trust Center Extensions that allow customers to integrate third-party security scanners tailored to their compliance requirements. AI Observability Tools, now generally available, deliver real-time diagnostics and performance insights across an the entire Snowflake data environment.

Faster data ingestion

In a bid to make data integration easier, Snowflake is introducing Openflow, a managed, extensible service that the company said simplifies ingesting structured and unstructured data from virtually any source. Based on Apache NiFi, a widely used open-source tool for data flow automation, Openflow supports batch and streaming workloads, including Snowpipe Streaming, and includes hundreds of pre-built connectors to third-party services.

Openflow addresses what Snowflake said is the disproportionate time data engineering teams spend wrangling ingest pipelines. It’s intended to reduce that effort while maintaining governance and flexibility. It also moves Snowflake into the $15 billion data integration market, which has historically been served mainly by third-party ETL tools.

“AI has materially changed the ability for organizations to get value out of their unstructured data,” Kleinerman said. “Openflow takes data from SharePoint or Drive or Slack and makes it available for customers to combine with their data.”

Also for the data engineering community, dbt Projects can soon be built and run natively in Snowflake, offering direct integration with the dbt framework that many teams already use for SQL-based data transformation. It’s part of Snowflake Workspaces, a new file-based development environment that features AI-assisted coding, Git integration and side-by-side code comparison.

Snowflake is deepening its support for Apache Iceberg by allowing customers to manage Iceberg tables more efficiently, take advantage of semi-structured data types such as VARIANT and tune partitions and file sizes.

Finally, Snowflake is enhancing streaming capabilities in Snowpipe Streaming to permit ingestion rates of up to 10 gigabytes per second, with data available for querying within 10 seconds after ingest. The capability is particularly relevant for operational analytics, real-time personalization and other use cases where timeliness is critical.

Photo: SiliconANGLE