Snowflake-led coalition targets data fragmentation with vendor-neutral semantic standard

Snowflake Inc. and a group of partners today launched the Open Semantic Interchange initiative, an open-source effort to standardize how organizations define and share business semantics across data and analytics platforms.

The initiative aims to create a vendor-neutral specification for business metrics, dimensions and metadata that can be used consistently across business intelligence tools, data catalogs and artificial intelligence applications. The 17 initial participants include Salesforce, BlackRock Inc. and dbt Labs Inc.

Data semantics describe the meaning and context of data relative to business concepts, processes and rules. For example, consistent semantics ensure that different systems and teams interpret terms like “customer,” “revenue” and “operating margin” the same way.

Many organizations manage business metrics across multiple BI and analytics systems, each with its own set of semantic definitions. That leads to duplication of effort and inconsistent definitions. The initiative is an effort to resolve those differences.

“A strong request from customers has been to make semantic models interoperable,” said Josh Klahr, director of analytics product management at Snowflake. “They want to make it easy for a model in one place to show up in another location, avoid the risk of vendor lock-in and think about interoperability for the long term.”

Fragmented semantics

The effort will initially focus on analytics and BI use cases with an additional emphasis on enabling AI interfaces to work with consistent, verified data definitions.

Work on the specification has just begun, with a working group defining the first version of a draft standard that’s likely to be published within the next few months, Klahr said. Numerous technology- and industry-specific semantic standards already exist, with several administered by the World Wide Web Consortium.

The standard is expected to build on existing approaches where possible, using widely adopted formats such as YAML to define metrics, hierarchies and relationships. “YAML will likely end up being the definition language,” Klahr said.

Snowflake said the initiative will probably be donated to either the Apache Software Foundation or the Linux Foundation. “It’s early to say which because we want the working group to agree on what the licensing model will be,” Klahr said, “but the plan is to be an open-source project.”

The company expects the list of participating vendors to grow rapidly. “There’s lots of interest,” Klahr said. “This is something that makes sense to everybody.”

Snowflake rival Databricks Inc. isn’t a founding member, but “I would love for additional partners like Databricks to be part of the effort as well,” he said.

Photo: Unsplash