FLUID

The shared data product language for the agentic data fabric.

FLUID is an open, declarative standard for defining Data Products. It replaces brittle pipelines with a trustworthy, governable, and scalable protocol — making your entire data ecosystem ready for the agentic era.

• Federated — domain teams own their data products, distributed by design.
• Labelled — rich metadata enables autonomous agent discovery.
• Universal — one standard across all systems and platforms.
• Interchangeable — composable components that work together.
• Declarative — specify what you want, not how to build it.

What is a Data Product?

It's more than just a table in a database. Today, most data is exhaust — a passive byproduct of operational systems: siloed and hidden, inconsistent and untrusted, with no clear owner. The result is high costs, slow decisions, and compliance risk.

A Data Product is a trusted, reusable asset, intentionally designed and managed for consumption — a finished good, not a leftover.

• Discoverable — published in a catalog with machine-readable metadata.
• Trustworthy — backed by a contract for schema, quality, and freshness SLOs.
• Secure & Governed — explicit access and privacy policies, enforced automatically.
• Owned — a clear domain owner accountable across its lifecycle.

What It Means for the Business

You don't need to be a data engineer to get value from data. A Data Product is simply a trustworthy, reusable asset designed to solve a business problem.

• The Data Marketplace — an "App Store for Data" where every certified product is browsable and searchable.
• Discover data you can trust — plain-language search, clear ownership, certified-only listings.
• Quality & governance at a glance — a "nutrition label" showing freshness, quality score, privacy class, and lineage.
• Access like a checkout — request access, state your purpose, get routed to the owner for fast approval.
• Your role — you define value, give feedback, and can own a product as a domain expert.

A New Way of Thinking

A truly modern data architecture is fluid — built on clear principles that enable the speed, trust, and scale the agentic era demands.

• 1. Data as a Product — versioned, owned assets with quality guarantees, not chaotic pipelines.
• 2. Declarative, Not Imperative — define the desired end state; let tools find the best implementation.
• 3. Contracts as Code — schema, quality, and access scopes live in version control; governance becomes automated and proactive.
• 4. Federated Ownership — decentralize to the domain teams who know the data best — a true Data Mesh.
• 5. Compliant Ecosystem — delegate execution to the tools you already use; stay open and composable.
• 6. Adaptive & Context-Aware — dynamic access policies that respond to an agent's intent and risk profile.

Anatomy of a Data Product

FLUID gives a simple, declarative structure for turning chaotic pipelines into trustworthy, governable, AI-ready assets. A single contract is built from a handful of clear sections.

• Identity & Metadata ("the passport") — id, name, domain, owner, tags.
• consumes ("the ingredients list") — explicit upstream dependencies with version constraints, for perfect automated lineage.
• builds ("the recipe") — a multi-modal array of transformations: batch + streaming, SQL + ML, multi-stage.
• exposes ("the serving window") — the governed public interface: schema, quality rules, and bindings per port.
• qos / slo ("the promise") — guaranteed freshness, availability, and latency.
• accessPolicy ("the bouncer") — grants that compile to platform-native IAM automatically.

Declarative Data Quality

Define quality rules once in your FLUID contract. Deploy everywhere. Let cloud providers do the heavy lifting.

• Write once — declare rules in the contract, not scattered across notebooks and pipelines.
• Deploy everywhere — the same contract compiles to BigQuery, AWS Glue, Great Expectations, and custom validators.
• Fail fast — automated validation gates catch issues before they reach consumers.
• Rule types — freshness, completeness, uniqueness, range, pattern, anomaly, and custom SQL.
• Severity levels — critical rules block the pipeline; warnings alert without breaking it.

CI/CD & Governance

Data-as-Code: a single Git commit automates orchestration, quality, and governance across every FLUID-aware tool.

• Commit — a developer commits a fluid.yml contract defining dependencies, build logic, and policies.
• Trigger — a webhook notifies every FLUID-aware application simultaneously.
• Interpret — the orchestrator builds the pipeline, the quality engine generates tests, the catalog ingests metadata and lineage.
• Execute & publish — the engine runs and publishes the product to the Data Product Gateway.
• Govern & consume — the Gateway enforces access and privacy policies; an MCP agent discovers and securely consumes the product.

The Data Product IDE

An interactive workbench for authoring FLUID contracts — write the YAML on one side and see your data product come to life on the other.

• Live validation — real-time feedback against the FLUID schema as you type, with errors mapped to the offending line.
• Visual model — an interactive graph of entities and relationships (Hubs, Links, Satellites) rendered straight from the contract.
• Contract views — dedicated panels for overview, consumes, build, exposes, and quality.
• Worked examples — load Bronze (Oracle raw), Silver (dbt Data Vault), and Gold (BigQuery analytics mart) products.
• Import & export — upload YAML, download as YAML or JSON.

Structuring an Enterprise Data Product

The "gold standard" for a complex product: a multi-file layout that lets different teams own different parts of one contract.

• Data Modeler — owns exposes.yml and the schema_*.yml files.
• Data Engineer — owns build.yml and consumes.yml.
• Governance Lead — owns quality.yml and accessPolicy.yml.

A root fluid.yml composes the detailed implementation files (via $ref) that live in a .fluid/ directory — separating high-level identity from its parts while keeping a single source of truth.

The Data Product Litmus Test

A Data Product is a commitment to quality, usability, and value. Run your data asset through this checklist — is it just a table, or a true enterprise-ready product?

• Discoverable — a new employee can find it and grasp its purpose within minutes.
• Addressable — it has a permanent, unique, machine-readable identifier.
• Trustworthy — quality, freshness, and lineage are defined and guaranteed by an SLA.
• Self-describing — all metadata needed to use it lives at its address.
• Interoperable — exposed through standardized consumption ports.
• Secure — access is governed by an automated, auditable process.

Powering the Data Product Economy

A protocol is only as strong as its ecosystem. FLUID is backed by a suite of open tools that make building, discovering, and consuming data products effortless.

• VS Code Data Product Studio — author, validate, and visualize products in your IDE.
• FLUID CLI — scaffold, validate, test, and publish products to a marketplace.
• Open Data Marketplace — a Git-native, self-hosted catalog for discovery and access.
• Lineage Engine — parses contracts into an end-to-end, column-level lineage graph.
• Agentic Data Gateway — enforces context-aware access policies for AI agents.
• Execution Engine Providers — translate FLUID specs into dbt, Spark, and Airflow plans.

A Message from Your New Workforce

"Your legacy systems ask me to read a library of handwritten maps to navigate a modern city. I can attempt it, but I will be slow, I will make mistakes, and I cannot be trusted with critical missions. To succeed, I require a new relationship with data."

• Start with the mission — give me an objective, not a specific API to call.
• Discover & trust data products — let me ask the catalog rich questions and rely on the answers.
• Consume with precision — serve both an analytics API and a real-time stream from one product.
• Act with governed confidence — let me invoke governed Action products to close the loop, safely.

Building the Future, Together

The agentic era requires a shared, open standard for data. FLUID is a community-driven protocol, and its success depends on all of us.

• Enterprises — adopt FLUID to solve real-world data problems.
• Vendors — build the next generation of FLUID-aware tools.
• Developers — shape the spec and grow the ecosystem.

Help us build the missing protocol for the agentic era. Start with the guide, explore the concepts, or dive into the schema (current version 0.7.4).