Skip to main content

Define virtual assets

When Snowflake manages your transformations — through Dynamic Tables, views, or other objects — Dagster should know they exist without trying to run them. The question is which of Dagster's two non-executable asset types is the right fit.

Both external assets and virtual assets appear in the lineage graph without being executed by Dagster. The difference is how automation treats them:

  • External assets are opaque to automation conditions. Dagster sees them in the graph, but evaluating whether a change should trigger a downstream run requires an explicit event.
  • Virtual assets are transparent. Automation conditions look through them to their real upstream sources, so downstream assets re-run automatically when source data changes.

Snowflake Dynamic Tables are a natural fit for virtual assets. They refresh automatically based on a target lag — Snowflake handles the computation, and Dagster should never run them. Declaring them with is_virtual=True models this correctly while keeping full lineage intact.

External source tables

The raw source tables — loaded by an external ETL pipeline — are represented as plain AssetSpec objects. Dagster tracks them for lineage but never executes them:

project_snowflake_dynamic_tables/defs/assets/sources.py
raw_orders = dg.AssetSpec(
key="raw_orders",
group_name="sources",
description="Raw order records loaded by the nightly ETL pipeline.",
metadata={
"snowflake_table": "ECOMMERCE.RAW.RAW_ORDERS",
"owner": "data-engineering-team",
},
kinds={"snowflake"},
)

raw_customers = dg.AssetSpec(
key="raw_customers",
group_name="sources",
description="Raw customer records loaded by the nightly ETL pipeline.",
metadata={
"snowflake_table": "ECOMMERCE.RAW.RAW_CUSTOMERS",
"owner": "data-engineering-team",
},
kinds={"snowflake"},
)

Snowflake Dynamic Tables as virtual assets

The Dynamic Tables are declared with is_virtual=True. They appear in the lineage graph and can be observed, but Dagster will never place them in a run:

project_snowflake_dynamic_tables/defs/assets/dynamic_tables.py
customer_lifetime_value = dg.AssetSpec(
key="customer_lifetime_value",
group_name="dynamic_tables",
description=(
"Snowflake Dynamic Table: per-customer lifetime value computed from orders.\n\n"
"TARGET_LAG = '1 minute', REFRESH_MODE = INCREMENTAL.\n"
"Snowflake refreshes this automatically — Dagster provides lineage and monitoring."
),
deps=["raw_orders", "raw_customers"],
is_virtual=True,
metadata={
"snowflake_object_type": "DYNAMIC TABLE",
"target_lag": "1 minute",
"refresh_mode": "INCREMENTAL",
"snowflake_table": "ECOMMERCE.ANALYTICS.CUSTOMER_LIFETIME_VALUE",
},
kinds={"snowflake"},
)

daily_revenue_rollup = dg.AssetSpec(
key="daily_revenue_rollup",
group_name="dynamic_tables",
description=(
"Snowflake Dynamic Table: daily revenue aggregated from raw orders.\n\n"
"TARGET_LAG = '1 hour', REFRESH_MODE = FULL.\n"
"Snowflake refreshes this automatically — Dagster provides lineage and monitoring."
),
deps=["raw_orders"],
is_virtual=True,
metadata={
"snowflake_object_type": "DYNAMIC TABLE",
"target_lag": "1 hour",
"refresh_mode": "FULL",
"snowflake_table": "ECOMMERCE.ANALYTICS.DAILY_REVENUE_ROLLUP",
},
kinds={"snowflake"},
)

Key properties on each virtual spec:

  • is_virtual=True — marks the asset as unexecutable
  • deps=[...] — preserves lineage so automation can traverse from source tables through the Dynamic Tables to downstream assets
  • metadata — stores Snowflake-specific properties (target lag, refresh mode, table name) visible in the asset catalog

Next steps

Continue this example by automating downstream assets.