Dagster orchestrates dbt alongside other technologies, so you can combine dbt with Spark, Python, etc. in a single workflow. Dagster's software-defined asset abstractions make it simple to define data assets that depend on specific dbt models, or to define the computation required to compute the sources that your dbt models depend on. You could, for example:
Dagster has built-in support for loading dbt models, seeds, and snapshots as software-defined assets, enabling you to:
Dagster’s software-defined assets (SDAs) bear several similarities to dbt models. A software-defined asset contains an asset key, a set of upstream asset keys, and an operation that is responsible for computing the asset from its upstream dependencies. Models defined in a dbt project are similar to Dagster SDAs in that:
sourcecalls within the model's definition.
These similarities make it natural to interact with dbt models as SDAs. Let’s take a look at a dbt model and an SDA, in code:
Here's what's happening in this example:
raw_ordersis provided as an argument to the asset, defining it as a dependency
To learn how to load dbt models into Dagster as assets, check out the tutorial below or the quick version in the dagster-dbt reference.
In this tutorial, we'll walk you through integrating dbt with Dagster using dbt's example jaffle shop project, the dagster-dbt library, and a data warehouse, such as DuckDB.
By the end of the tutorial, you'll have a working dbt and Dagster integration and a handful of materialized Dagster assets, including a plotly chart powered by data computed from your dbt models.