Examples
This section contains examples of how to use Dagster. All examples can be found on Github.
Name | Description | |
---|---|---|
Airflow Ingest | Create a pipeline from an Airflow DAG | Open |
Airline Demo | A comprehensive demo to showcase Dagster and Dagit capabilities | Open |
Asset Materialization | Record that a solid materialized an asset | Open |
PySpark | Run PySpark code in solids | Open |
Conditional Execution | Conditionally execute part of a pipeline | Open |
Config Mapping | Simplify solid config schemas with config mapping. | Open |
dbt Integration | Use dagster-dbt to integrate dbt projects with Dagster. | Open |
Dependency DSL | Create a custom DSL for defining pipelines | Open |
Docker Deployment | Example Docker Deployment | Open |
EMR PySpark | Run a PySpark solids in EMR | Open |
Fan-In Dependency | Create a pipeline with a fan-in dependency | Open |
Great Expectations integration | Use Great Expectations to check data quality | Open |
Hooks | Create a pipeline with hooks | Open |
Kubernetes Deployment | Deploy Dagster on Kubernetes | Open |
Lakehouse (Experimental) | Define a computation graph in terms of the tables it produces. | Open |
Lakehouse with Pandas and Pyspark (Experimental) | Use the Lakehouse API with multiple compute types. | Open |
Multi-Location Workspace | Setup a Dagster workspace with multiple locations. | Open |
No-Data Dependencies | Modeling dependencies without inputs and outputs | Open |
Pipeline Tags | Tag a pipeline | Open |
Pipeline Unit Test | Unit-test a pipeline | Open |
Open |