Integrations#

Dagster is flexible and allows incremental adoption. It provides add-on libraries to integrate with your existing tools and infrastructure.

Guides#

This section includes guides on how to use Dagster with other tools.

NameDescription
Dagster with dbtOrchestrate dbt from Dagster.
Dagster with Great ExpectationsRun data quality tests using Great Expectations in a Dagster pipeline.
Dagster with SparkDefine and execute Spark jobs in Dagster.
Dagster with PandasHow Dagster works with Pandas.
Dagster with Jupyter/PapermillHow to orchestrate Jupyter notebooks from Dagster.
Dagster with AirflowUse Dagster in an Airflow cluster, or transform Airflow DAGs into Dagster pipelines.

Libraries#

Here is a complete list of Dagster's integration libraries. See full documentation in API Reference.

IntegrationLibrary
Airflow dagster-airflow
AWS dagster-aws
Azure dagster-azure
Celery dagster-celery
Celery + Docker dagster-celery-docker
Dask dagster-dask
Databricks dagster-databricks
Datadog dagster-datadog
Docker dagster-docker
dbt dagster-dbt
Fivetran dagster-dbt
GCP dagster-gcp
Great Expectations dagster-ge
Github dagster-github
Kubernetes dagster-k8s
Microsoft Teams dagster-msteams
MLflow dagster-mlflow
MySQL dagster-mysql
PagerDuty dagster-pagerduty
Pandas dagster-pandas
Papermill dagstermill
Papertrail dagster-papertrail
PostgreSQL dagster-postgres
Prometheus dagster-prometheus
Pyspark dagster-pyspark
Shell dagster-shell
Slack dagster-slack
Snowflake dagster-snowflake
Spark dagster-spark
SSH / SFTP dagster-ssh
Twilio dagster-twilio