Skip to main content

14 docs tagged with "compute"

Compute integrations.

View all tags

Dagster & AWS EMR

The AWS EMR integration allows you to seamlessly integrate AWS EMR into your Dagster pipelines for petabyte-scale data processing using open source tools like Apache Spark, Hive, Presto, and more.

Dagster & AWS Glue

The AWS Glue integration enables you to initiate AWS Glue jobs directly from Dagster, seamlessly pass parameters to your code, and stream logs and structured messages back into Dagster.

Dagster & AWS Lambda

Using the AWS Lambda integration with Dagster, you can leverage serverless functions to execute external code in your pipelines.

Dagster & Databricks

The Databricks integration enables you to initiate Databricks jobs directly from Dagster, seamlessly pass parameters to your code, and stream logs and structured messages back into Dagster.

Dagster & Docker

Run runs external processes in docker containers directly from Dagster.

Dagster & GCP Cloud Run

The community-supported `dagster-contrib-gcp` package provides integrations with Google Cloud Platform (GCP) services.

Dagster & HashiCorp

The community-supported `dagster-nomad` package provides an integration with HashiCorp Nomad.

Dagster & Hex

The community-supported `dagster-hex` package provides an integration with Hex.

Dagster & Modal

The community-supported `dagster-modal` package provides an integration with Modal.

Dagster & Perian

The `dagster-perian` integration allows you to easily dockerize your codebase and execute it on the PERIAN platform, PERIAN's serverless GPU environment.