Jobs#

Jobs are the main unit of execution and monitoring in Dagster. They allow you to execute a portion of a graph of Software-defined Assets or ops based on a schedule or an external trigger.

When a job begins, it kicks off a run. A run is a single execution of a job in Dagster. Runs can be launched and viewed in the Dagster UI.


Benefits#

Using jobs provides the following benefits:

  • Automation: With schedules and sensors, jobs can be used to automate the execution of your Dagster pipelines. Refer to the Automation guide for more info.
  • Control job run priority: If using a run queue, you can apply custom prioritization rules to how job runs are prioritized and executed.
  • Potential for improved efficency: By applying concurrency limits to job runs, there may be benefits to your pipeline's efficiency. Refer to the Limiting run concurrency guide for more info and examples.

Uses#

Jobs are supported for both Software-defined Assets and ops, but the usage for each concept is unique. Refer to the following documentation for more info:

With jobs, you can:

  • Automate the execution of Software-defined Assets and ops
  • Materialize a selection of assets based on a schedule or external trigger (sensor)
  • Apply metadata about jobs and runs using tags
  • View and launch runs of jobs in the Dagster UI