In production Dagster deployments, there are often many runs being launched at once. The run coordinator lets you control the policy that Dagster uses to manage the set of runs in your deployment.
When you submit a run from the Dagster UI or the Dagster command line, it’s first sent to the run coordinator, which applies any limits or prioritization policies before eventually sending it to the run launcher to be launched.
The following run coordinators can be configured on your Dagster instance:
| Term | Definition |
|---|---|
DefaultRunCoordinator | The DefaultRunCoordinator calls launch_run on the instance’s run launcher immediately in the same process, without applying any limits or prioritization rules.When this coordinator is set, clicking Launch Run in the Dagster UI will immediately launch the run from the Dagster daemon process. Similarly, scheduled runs will immediately launch from the scheduler process. |
QueuedRunCoordinator | The QueuedRunCoordinator sends runs to the Dagster daemon via a run queue. The daemon pulls runs from the queue and calls launch_run on submitted runs.Using this run coordinator enables instance-level limits on run concurrency, as well as custom run prioritization rules. |
If you opt to use the DefaultRunCoordinator, no configuration is required on your part.
However, if using the QueuedRunCoordinator or building a custom implementation, you can define custom run prioritization rules and instance-level concurrency limits.