Several Dagster features, like schedules, sensors, and run queueing, require a long-running dagster-daemon process to be included with your deployment.
The easiest way to run the Dagster daemon locally is to run the dagster dev command:
dagster dev
This command launches both the Dagster webserver/UI and the Dagster daemon, allowing you to start a full local deployment of Dagster from the command line. Refer to the Running Dagster locally guide for more information about dagster dev.
For information on deploying the daemon to environments like Docker or Kubernetes, refer to the deployment guides.
The dagster-daemon process reads from your Dagster instance file to determine which daemons should be included. Each of the included daemons then runs on a regular interval in its own threads.
The following daemons are currently available:
| Name | Description | Enabled by |
|---|---|---|
| Scheduler daemon | Creates runs from active schedules | Enabled / runs as long as the default DagsterDaemonScheduler isn't overriden as the scheduler on your instance. |
| Run queue daemon | Launches queued runs, taking into account any limits and prioritization rules set on your instance | Setting the run coordinator on your instance to QueuedRunCoordinator. |
| Sensor daemon | Creates runs from active sensors that are turned on | Always enabled |
| Run monitoring daemon | Handles run worker failures | Using the run_monitoring field in your instance. Refer to the Run monitoring documentation for more info. |
If the daemon is configured to use a workspace file to load code location(s), note that they will periodically reload the file. This means that the dagster-daemon process doesn't need to be restarted when workspace files are changed.
To check the status of the dagster-daemon process in the UI:
This tab displays information about all the daemons currently configured on your instance.
Each daemon periodically writes a heartbeat to your instance storage. If a daemon doesn't show a recent heartbeat, check the logs from your dagster-daemon process for errors.