Ask AI

Dagster daemon#

Several Dagster features, like schedules, sensors, and run queueing, require a long-running dagster-daemon process to be included with your deployment.

Starting the daemon#

Running locally#

The easiest way to run the Dagster daemon locally is to run the dagster dev command:

dagster dev

This command launches both the Dagster webserver/UI and the Dagster daemon, allowing you to start a full local deployment of Dagster from the command line. Refer to the Running Dagster locally guide for more information about dagster dev.

Deploying the daemon#

For information on deploying the daemon to environments like Docker or Kubernetes, refer to the deployment guides.

Available daemons#

The dagster-daemon process reads from your Dagster instance file to determine which daemons should be included. Each of the included daemons then runs on a regular interval in its own threads.

The following daemons are currently available:

NameDescriptionEnabled by
Scheduler daemonCreates runs from active schedulesEnabled / runs as long as the default DagsterDaemonScheduler isn't overriden as the scheduler on your instance.
Run queue daemonLaunches queued runs, taking into account any limits and prioritization rules set on your instanceSetting the run coordinator on your instance to QueuedRunCoordinator.
Sensor daemonCreates runs from active sensors that are turned onAlways enabled
Run monitoring daemonHandles run worker failuresUsing the run_monitoring field in your instance. Refer to the Run monitoring documentation for more info.

If the daemon is configured to use a workspace file to load code location(s), note that they will periodically reload the file. This means that the dagster-daemon process doesn't need to be restarted when workspace files are changed.

Checking daemon status in the Dagster UI#

To check the status of the dagster-daemon process in the UI:

  1. In the top navigation, click Deployment.
  2. Click the Daemons tab.

This tab displays information about all the daemons currently configured on your instance.

Each daemon periodically writes a heartbeat to your instance storage. If a daemon doesn't show a recent heartbeat, check the logs from your dagster-daemon process for errors.