To start the
dagster-daemon process locally, launch the following command in the same folder as your
workspace.yaml file and keep the process running:
$ dagster-daemon run
Check the Deployment Guides section for more information on how to deploy the daemon in other environments, like Docker or Kubernetes.
dagster-daemon process reads from your Dagster instance file to determine which daemons it should include. Each of those daemons then runs on a regular interval in its own threads. Daemons that need to load your workspace will periodically reload the workspace from your
workspace.yaml file. If you change your
workspace.yaml file, you do not need to reload the
dagster-daemon process in order for those changes to be included.
The following daemons are currently available:
The scheduler daemon is responsible for creating runs from any schedules that are turned on. This daemon will run as long as you have not overridden the default
DagsterDaemonScheduler as the scheduler on your instance.
The run queue daemon is responsible for launching queued runs, taking into account any limits and prioritization rules you've set on your instance. You can enable this daemon by setting the run coordinator on your instance to
The sensor daemon is responsible for creating runs from any sensors that are turned on. This daemon is always enabled.
To check the status of your
dagster-daemon process within Dagit, click on "Status" in the left nav. This will take you a page where you can see information about each daemon that's currently configured on your instance.
Each daemon periodically writes a heartbeat to your instance storage, so if a daemon doesn't show a recent heartbeat on this page, it likely indicates that you should check the logs from your
dagster-daemon process for errors.