Ask AI

Running Dagster as a Service#

Running the Dagster webserver#

The core of any deployment of Dagster is the Dagster webserver, a process that serves the Dagster UI and responds to GraphQL queries.

To run the webserver locally, first ensure that you are running a recent Python version. Typically, you'll want to run the webserver inside a virtualenv. Then, you can install the webserver and any additional libraries you might need.

pip install dagster-webserver

To run the webserver, use a command like the following:

DAGSTER_HOME=/opt/dagster/dagster_home dagster-webserver -h 0.0.0.0 -p 3000

In this configuration, the webserver will write execution logs to $DAGSTER_HOME/logs and listen on 0.0.0.0:3000.

Running the Dagster daemon#

If you're using schedules, sensors, or backfills, or want to set limits on the number of runs that can be executed at once, you'll want to also run a dagster-daemon service as part of your deployment. To run this service locally, run the following command:

pip install dagster

DAGSTER_HOME=/opt/dagster/dagster_home dagster-daemon run

The dagster-daemon process will periodically check your instance for any new runs that should be launched from your run queue or triggered by your running schedules or sensors. If you're running dagster-daemon in a different environment than dagster-webserver, it must also have access to your dagster.yaml file and workspace.yaml file and be able to access the components defined on your instance, and also be able to load the repositories defined in your workspace.

You can check the status of your dagster-daemon process in the UI by navigating to the Instance tab in the navigation bar on the left-hand side of the screen. A deployment can have multiple simultaneous instances of dagster-webserver, but should include only a single dagster-daemon process.