Managing full deployments in Dagster+
Full deployments are standalone environments, allowing you to operate independent instances of Dagster with separately managed permissions.
When a Dagster+ organization is created, a single deployment named prod
will also be created. To create additional full deployments, you must sign up for a Pro plan.
Each full deployment can have one or multiple code locations.
In Dagster+, there are two types of deployments:
- Branch deployments, which are temporary deployments built for testing purposes. We recommend using branch deployments to test your changes, even if you're able to create additional deployments. Branch deployments are available for all Dagster+ users, regardless of plan.
- Full deployments, which are persistent, fully-featured deployments intended to perform actions on a recurring basis.
This guide focuses on full deployments, hereafter referred to as deployments.
Viewing and switching deployments
In Dagster+, you can view and switch between deployments using the deployment switcher:
To view all deployments, click View all deployments.
Creating deployments
Organization Admin permissions are required to create deployments. Additionally, note that creating multiple deployments requires a Pro plan.
To create a deployment:
- Sign in to your Dagster+ account.
- Access the Deployments page using one of the following options:
- Click the deployment switcher > View all deployments.
- Click your user icon > Organization Settings > Deployments.
- Click the + New deployment button.
- In the modal that displays, fill in the following:
- Name - Enter a name for the deployment.
- Initial deployment permissions - Select the permissions you want to use to create the deployment:
- Empty permissions - Creates the deployment with an empty set of permissions. Note: Only Organization Admins will be able to manage the deployment until other uses are granted Admin or Editor permissions.
- Copy from - Creates the deployment using permissions duplicated from an existing deployment.
- When finished, click Create deployment.
Deleting deployments
Organization Admin permissions are required to delete deployments. Additionally, note that deleting a deployment also deletes all its associated data, including code locations, jobs, schedules, and sensors.
To delete a deployment:
- Sign in to your Dagster+ account.
- Access the Deployments page using one of the following options:
- Click the deployment switcher > View all deployments.
- Click the deployment switcher, then the gear icon next to the deployment.
- Click your user icon > Organization Settings > Deployments.
- Click the Delete button next to the deployment you want to delete.
- When prompted, confirm the deletion.
Configuring deployment settings
Editor permissions are required to modify deployment settings.
Deployment settings can be configured in the Dagster+ interface or using the dagster-cloud
CLI. Refer to the deployment settings reference for more info about individual settings.
- Dagster+
- dagster-cloud CLI
To configure deployment settings in the Dagster+ UI:
- Sign in to your Dagster+ account.
- Access the Deployments page using one of the following:
- Click the deployment switcher > View all deployments.
- Click the deployment switcher, then the gear icon next to the deployment.
- Click your user icon > Organization Settings > Deployments.
- Click the Settings button next to the deployment you want to configure.
- In the window that displays, configure settings for the deployment.
- When finished, click Save deployment settings.
dagster-cloud
0.13.14 or later must be installed to run the CLI. Agent and/or job code doesn't need to be upgraded.
Create a file with the settings you'd like to configure. For example:
# my-settings.yaml
run_queue:
max_concurrent_runs: 10
tag_concurrency_limits:
- key: "special-runs"
limit: 5
run_monitoring:
start_timeout_seconds: 1200
cancel_timeout_seconds: 1200
run_retries:
max_retries: 0
Use the CLI to upload the settings file:
dagster-cloud deployment settings set-from-file my-settings.yaml
This will replace all of your configured settings. Any that are not specified will resort to their default values. You also use the CLI to read your current settings, including the default values:
dagster-cloud deployment settings get