Ask AI

Dagster+ deployment settings reference#

This guide is applicable to Dagster+.

This reference describes the settings that can be configured for full deployments in Dagster+.

Refer to the Managing deployments in Dagster+ guide for info about configuring settings in the Dagster+ interface or using the dagster-plus CLI.


Settings schema#

Settings are formatted using YAML. For example:

run_queue:
  max_concurrent_runs: 10
  tag_concurrency_limits:
    - key: "database"
      value: "redshift"
      limit: 5

run_monitoring:
  start_timeout_seconds: 1200
  cancel_timeout_seconds: 1200

run_retries:
  max_retries: 0

sso_default_role: EDITOR

auto_materialize:
  run_tags: {}

Settings#

For each deployment, you can configure settings for:

Run queue (run_queue)#

The run_queue settings allow you to specify how many runs can execute concurrently in the deployment.

run_queue:
  max_concurrent_runs: 10
  tag_concurrency_limits:
    - key: "database"
      value: "redshift"
      limit: 5
PropertyDescription
run_queue.max_concurrent_runsThe maximum number of runs that are allowed to be in progress at once. Set to 0 to stop any runs from launching. Negative values aren't permitted.
run_queue.tag_concurrency_limitsA list of limits applied to runs with particular tags.
  • Default - []
Each list item may have the following properties:
  • key
  • value
    • If defined, the limit is applied only to the key-value pair.
    • If not defined, the limit is applied across all values of thekey.
    • If set to a dict with applyLimitPerUniqueValue: true, the limit is applied to the number of unique values for the key.
  • limit

Run monitoring (run_monitoring)#

The run_monitoring settings allow you to define how long Dagster+ should wait for runs to start before making them as failed, or to terminate before marking them as canceled.

run_monitoring:
  start_timeout_seconds: 1200
  cancel_timeout_seconds: 1200
PropertyDescription
run_monitoring.start_timeout_secondsThe number of seconds that Dagster+ will wait after a run is launched for the process or container to start executing. After the timeout, the run will fail. This prevents runs from hanging in STARTING indefinitely when the process or container doesn't start.
  • Default - 1200 (20 minutes)

Run retries (run_retries)#

The run_retries settings allow you to define how Dagster+ handles retrying failed runs in the deployment.

run_retries:
  max_retries: 0
PropertyDescription
run_retries.max_retriesThe maximum number of times Dagster+ should attempt to retry a failed run. Dagster+ will use the default if this setting is undefined.
  • Default - 0
run_retries.retry_on_asset_or_op_failureWhether to retry runs that failed due to assets or ops in the run failing. Set this to false if you only want to retry failures that occur due to the run worker crashing or unexpectedly terminating, and instead rely on op or asset-level retry policies to retry assert or op failures. Setting this field to false will only change retry behavior for runs on dagster version 1.6.7 or greater.
  • Default - 0

SSO default role#

The sso_default_role setting lets you configure the default role on the deployment which is granted to new users that log in via SSO. For more information on available roles, see the Dagster+ permissions reference.

sso_default_role: EDITOR
PropertyDescription
sso_default_roleIf SAML SSO is enabled, this is the default role that will be assigned to Dagster+ users for this deployment. If SAML SSO is not enabled, this setting is ignored.
  • Default - VIEWER

Non-isolated runs#

Configure non-isolated runs on your deployment.

non_isolated_runs:
  enabled: True
  max_concurrent_non_isolated_runs: 1
PropertyDescription
enabledIf enabled, the `Isolate run environment` checkbox will appear in the Launchpad.
  • Default - true
max_concurrent_non_isolated_runsA limit for how many non-isolated runs to launch at once. Once this limit is reached, the checkbox will be greyed out and all runs will be isolated. This helps to avoid running out of RAM on the code location server.
  • Default - 1

Auto-materialize#

Configure auto-materializing assets for your deployment.

auto_materialize:
  run_tags:
    foo: bar
  use_sensors: False
PropertyDescription
run_tagsTags to apply to all runs created for an auto-materialize policy.
  • Default - None