Presets

class dagster.PresetDefinition[source]

Defines a preset configuration in which a pipeline can execute.

Presets can be used in Dagit to load predefined configurations into the tool.

Presets may also be used from the Python API (in a script, or in test) as follows:

execute_pipeline(pipeline_def, preset='example_preset')

Presets may also be used with the command line tools:

$ dagster pipeline execute example_pipeline --preset example_preset
Parameters

  • name (str) – The name of this preset. Must be unique in the presets defined on a given pipeline.

  • run_config (Optional[dict]) – A dict representing the config to set with the preset. This is equivalent to the run_config argument to execute_pipeline().

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute with the preset. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The mode to apply when executing this preset. (default: ‘default’)

  • tags (Optional[Dict[str, Any]]) – The tags to apply when executing this preset.

static from_files(name, environment_files=None, config_files=None, solid_selection=None, mode=None, tags=None)[source]

Static constructor for presets from YAML files.

Parameters

  • name (str) – The name of this preset. Must be unique in the presets defined on a given pipeline.

  • config_files (Optional[List[str]]) – List of paths or glob patterns for yaml files to load and parse as the environment config for this preset.

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute with the preset. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The mode to apply when executing this preset. (default: ‘default’)

  • tags (Optional[Dict[str, Any]]) – The tags to apply when executing this preset.

Returns

A PresetDefinition constructed from the provided YAML files.

Return type

PresetDefinition

Raises

DagsterInvariantViolationError – When one of the YAML files is invalid and has a parse error.

static from_pkg_resources(name, pkg_resource_defs=None, solid_selection=None, mode=None, tags=None)[source]

Load a preset from a package resource, using pkg_resources.resource_string().

Example:

PresetDefinition.from_pkg_resources(
    name='local',
    mode='local',
    pkg_resource_defs=[
        ('dagster_examples.airline_demo.environments', 'local_base.yaml'),
        ('dagster_examples.airline_demo.environments', 'local_warehouse.yaml'),
    ],
)
Parameters

  • name (str) – The name of this preset. Must be unique in the presets defined on a given pipeline.

  • pkg_resource_defs (Optional[List[(str, str)]]) – List of pkg_resource modules/files to load as environment config for this preset.

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute with this partition. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The mode to apply when executing this preset. (default: ‘default’)

  • tags (Optional[Dict[str, Any]]) – The tags to apply when executing this preset.

Returns

A PresetDefinition constructed from the provided YAML strings

Return type

PresetDefinition

Raises

DagsterInvariantViolationError – When one of the YAML documents is invalid and has a parse error.

static from_yaml_strings(name, yaml_strings=None, solid_selection=None, mode=None, tags=None)[source]

Static constructor for presets from YAML strings.

Parameters

  • name (str) – The name of this preset. Must be unique in the presets defined on a given pipeline.

  • yaml_strings (Optional[List[str]]) – List of yaml strings to parse as the environment config for this preset.

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute with the preset. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The mode to apply when executing this preset. (default: ‘default’)

  • tags (Optional[Dict[str, Any]]) – The tags to apply when executing this preset.

Returns

A PresetDefinition constructed from the provided YAML strings

Return type

PresetDefinition

Raises

DagsterInvariantViolationError – When one of the YAML documents is invalid and has a parse error.

get_environment_yaml()[source]

Get the environment dict set on a preset as YAML.

Returns

The environment dict as YAML.

Return type

str

with_additional_config(run_config)[source]

Return a new PresetDefinition with additional config merged in to the existing config.