Execution¶
Executing pipelines¶
-
dagster.
execute_pipeline
(pipeline: Union[dagster.core.definitions.pipeline.PipelineDefinition, dagster.core.definitions.pipeline_base.IPipeline], run_config: Optional[dict] = None, mode: Optional[str] = None, preset: Optional[str] = None, tags: Optional[Dict[str, Any]] = None, solid_selection: Optional[List[str]] = None, instance: Optional[dagster.core.instance.DagsterInstance] = None, raise_on_error: bool = True) → dagster.core.execution.results.PipelineExecutionResult[source]¶ Execute a pipeline synchronously.
Users will typically call this API when testing pipeline execution, or running standalone scripts.
- Parameters
pipeline (Union[IPipeline, PipelineDefinition]) – The pipeline to execute.
run_config (Optional[dict]) – The environment configuration that parametrizes this run, as a dict.
mode (Optional[str]) – The name of the pipeline mode to use. You may not set both
mode
andpreset
.preset (Optional[str]) – The name of the pipeline preset to use. You may not set both
mode
andpreset
.tags (Optional[Dict[str, Any]]) – Arbitrary key-value pairs that will be added to pipeline logs.
instance (Optional[DagsterInstance]) – The instance to execute against. If this is
None
, an ephemeral instance will be used, and no artifacts will be persisted from the run.raise_on_error (Optional[bool]) – Whether or not to raise exceptions when they occur. Defaults to
True
, since this is the most useful behavior in test.solid_selection (Optional[List[str]]) –
A list of solid selection queries (including single solid names) to execute. For example: - [‘some_solid’]: select “some_solid” itself. - [‘*some_solid’]: select “some_solid” and all its ancestors (upstream dependencies). - [‘*some_solid+++’]: select “some_solid”, all its ancestors, and its descendants
(downstream dependencies) within 3 levels down.
- [‘*some_solid’, ‘other_solid_a’, ‘other_solid_b+’]: select “some_solid” and all its
ancestors, “other_solid_a” itself, and “other_solid_b” and its direct child solids.
- Returns
The result of pipeline execution.
- Return type
For the asynchronous version, see
execute_pipeline_iterator()
.
-
dagster.
execute_pipeline_iterator
(pipeline: Union[dagster.core.definitions.pipeline.PipelineDefinition, dagster.core.definitions.pipeline_base.IPipeline], run_config: Optional[dict] = None, mode: Optional[str] = None, preset: Optional[str] = None, tags: Optional[Dict[str, Any]] = None, solid_selection: Optional[List[str]] = None, instance: Optional[dagster.core.instance.DagsterInstance] = None) → Iterator[dagster.core.events.DagsterEvent][source]¶ Execute a pipeline iteratively.
Rather than package up the result of running a pipeline into a single object, like
execute_pipeline()
, this function yields the stream of events resulting from pipeline execution.This is intended to allow the caller to handle these events on a streaming basis in whatever way is appropriate.
- Parameters
pipeline (Union[IPipeline, PipelineDefinition]) – The pipeline to execute.
run_config (Optional[dict]) – The environment configuration that parametrizes this run, as a dict.
mode (Optional[str]) – The name of the pipeline mode to use. You may not set both
mode
andpreset
.preset (Optional[str]) – The name of the pipeline preset to use. You may not set both
mode
andpreset
.tags (Optional[Dict[str, Any]]) – Arbitrary key-value pairs that will be added to pipeline logs.
solid_selection (Optional[List[str]]) –
A list of solid selection queries (including single solid names) to execute. For example: - [‘some_solid’]: select “some_solid” itself. - [‘*some_solid’]: select “some_solid” and all its ancestors (upstream dependencies). - [‘*some_solid+++’]: select “some_solid”, all its ancestors, and its descendants
(downstream dependencies) within 3 levels down.
- [‘*some_solid’, ‘other_solid_a’, ‘other_solid_b+’]: select “some_solid” and all its
ancestors, “other_solid_a” itself, and “other_solid_b” and its direct child solids.
instance (Optional[DagsterInstance]) – The instance to execute against. If this is
None
, an ephemeral instance will be used, and no artifacts will be persisted from the run.
- Returns
The stream of events resulting from pipeline execution.
- Return type
Iterator[DagsterEvent]
Re-executing pipelines¶
-
dagster.
reexecute_pipeline
(pipeline: Union[dagster.core.definitions.pipeline_base.IPipeline, dagster.core.definitions.pipeline.PipelineDefinition], parent_run_id: str, run_config: Optional[dict] = None, step_selection: Optional[List[str]] = None, mode: Optional[str] = None, preset: Optional[str] = None, tags: Optional[Dict[str, Any]] = None, instance: dagster.core.instance.DagsterInstance = None, raise_on_error: bool = True) → dagster.core.execution.results.PipelineExecutionResult[source]¶ Reexecute an existing pipeline run.
Users will typically call this API when testing pipeline reexecution, or running standalone scripts.
- Parameters
pipeline (Union[IPipeline, PipelineDefinition]) – The pipeline to execute.
parent_run_id (str) – The id of the previous run to reexecute. The run must exist in the instance.
run_config (Optional[dict]) – The environment configuration that parametrizes this run, as a dict.
step_selection (Optional[List[str]]) –
A list of step selection queries (including single step keys) to execute. For example: - [‘some_solid’]: select the execution step “some_solid” itself. - [‘*some_solid’]: select the step “some_solid” and all its ancestors
(upstream dependencies).
- [‘*some_solid+++’]: select the step “some_solid”, all its ancestors,
and its descendants (downstream dependencies) within 3 levels down.
- [‘*some_solid’, ‘other_solid_a’, ‘other_solid_b+’]: select
”some_solid” and all its ancestors, “other_solid_a” itself, and “other_solid_b” and its direct child execution steps.
mode (Optional[str]) – The name of the pipeline mode to use. You may not set both
mode
andpreset
.preset (Optional[str]) – The name of the pipeline preset to use. You may not set both
mode
andpreset
.tags (Optional[Dict[str, Any]]) – Arbitrary key-value pairs that will be added to pipeline logs.
instance (Optional[DagsterInstance]) – The instance to execute against. If this is
None
, an ephemeral instance will be used, and no artifacts will be persisted from the run.raise_on_error (Optional[bool]) – Whether or not to raise exceptions when they occur. Defaults to
True
, since this is the most useful behavior in test.
- Returns
The result of pipeline execution.
- Return type
For the asynchronous version, see
reexecute_pipeline_iterator()
.
-
dagster.
reexecute_pipeline_iterator
(pipeline: Union[dagster.core.definitions.pipeline_base.IPipeline, dagster.core.definitions.pipeline.PipelineDefinition], parent_run_id: str, run_config: Optional[dict] = None, step_selection: Optional[List[str]] = None, mode: Optional[str] = None, preset: Optional[str] = None, tags: Optional[Dict[str, Any]] = None, instance: dagster.core.instance.DagsterInstance = None) → Iterator[dagster.core.events.DagsterEvent][source]¶ Reexecute a pipeline iteratively.
Rather than package up the result of running a pipeline into a single object, like
reexecute_pipeline()
, this function yields the stream of events resulting from pipeline reexecution.This is intended to allow the caller to handle these events on a streaming basis in whatever way is appropriate.
- Parameters
pipeline (Union[IPipeline, PipelineDefinition]) – The pipeline to execute.
parent_run_id (str) – The id of the previous run to reexecute. The run must exist in the instance.
run_config (Optional[dict]) – The environment configuration that parametrizes this run, as a dict.
step_selection (Optional[List[str]]) –
A list of step selection queries (including single step keys) to execute. For example: - [‘some_solid’]: select the execution step “some_solid” itself. - [‘*some_solid’]: select the step “some_solid” and all its ancestors
(upstream dependencies).
- [‘*some_solid+++’]: select the step “some_solid”, all its ancestors,
and its descendants (downstream dependencies) within 3 levels down.
- [‘*some_solid’, ‘other_solid_a’, ‘other_solid_b+’]: select
”some_solid” and all its ancestors, “other_solid_a” itself, and “other_solid_b” and its direct child execution steps.
mode (Optional[str]) – The name of the pipeline mode to use. You may not set both
mode
andpreset
.preset (Optional[str]) – The name of the pipeline preset to use. You may not set both
mode
andpreset
.tags (Optional[Dict[str, Any]]) – Arbitrary key-value pairs that will be added to pipeline logs.
instance (Optional[DagsterInstance]) – The instance to execute against. If this is
None
, an ephemeral instance will be used, and no artifacts will be persisted from the run.
- Returns
The stream of events resulting from pipeline reexecution.
- Return type
Iterator[DagsterEvent]
Executing solids¶
-
dagster.
execute_solid
(solid_def, mode_def=None, input_values=None, tags=None, run_config=None, raise_on_error=True)[source]¶ Execute a single solid in an ephemeral pipeline.
Intended to support unit tests. Input values may be passed directly, and no pipeline need be specified – an ephemeral pipeline will be constructed.
- Parameters
solid_def (SolidDefinition) – The solid to execute.
mode_def (Optional[ModeDefinition]) – The mode within which to execute the solid. Use this if, e.g., custom resources, loggers, or executors are desired.
input_values (Optional[Dict[str, Any]]) – A dict of input names to input values, used to pass inputs to the solid directly. You may also use the
run_config
to configure any inputs that are configurable.tags (Optional[Dict[str, Any]]) – Arbitrary key-value pairs that will be added to pipeline logs.
run_config (Optional[dict]) – The environment configuration that parameterized this execution, as a dict.
raise_on_error (Optional[bool]) – Whether or not to raise exceptions when they occur. Defaults to
True
, since this is the most useful behavior in test.
- Returns
The result of executing the solid.
- Return type
-
dagster.
execute_solid_within_pipeline
(pipeline_def, solid_name, inputs=None, run_config=None, mode=None, preset=None, tags=None, instance=None)[source]¶ Execute a single solid within an existing pipeline.
Intended to support tests. Input values may be passed directly.
- Parameters
pipeline_def (PipelineDefinition) – The pipeline within which to execute the solid.
solid_name (str) – The name of the solid, or the aliased solid, to execute.
inputs (Optional[Dict[str, Any]]) – A dict of input names to input values, used to pass input values to the solid directly. You may also use the
run_config
to configure any inputs that are configurable.run_config (Optional[dict]) – The environment configuration that parameterized this execution, as a dict.
mode (Optional[str]) – The name of the pipeline mode to use. You may not set both
mode
andpreset
.preset (Optional[str]) – The name of the pipeline preset to use. You may not set both
mode
andpreset
.tags (Optional[Dict[str, Any]]) – Arbitrary key-value pairs that will be added to pipeline logs.
instance (Optional[DagsterInstance]) – The instance to execute against. If this is
None
, an ephemeral instance will be used, and no artifacts will be persisted from the run.
- Returns
The result of executing the solid.
- Return type
-
dagster.
execute_solids_within_pipeline
(pipeline_def, solid_names, inputs=None, run_config=None, mode=None, preset=None, tags=None, instance=None)[source]¶ Execute a set of solids within an existing pipeline.
Intended to support tests. Input values may be passed directly.
- Parameters
pipeline_def (PipelineDefinition) – The pipeline within which to execute the solid.
solid_names (FrozenSet[str]) – A set of the solid names, or the aliased solids, to execute.
inputs (Optional[Dict[str, Dict[str, Any]]]) – A dict keyed on solid names, whose values are dicts of input names to input values, used to pass input values to the solids directly. You may also use the
run_config
to configure any inputs that are configurable.run_config (Optional[dict]) – The environment configuration that parameterized this execution, as a dict.
mode (Optional[str]) – The name of the pipeline mode to use. You may not set both
mode
andpreset
.preset (Optional[str]) – The name of the pipeline preset to use. You may not set both
mode
andpreset
.tags (Optional[Dict[str, Any]]) – Arbitrary key-value pairs that will be added to pipeline logs.
instance (Optional[DagsterInstance]) – The instance to execute against. If this is
None
, an ephemeral instance will be used, and no artifacts will be persisted from the run.
- Returns
The results of executing the solids, keyed by solid name.
- Return type
Dict[str, Union[CompositeSolidExecutionResult, SolidExecutionResult]]
Execution context¶
-
class
dagster.core.execution.context.compute.
SolidExecutionContext
(system_compute_execution_context: dagster.core.execution.context.system.SystemComputeExecutionContext)[source]¶ The
context
object available to solid compute logic.-
property
instance
¶ The current Instance
-
property
pdb
¶ Allows pdb debugging from within the solid.
Example:
@solid def debug_solid(context): context.pdb.set_trace()
-
property
pipeline_run
¶ The current PipelineRun
-
property
solid_config
¶ The parsed config specific to this solid.
-
property
-
class
dagster.core.execution.context.compute.
AbstractComputeExecutionContext
[source]¶ Base class for solid context implemented by SolidExecutionContext and DagstermillExecutionContext
-
abstract property
log
¶ The log manager available in the execution context.
-
abstract property
pipeline_def
¶ The pipeline being executed.
-
abstract property
pipeline_run
¶ The PipelineRun object corresponding to the execution.
-
abstract property
resources
¶ Resources available in the execution context.
-
abstract property
run_id
¶ The run id for the context.
-
abstract property
solid
¶ The solid corresponding to the execution step being executed.
-
abstract property
solid_config
¶ The parsed config specific to this solid.
-
abstract property
solid_def
¶ The solid definition corresponding to the execution step being executed.
-
abstract property
-
class
dagster.
reconstructable
[source]¶ Create a ReconstructablePipeline from a function that returns a PipelineDefinition, or a function decorated with
@pipeline
When your pipeline must cross process boundaries, e.g., for execution on multiple nodes or in different systems (like dagstermill), Dagster must know how to reconstruct the pipeline on the other side of the process boundary.
This function implements a very conservative strategy for reconstructing pipelines, so that its behavior is easy to predict, but as a consequence it is not able to reconstruct certain kinds of pipelines, such as those defined by lambdas, in nested scopes (e.g., dynamically within a method call), or in interactive environments such as the Python REPL or Jupyter notebooks.
If you need to reconstruct pipelines constructed in these ways, you should use
build_reconstructable_pipeline()
instead, which allows you to specify your own strategy for reconstructing a pipeline.Examples:
from dagster import PipelineDefinition, pipeline, reconstructable @pipeline def foo_pipeline(): ... reconstructable_foo_pipeline = reconstructable(foo_pipeline) def make_bar_pipeline(): return PipelineDefinition(...) reconstructable_bar_pipeline = reconstructable(bar_pipeline)
Pipeline and solid results¶
-
class
dagster.
PipelineExecutionResult
(pipeline_def, run_id, event_list, reconstruct_context, resource_instances_to_override=None)[source]¶ The result of executing a pipeline.
Returned by
execute_pipeline()
. Users should not instantiate this class.
-
class
dagster.
SolidExecutionResult
(solid, step_events_by_kind, reconstruct_context, resource_instances_to_override=None)[source]¶ Execution result for a leaf solid in a pipeline.
Users should not instantiate this class.
-
property
compute_input_event_dict
¶ All events of type
STEP_INPUT
, keyed by input name.- Type
Dict[str, DagsterEvent]
-
property
compute_output_events_dict
¶ All events of type
STEP_OUTPUT
, keyed by output name- Type
Dict[str, List[DagsterEvent]]
-
property
compute_step_events
¶ All events generated by execution of the solid compute function.
- Type
List[DagsterEvent]
-
property
compute_step_failure_event
¶ The
STEP_FAILURE
event, throws if it did not fail.- Type
-
property
expectation_events_during_compute
¶ All events of type
STEP_EXPECTATION_RESULT
.- Type
List[DagsterEvent]
-
property
expectation_results_during_compute
¶ All expectation results yielded by the solid
- Type
List[ExpectationResult]
-
property
failure_data
¶ Any data corresponding to this step’s failure, if it failed.
- Type
Union[None, StepFailureData]
-
get_output_event_for_compute
(output_name='result')[source]¶ The
STEP_OUTPUT
event for the given output name.Throws if not present.
- Parameters
output_name (Optional[str]) – The name of the output. (default: ‘result’)
- Returns
The corresponding event.
- Return type
-
get_output_events_for_compute
(output_name='result')[source]¶ The
STEP_OUTPUT
event for the given output name.Throws if not present.
- Parameters
output_name (Optional[str]) – The name of the output. (default: ‘result’)
- Returns
The corresponding events.
- Return type
List[DagsterEvent]
-
property
input_events_during_compute
¶ All events of type
STEP_INPUT
.- Type
List[DagsterEvent]
-
property
materialization_events_during_compute
¶ All events of type
STEP_MATERIALIZATION
.- Type
List[DagsterEvent]
-
property
materializations_during_compute
¶ All materializations yielded by the solid.
- Type
List[Materialization]
-
property
output_events_during_compute
¶ All events of type
STEP_OUTPUT
.- Type
List[DagsterEvent]
-
output_value
(output_name='result')[source]¶ Get a computed output value.
Note that calling this method will reconstruct the pipeline context (including, e.g., resources) to retrieve materialized output values.
-
property
output_values
¶ The computed output values.
Returns
None
if execution did not succeed.- Returns a dictionary where keys are output names and the values are:
the output values in the normal case
a dictionary from mapping key to corresponding value in the mapped case
Note that accessing this property will reconstruct the pipeline context (including, e.g., resources) to retrieve materialized output values.
-
property
-
class
dagster.
CompositeSolidExecutionResult
(solid, event_list, step_events_by_kind, reconstruct_context, handle=None, resource_instances_to_override=None)[source]¶ Execution result for a composite solid in a pipeline.
Users should not instantiate this class.
-
class
dagster.
DagsterEvent
[source]¶ Events yielded by solid and pipeline execution.
Users should not instantiate this class.
-
solid_handle
¶ - Type
SolidHandle
-
event_specific_data
¶ Type must correspond to event_type_value.
- Type
Any
-
step_key
DEPRECATED
- Type
Optional[str]
-
property
event_type
¶ The type of this event.
- Type
-
-
class
dagster.
DagsterEventType
[source]¶ The types of events that may be yielded by solid and pipeline execution.
-
ASSET_STORE_OPERATION
= 'ASSET_STORE_OPERATION'¶
-
ENGINE_EVENT
= 'ENGINE_EVENT'¶
-
HANDLED_OUTPUT
= 'HANDLED_OUTPUT'¶
-
HOOK_COMPLETED
= 'HOOK_COMPLETED'¶
-
HOOK_ERRORED
= 'HOOK_ERRORED'¶
-
HOOK_SKIPPED
= 'HOOK_SKIPPED'¶
-
LOADED_INPUT
= 'LOADED_INPUT'¶
-
OBJECT_STORE_OPERATION
= 'OBJECT_STORE_OPERATION'¶
-
PIPELINE_CANCELED
= 'PIPELINE_CANCELED'¶
-
PIPELINE_CANCELING
= 'PIPELINE_CANCELING'¶
-
PIPELINE_DEQUEUED
= 'PIPELINE_DEQUEUED'¶
-
PIPELINE_ENQUEUED
= 'PIPELINE_ENQUEUED'¶
-
PIPELINE_FAILURE
= 'PIPELINE_FAILURE'¶
-
PIPELINE_INIT_FAILURE
= 'PIPELINE_INIT_FAILURE'¶
-
PIPELINE_START
= 'PIPELINE_START'¶
-
PIPELINE_STARTING
= 'PIPELINE_STARTING'¶
-
PIPELINE_SUCCESS
= 'PIPELINE_SUCCESS'¶
-
STEP_EXPECTATION_RESULT
= 'STEP_EXPECTATION_RESULT'¶
-
STEP_FAILURE
= 'STEP_FAILURE'¶
-
STEP_INPUT
= 'STEP_INPUT'¶
-
STEP_MATERIALIZATION
= 'STEP_MATERIALIZATION'¶
-
STEP_OUTPUT
= 'STEP_OUTPUT'¶
-
STEP_RESTARTED
= 'STEP_RESTARTED'¶
-
STEP_SKIPPED
= 'STEP_SKIPPED'¶
-
STEP_START
= 'STEP_START'¶
-
STEP_SUCCESS
= 'STEP_SUCCESS'¶
-
STEP_UP_FOR_RETRY
= 'STEP_UP_FOR_RETRY'¶
-
Pipeline configuration¶
Run Config Schema¶
The
run_config
used byexecute_pipeline()
andexecute_pipeline_iterator()
has the following schema:{ # configuration for execution, required if executors require config execution: { # the name of one, and only one available executor, typically 'in_process' or 'multiprocess' __executor_name__: { # executor-specific config, if required or permitted config: { ... } } }, # configuration for loggers, required if loggers require config loggers: { # the name of an available logger __logger_name__: { # logger-specific config, if required or permitted config: { ... } }, ... }, # configuration for resources, required if resources require config resources: { # the name of a resource __resource_name__: { # resource-specific config, if required or permitted config: { ... } }, ... }, # configuration for solids, required if solids require config solids: { # these keys align with the names of the solids, or their alias in this pipeline __solid_name__: { # pass any data that was defined via config_field config: ..., # configurably specify input values, keyed by input name inputs: { __input_name__: { # if an dagster_type_loader is specified, that schema must be satisfied here; # scalar, built-in types will generally allow their values to be specified directly: value: ... } }, # configurably materialize output values outputs: { __output_name__: { # if an dagster_type_materializer is specified, that schema must be satisfied # here; pickleable types will generally allow output as follows: pickle: { path: String } } } } }, # optionally use an available system storage for intermediates etc. intermediate_storage: { # the name of one, and only one available system storage, typically 'filesystem' or # 'in_memory' __storage_name__: { config: { ... } } } }
Intermediate Storage¶
-
dagster.
io_manager_from_intermediate_storage
(intermediate_storage_def)[source]¶ Define an
IOManagerDefinition
from an existingIntermediateStorageDefinition
.This method is used to adapt an existing user-defined intermediate storage to a IO manager resource, for example:
my_io_manager_def = io_manager_from_intermediate_storage(my_intermediate_storage_def) @pipeline(mode_defs=[ModeDefinition(resource_defs={"io_manager": my_io_manager_def})]) def my_pipeline(): ...
- Parameters
intermediate_storage_def (IntermediateStorageDefinition) – The intermediate storage definition to be converted to an IO manager definition.
- Returns
IOManagerDefinition
-
dagster.
mem_intermediate_storage
IntermediateStorageDefinition[source]¶ The default in-memory intermediate storage.
In-memory intermediate storage is the default on any pipeline run that does not configure any custom intermediate storage.
Keep in mind when using this storage that intermediates will not be persisted after the pipeline run ends. Use a persistent intermediate storage like
fs_intermediate_storage()
to persist intermediates and take advantage of advanced features like pipeline re-execution.
-
dagster.
fs_intermediate_storage
IntermediateStorageDefinition[source]¶ The default filesystem intermediate storage.
Filesystem system storage is available by default on any
ModeDefinition
that does not provide custom system storages. To select it, include a fragment such as the following in config:intermediate_storage: filesystem: base_dir: '/path/to/dir/'
You may omit the
base_dir
config value, in which case the filesystem storage will use theDagsterInstance
-provided default.
-
dagster.
default_intermediate_storage_defs
List[IntermediateStorageDefinition]¶ Built-in mutable sequence.
If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified. The default intermediate storages available on any
ModeDefinition
that does not provide custom intermediate storages. These are currently [mem_intermediate_storage
,fs_intermediate_storage
].
Executors¶
-
dagster.
in_process_executor
ExecutorDefinition[source]¶ The default in-process executor.
In most Dagster environments, this will be the default executor. It is available by default on any
ModeDefinition
that does not provide custom executors. To select it explicitly, include the following top-level fragment in config:execution: in_process:
Execution priority can be configured using the
dagster/priority
tag via solid metadata, where the higher the number the higher the priority. 0 is the default and both positive and negative numbers can be used.
-
dagster.
multiprocess_executor
ExecutorDefinition[source]¶ The default multiprocess executor.
This simple multiprocess executor is available by default on any
ModeDefinition
that does not provide custom executors. To select the multiprocess executor, include a fragment such as the following in your config:execution: multiprocess: config: max_concurrent: 4
The
max_concurrent
arg is optional and tells the execution engine how many processes may run concurrently. By default, or if you setmax_concurrent
to be 0, this is the return value ofmultiprocessing.cpu_count()
.Execution priority can be configured using the
dagster/priority
tag via solid metadata, where the higher the number the higher the priority. 0 is the default and both positive and negative numbers can be used.
-
dagster.
default_executors
List[ExecutorDefinition]¶ Built-in mutable sequence.
If no argument is given, the constructor creates a new empty list. The argument must be an iterable if specified. The default executors available on any
ModeDefinition
that does not provide custom executors. These are currently [in_process_executor
,multiprocess_executor
].
Contexts¶
-
class
dagster.
SystemComputeExecutionContext
(execution_context_data: dagster.core.execution.context.system.SystemExecutionContextData, log_manager: dagster.core.log_manager.DagsterLogManager, step: dagster.core.execution.plan.step.ExecutionStep, output_capture: Optional[Dict[dagster.core.execution.plan.outputs.StepOutputHandle, Any]] = None)[source]¶
-
class
dagster.
TypeCheckContext
(execution_context_data: dagster.core.execution.context.system.SystemExecutionContextData, log_manager: dagster.core.log_manager.DagsterLogManager, dagster_type: dagster.core.types.dagster_type.DagsterType)[source]¶ The
context
object available to a type check function on a DagsterType.-
log
¶ Centralized log dispatch from user code.
- Type
-
resources
¶ An object whose attributes contain the resources available to this solid.
- Type
Any
-
-
class
dagster.
HookContext
(execution_context_data: dagster.core.execution.context.system.SystemExecutionContextData, log_manager: dagster.core.log_manager.DagsterLogManager, hook_def: dagster.core.definitions.hook.HookDefinition, step: dagster.core.execution.plan.step.ExecutionStep)[source]¶ The
context
object available to a hook function on an DagsterEvent.-
log
¶ Centralized log dispatch from user code.
- Type
-
hook_def
¶ The hook that the context object belongs to.
- Type
HookDefinition
-
step
¶ The compute step associated with the hook.
- Type
ExecutionStep
-
solid
¶ The solid instance associated with the hook.
- Type
Solid
-
resources
¶ Resources available in the hook context.
- Type
NamedTuple
-
solid_config
¶ The parsed config specific to this solid.
- Type
Any
-