Op inputs and outputs can be given Dagster Types. Assets can be given Dagster Types to validate their output types.
The type system:
Is gradual and optional. Jobs can run without types specified explicitly, and specifying types in some places doesn't require that types be specified everywhere. Inputs and outputs default to the Any type.
Happens at op execution time - each type defines a type_check_fn that knows how to check whether values match what it expects. When a type is specified for a op's input, then the type check occurs immediately before the op is executed. When a type is specified for a op's output, then the type check occurs immediately after the op is executed.
Is complimentary to the PEP 484 Python type system. PEP 484 annotations enable static checks that verify variables and return values match particular Python types, while the Dagster type system enables runtime checks that include arbitrary validation logic.
DagsterTypes vs Python Types (mypy type-checking)#
You should not use manually-defined DagsterType objects in your python type annotations. The simple rule of thumb is to annotate op and asset bodies just as you would a regular function, and if you wish to use a DagsterType alongside your annotations (to perform more complex validation than the default runtime typechecks), you should include this information in the relevant InputDefinition or OutputDefinition.
If a Python input or output has a PEP 484 type annotation, and a DagsterType is not provided on the corresponding input or output definition, then Dagster will automatically generate a DagsterType that corresponds to the annotated Python type.
In this example, the defined op will end up with a DagsterType named "MyClass" that:
Shows up in the Dagster UI in the representation of the op.
Is checked at runtime on the value returned by the op.
Dagster offers a special type called Nothing, which is used when you need to model a dependency between ops where Dagster is passing no data along the edge. See details in the Nothing dependencies example.
Using Dagster Types with PEP 484 Type Annotations#
Dagster types peacefully coexist with Python type annotations. In this example, the inputs and outputs of the op compute function are integers, and the type check function for EvenDagsterType will be invoked at runtime to verify that they are even.