Asset selection syntax
This reference contains information about the syntax for asset selections, including a variety of examples for selecting assets and their downstream and upstream dependencies.
Asset selection may be used to:
- Define a job that targets a selection of assets
- Select a set of assets to view in the Dagster UI
- Select a set of assets for an adhoc run
Syntax usage
A query includes a list of clauses. Clauses are separated by commas, except in the case of the selection
parameter of the following methods. In these cases, each clause is a separate element in a list:
define_asset_job
materialize
materialize_to_memory
Clause syntax | Description |
---|---|
ASSET_KEY | Selects a single asset by asset key. See an example. |
COMPONENT/COMPONENT | Selects an asset key with multiple components, such as a prefix, where slashes (/ ) are inserted between components. See an example. |
*ASSET_KEY | Selects an asset and all of its upstream dependencies. See an example. |
ASSET_KEY* | Selects an asset and all of its downstream dependencies. See an example. |
+ASSET_KEY | Selects an asset and one layer upstream of the asset. Including multiple + s will select that number of upstream layers from the asset. Any number of + s is supported. See an example. |
ASSET_KEY+ | Selects an asset and one layer downstream of the asset. Including multiple + s will select that number of downstream layers from the asset. Any number of + s is supported. See an example. |
Examples
The examples in this section use the following asset graph from the Dagster University Essentials project to demonstrate how to use the selection syntax:
Selecting a single asset
To select a single asset, use the asset's asset key. This example selects the taxi_zones_file
asset:
- Python
- CLI
- Dagster UI
raw_data_job = define_asset_job(name="raw_data_job", selection="taxi_zones_file")
dagster asset list --select taxi_zones_file
dagster asset materialize --select taxi_zones_file
taxi_zones_file
Which would result in the following asset graph:
Selecting assets with multiple key components
To select an asset with a key containing multiple components, such as a prefix, insert slashes (/
) between the components.
This example selects the manhattan/manhattan_stats
asset, which is defined below:
@asset(
deps=[AssetKey(["taxi_trips"]), AssetKey(["taxi_zones"])], key_prefix="manhattan"
)
def manhattan_stats(database: DuckDBResource):
...
- Python
- CLI
- Dagster UI
manhattan_job = define_asset_job(name="manhattan_job", selection="manhattan/manhattan_stats")
dagster asset list --select manhattan/manhattan_stats
dagster asset materialize --select manhattan/manhattan_stats
manhattan/manhattan_stats
Which would result in the following asset graph:
Selecting multiple assets
To select multiple assets, use a list of the assets' asset keys. The assets don't have to be dependent on each other.
This example selects the taxi_zones_file
and taxi_trips_file
assets, which are defined below:
- Python
- CLI
- Dagster UI
raw_data_job = define_asset_job(
name="taxi_zones_job", selection=["taxi_zones_file", "taxi_trips_file"]
)
When selecting multiple assets, enclose the list of asset keys in double quotes ("
) and separate each asset key with a comma:
dagster asset list --select "taxi_zones_file,taxi_trips_file"
dagster asset materialize --select "taxi_zones_file,taxi_trips_file"
taxi_zones_file taxi_trips_file
Which would result in the following asset graph:
Selecting an asset's entire lineage
To select an asset's entire lineage, add an asterisk (*
) before and after the asset key in the query.
This example selects the entire lineage for the taxi_zones
asset.
- Python
- CLI
- Dagster UI
taxi_zones_job = define_asset_job(name="taxi_zones_job", selection="*taxi_zones*")
When selecting an asset's entire lineage using the CLI, enclose the asterisk (*
) and the asset key in double quotes ("
):
dagster asset list --select "*taxi_zones*"
dagster asset materialize --select "*taxi_zones*"
*taxi_zones*
Which would result in the following asset graph:
Selecting upstream dependencies
Selecting all upstream dependencies
To select an asset and all its upstream dependencies, add an asterisk (*
) before the asset key in the query.
This example selects the manhattan_map
asset and all its upstream dependencies.
- Python
- CLI
- Dagster UI
manhattan_job = define_asset_job(name="manhattan_job", selection="*manhattan_map")
When selecting an asset's dependencies using the CLI, enclose the asterisk (*
) and the asset key in double quotes ("
):
dagster asset list --select "*manhattan_map"
dagster asset materialize --select "*manhattan_map"
*manhattan_map
Which would result in the following asset graph:
Selecting a specific number of upstream layers
To select an asset and multiple upstream layers, add a plus sign (+
) for each layer you want to select before the asset key in the query.
This example selects the manhattan_map
asset and two upstream layers.
- Python
- CLI
- Dagster UI
manhattan_job = define_asset_job(name="manhattan_job", selection="++manhattan_map")
When selecting an asset's dependencies using the CLI, enclose the plus sign (+
) and the asset key in double quotes ("
):
dagster asset list --select "++manhattan_map"
dagster asset materialize --select "++manhattan_map"
++manhattan_map
Which would result in the following asset graph:
Selecting downstream dependencies
Selecting all downstream dependencies
To select an asset and all its downstream dependencies, add an asterisk (*
) after the asset key in the query.
This example selects the taxi_zones_file
asset and all its downstream dependencies.
- Python
- CLI
- Dagster UI
taxi_zones_job = define_asset_job(name="taxi_zones_job", selection="taxi_zones_file*")
When selecting an asset's dependencies using the CLI, enclose the asterisk (*
) and the asset key in double quotes ("
):
dagster asset list --select "taxi_zones_file*"
dagster asset materialize --select "taxi_zones_file*"
taxi_zones_file*
Which would result in the following asset graph:
Selecting a specific number of downstream layers
To select an asset and multiple downstream layers, add plus sign (+
) for each layer you want to select after the asset key in the query.
This example selects the taxi_trips_file
asset and two downstream layers.
- Python
- CLI
- Dagster UI
taxi_zones_job = define_asset_job(name="taxi_zones_job", selection="taxi_zones_file++")
When selecting an asset's dependencies using the CLI, enclose the plus sign (+
) and the asset key in double quotes ("
):
dagster asset list --select "taxi_zones_file++"
dagster asset materialize --select "taxi_zones_file++"
taxi_zones_file++
Which would result in the following asset graph: