load_assets_from_x functions where we began erroring when a spec and AssetsDefinition had the same key in a given module. We now only error in this case if include_specs=True.show_url_only option to True to enable the URL showing behavior.@dbt_assets decorator would be ignored--live-data-poll-rate that allows configuring how often the UI polls for new asset data when viewing the asset graph, asset catalog, or overview assets page. It defaults to 2000 ms.report_asset_materialization REST API endpoint for creating external asset materialization events. This is available in cloud as well.@dbt_assets decorator now accepts a backfill_policy argument, for controlling how the assets are backfilled.@dbt_assets decorator now accepts a op_tags argument, for passing tags to the op underlying the produced AssetsDefinition.get_materialize_result & get_asset_check_result to PipesClientCompletedInvocationacryl-datahub pin in the dagster-datahub package has been removed.PipesDatabricksClient now performs stdout/stderr forwarding from the Databricks master node to Dagster.dagster-dbt-cloud CLI.K8sRunLauncher. See the docs for more information.Previously, the asset backfill page would display negative counts if failed partitions were manually re-executed. This has been fixed.
Fixed an issue where the run list dialog for viewing the runs occupying global op concurrency slots did not expand to fit the content size.
Fixed an issue where selecting a partition would clear the launchpad and typing in the launchpad would clear the partition selection
Fixed various issues with the asset-graph displaying the wrong graph
The IO manager’s handle_output method is no longer invoked when observing an observable source asset.
[ui] Fixed an issue where the run config dialog could not be scrolled.
[pipes] Fixed an issue in the PipesDockerClient with parsing logs fetched via the docker client.
[external assets] Fixed an issue in external_assets_from_specs where providing multiple specs would error
[external assets] Correct copy in tooltip to explain why Materialize button is disabled on an external asset.
dagster-pipes version used in the external process.is_dagster_pipes_process has been removed from the dagster-pipes package.EnvVar utility will now raise an exception if it is used outside of the context of a Dagster resource or config class. The get_value() utility will retrieve the value outside of this context.DbtCliResource, an explicit target_path can now be specified.DagsterDbtTranslator and DagsterDbtTranslatorSettings: see the docs for more information.Improved ergonomics for execution dependencies in assets - We introduced a set of APIs to simplify working with Dagster that don't use the I/O manager system for handling data between assets. I/O manager workflows will not be affected.
AssetDep type allows you to specify upstream dependencies with partition mappings when using the deps parameter of @asset and AssetSpec.MaterializeResult can be optionally returned from an asset to report metadata about the asset when the asset handles any storage requirements within the function body and does not use an I/O manager.AssetSpec has been added as a new way to declare the assets produced by @multi_asset. When using AssetSpec, the multi_asset does not need to return any values to be stored by the I/O manager. Instead, the multi_asset should handle any storage requirements in the body of the function.Asset checks (experimental) - You can now define, execute, and monitor data quality checks in Dagster [docs].
@asset_check decorator, as well as the check_specs argument to @asset and @multi_asset enable defining asset checks.Auto materialize customization (experimental) - AutoMaterializePolicies can now be customized [docs].
AutoMaterializeRules which determine if an asset should be materialized or skipped.PipesSubprocessClient, PipesDocketClient, PipesK8sClient, PipesDatabricksClientopen_pipes_session. One can augment existing invocations rather than replacing them wholesale.AssetExecutionContext is now a subclass of OpExecutionContext, not a type alias. The codedef my_helper_function(context: AssetExecutionContext): ... @op def my_op(context: OpExecutionContext): my_helper_function(context)
will cause type checking errors. To migrate, update type hints to respect the new subclassing.
AssetExecutionContext cannot be used as the type annotation for @ops run in @jobs. To migrate, update the type hint in @op to OpExecutionContext. @ops that are used in @graph_assets may still use the AssetExecutionContext type hint.# old @op def my_op(context: AssetExecutionContext): ... # correct @op def my_op(context: OpExecutionContext): ...
backfill_policy=BackfillPolicy.single_run() to your assets.has_dynamic_partition implementation has been optimized. Thanks @edvardlindelof!stream_to_asset_map argument to build_airbyte_assets to support the Airbyte prefix setting with special characters. Thanks @chollinger93!DatabricksPysparkStepLauncher fails to get logs when job_run doesn’t have cluster_id at root level. Thanks @PadenZach!dagster_cloud.dagster_insights module that contains utilities for capturing and submitting external metrics about data operations to Dagster Cloud via an api. Dagster Cloud Insights is a soon-to-be released feature that shows improves visibility into usage and cost metrics such as run duration and Snowflake credits in the Cloud UI.DbtCliResource now enforces that the current installed version of dbt-core is at least version 1.4.0.DbtCliResource now properly respects DBT_TARGET_PATH if it is set by the user. Artifacts from dbt invocations using DbtCliResource will now be placed in unique subdirectories of DBT_TARGET_PATH.partition_time_window attribute on OpExecutionContext and AssetExecutionContext now returns the time range, instead of raising an error.job_name property on the result object of build_hook_context.AssetSpec has been added as a new way to declare the assets produced by @multi_asset.AssetDep type allows you to specify upstream dependencies with partition mappings when using the deps parameter of @asset and AssetSpec.report_asset_check method added to ExtContext.yield from to forward reported materializations and asset check results to Dagster. Results reported from ext that are not yielded will raise an error.os.getenv() versus Dagster’s EnvVar.