Ask AI

Changelog#

1.9.8 (core) / 0.25.8 (libraries)#

Bugfixes#

  • Fixed a bug with load_assets_from_x functions where we began erroring when a spec and AssetsDefinition had the same key in a given module. We now only error in this case if include_specs=True.
  • [dagster-azure] Fixed a bug in 1.9.6 and 1.9.7 where the default behavior of the compute log manager switched from showing logs in the UI to showing a URL. You can toggle the show_url_only option to True to enable the URL showing behavior.
  • [dagster-dbt] Fixed an issue where group names set on partitioned dbt assets created using the @dbt_assets decorator would be ignored

1.6.13 (core) / 0.22.13 (libraries)#

Bugfixes#

  • Fixed a bug where an asset with a dependency on a subset of the keys of a parent multi-asset could sometimes crash asset job construction.
  • Fixed a bug where a Definitions object containing assets having integrated asset checks and multiple partitions definitions could not be loaded.

1.6.12 (core) / 0.22.12 (libraries)#

New#

  • AssetCheckResult now has a text description property. Check evaluation descriptions are shown in the Checks tab on the asset details page.
  • Introduced TimestampMetadataValue. Timestamp metadata values are represented internally as seconds since the Unix epoch. They can be constructed using MetadataValue.timestamp. In the UI, they’re rendered in the local timezone, like other timestamps in the UI.
  • AssetSelection.checks can now accept AssetCheckKeys as well as AssetChecksDefinition.
  • [community-contribution] Metadata attached to an output at runtime (via either add_output_metadata or by passing to Output) is now available on HookContext under the op_output_metadata property. Thanks @JYoussouf!
  • [experimental] @asset, AssetSpec, and AssetOut now accept a tags property. Tags are key-value pairs meant to be used for organizing asset definitions. If "__dagster_no_value" is set as the value, only the key will be rendered in the UI. AssetSelection.tag allows selecting assets that have a particular tag.
  • [experimental] Asset tags can be used in asset CLI selections, e.g. dagster asset materialize --select tag:department=marketing
  • [experimental][dagster-dbt] Tags can now be configured on dbt assets, using DagsterDbtTranslator.get_tags. By default, we take the dbt tags configured on your dbt models, seeds, and snapshots.
  • [dagster-gcp] Added get_gcs_keys sensor helper function.

Bugfixes#

  • Fixed a bug that prevented external assets with dependencies from displaying properly in Dagster UI.
  • Fix a performance regression in loading code locations with large multi-assets.
  • [community-contribution][dagster-databricks] Fix a bug with the DatabricksJobRunner that led to an inability to use dagster-databricks with Databricks instance pools. Thanks @smats0n!
  • [community-contribution] Fixed a bug that caused a crash when external assets had hyphens in their AssetKey. Thanks @maxfirman!
  • [community-contribution] Fix a bug with load_assets_from_package_module that would cause a crash when any submodule had the same directory name as a dependency. Thanks @CSRessel!
  • [community-contribution] Fixed a mypy type error, thanks @parthshyara!
  • [community-contribution][dagster-embedded-elt] Fixed an issue where Sling assets would not properly read group and description metadata from replication config, thanks @jvyoralek!
  • [community-contribution] Ensured annotations from the helm chart properly propagate to k8s run pods, thanks @maxfirman!

Dagster Cloud#

  • Fixed an issue in Dagster Cloud Serverless runs where multiple runs simultaneously materializing the same asset would sometimes raise a “Key not found” exception.
  • Fixed an issue when using agent replicas where one replica would sporadically remove a code server created by another replica due to a race condition, leading to a “code server not found” or “Deployment not found” exception.
  • [experimental] The metadata key for specifying column schema that will be rendered prominently on the new Overview tab of the asset details page has been changed from "columns" to "dagster/column_schema". Materializations using the old metadata key will no longer result in the Columns section of the tab being filled out.
  • [ui] Fixed an Insights bug where loading a view filtered to a specific code location would not preserve that filter on pageload.

1.6.11 (core) / 0.22.11 (libraries)#

Bugfixes#

  • Fixed an issue where dagster dev or the Dagster UI would display an error when loading jobs created with op or asset selections.

1.6.10 (core) / 0.22.10 (libraries)#

New#

  • Latency improvements to the scheduler when running many simultaneous schedules.

Bugfixes#

  • The performance of loading the Definitions snapshot from a code server when large @multi_asset s are in use has been drastically improved.
  • The snowflake quickstart example project now renames the “by” column to avoid reserved snowflake names. Thanks @jcampbell!
  • The existing group name (if any) for an asset is now retained if the_asset.with_attributes is called without providing a group name. Previously, the existing group name was erroneously dropped. Thanks @ion-elgreco!
  • [dagster-dbt] Fixed an issue where Dagster events could not be streamed from dbt source freshness.
  • [dagster university] Removed redundant use of MetadataValue in Essentials course. Thanks @stianthaulow!
  • [ui] Increased the max number of plots on the asset plots page to 100.

Breaking Changes#

  • The tag_keys argument on DagsterInstance.get_run_tagsis no longer optional. This has been done to remove an easy way of accidentally executing an extremely expensive database operation.

Dagster Cloud#

  • The maximum number of concurrent runs across all branch deployments is now configurable. This setting can now be set via GraphQL or the CLI.
  • [ui] In Insights, fixed display of table rows with zero change in value from the previous time period.
  • [ui] Added deployment-level Insights.
  • [ui] Fixed an issue causing void invoices to show up as “overdue” on the billing page.
  • [experimental] Branch deployments can now indicate the new and modified assets in the branch deployment as compared to the main deployment. To enable this feature, turn on the “Enable experimental branch deployment asset graph diffing” user setting.

1.6.9 (core) / 0.22.9 (libraries)#

New#

  • [ui] When viewing logs for a run, the date for a single log row is now shown in the tooltip on the timestamp. This helps when viewing a run that takes place over more than one date.
  • Added suggestions to the error message when selecting asset keys that do not exist as an upstream asset or in an AssetSelection.
  • Improved error messages when trying to materialize a subset of a multi-asset which cannot be subset.
  • [dagster-snowflake] dagster-snowflake now requires snowflake-connector-python>=3.4.0
  • [embedded-elt] @sling_assets accepts an optional name parameter for the underlying op
  • [dagster-openai] dagster-openai library is now available.
  • [dagster-dbt] Added a new setting on DagsterDbtTranslatorSettings called enable_duplicate_source_asset_keys that allows users to set duplicate asset keys for their dbt sources. Thanks @hello-world-bfree!
  • Log messages in the Dagster daemon for unloadable sensors and schedules have been removed.
  • [ui] Search now uses a cache that persists across pageloads which should greatly improve search performance for very large orgs.
  • [ui] groups/code locations in the asset graph’s sidebar are now sorted alphabetically.

Bugfixes#

  • Fixed issue where the input/output schemas of configurable IOManagers could be ignored when providing explicit input / output run config.
  • Fixed an issue where enum values could not properly have a default value set in a ConfigurableResource.
  • Fixed an issue where graph-backed assets would sometimes lose user-provided descriptions due to a bug in internal copying.
  • [auto-materialize] Fixed an issue introduced in 1.6.7 where updates to ExternalAssets would be ignored when using AutoMaterializePolicies which depended on parent updates.
  • [asset checks] Fixed a bug with asset checks in step launchers.
  • [embedded-elt] Fix a bug when creating a SlingConnectionResource where a blank keyword argument would be emitted as an environment variable
  • [dagster-dbt] Fixed a bug where emitting events from dbt source freshness would cause an error.
  • [ui] Fixed a bug where using the “Terminate all runs” button with filters selected would not apply the filters to the action.
  • [ui] Fixed an issue where typing a search query into the search box before the search data was fetched would yield “No results” even after the data was fetched.

Community Contributions#

  • [docs] fixed typo in embedded-elt.mdx (thanks @cameronmartin)!
  • [dagster-databricks] log the url for the run of a databricks job (thanks @smats0n)!
  • Fix missing partition property (thanks christeefy)!
  • Add op_tags to @observable_source_asset decorator (thanks @maxfirman)!
  • [docs] typo in MultiPartitionMapping docs (thanks @dschafer)
  • Allow github actions to checkout branch from forked repo for docs changes (ci fix) (thanks hainenber)!

Experimental#

  • [asset checks] UI performance of asset checks related pages has been improved.
  • [dagster-dbt] The class DbtArtifacts has been added for managing the behavior of rebuilding the manifest during development but expecting a pre-built one in production.

Documentation#

  • Added example of writing compute logs to AWS S3 when customizing agent configuration.
  • "Hello, Dagster" is now "Dagster Quickstart" with the option to use a Github Codespace to explore Dagster.
  • Improved guides and reference to better running multiple isolated agents with separate queues on ECS.

Dagster Cloud#

  • Microsoft Teams is now supported for alerts. Documentation
  • A send sample alert button now exists on both the alert policies page and in the alert policies editor to make it easier to debug and configure alerts without having to wait for an event to kick them off.