Fixed a bug with load_assets_from_x functions where we began erroring when a spec and AssetsDefinition had the same key in a given module. We now only error in this case if include_specs=True.
[dagster-azure] Fixed a bug in 1.9.6 and 1.9.7 where the default behavior of the compute log manager switched from showing logs in the UI to showing a URL. You can toggle the show_url_only option to True to enable the URL showing behavior.
[dagster-dbt] Fixed an issue where group names set on partitioned dbt assets created using the @dbt_assets decorator would be ignored
ops or assets with multiple outputs that are all required and return type None/ Nothing will interpret an explicitly or implicitly returned value None to indicate that all outputs were successful.
The skip_reason argument to the constructor of SensorResult now accepts a string in addition to a SkipReason.
[dagster-k8s] Added a step_k8s_config field to k8s_job_executor that allows you to customize the raw Kubernetes config for each step in a job. See the docs for more information.
[dagster-k8s] Launched run pods now have an additional code location label.
[dagster-ui] The runs table now lets you toggle which tags are always visible.
[dagster-dbt] dagster-dbt project scaffold now creates the scaffold in multiple files:
constants.py contains a reference to your manifest and dbt project directory
assets.py contains your initial dbt assets definitions
definitions.py contains the code to load your asset definitions into the Dagster UI
schedules.py contains an optional schedule to add for your dbt assets
[dagster-dbt] Added new methods get_auto_materialize_policy and get_freshness_policy to DagsterDbtTranslator.
[dagster-fivertran] Sync options can now be passed to load_assets_from_fivetran_instance.
[dagster-wandb] W&B IO Manager now handles partitions natively. (Thanks @chrishiste!)
Previously, canceling large asset backfills would cause the daemon to time out and display a “not running” error. This has been fixed.
[dagster-ssh] Previously the SSHResource would warn when allow_host_key_change was set. Now known hosts are always loaded from the system hosts file, and the allow_host_key_change parameter is ignored.
Previously, when using AutoMaterializePolicies, partitioned assets downstream of partitioned observable source assets could be materialized before their parent partitions were observed. This has been fixed.
@graph_asset now takes a config parameter equivalent to the parameter on @graph.
Added an optional dynamic_partitions_store argument to DynamicPartitionsDefinition for multi-partition run properly with dynamic partitions (Thanks @elzzz!).
[dagster-grpahql] Added partitionsByAssets to backfillParams for ranged partition backfill (Thanks @ruizh22!).
[dagster-dbt] Support for dbt-core==1.6 has been added.
[dagster-dbt] DbtCliResource now supports configuring profiles_dir.
[dagster-k8s] Allow specifying restart_policy on k8s_job_op (Thanks @Taadas!).
[dagster-snowflake] Added authenticator to SnowflakePandasIOManager, which allows specifying the authentication mechanism to use (Thanks @pengw0048!).
[ui] The Asset > Events page now allows you to filter by event type, making it easy to hide observations.
[ui] The Asset > Partitions page checks to see if the selected partition is stale and displays stale causes in the details pane.
[ui] Hovering over "Fresh" tags now provides detailed information about why the last materialization meets the asset's freshness policy.
[ui] The global asset graph can now be filtered to display a subset of the available asset groups.
In some situations, multiple materializations of the same asset could be kicked off when using a lazy AutoMaterializePolicy with assets that had at least one source asset parent and at least one non-source asset parent. This has been fixed.
After applying an eager AutoMaterializePolicy to a time-partitioned asset downstream of an unpartitioned asset, the latest partition would only ever be materialized a single time, rather than updating in response to any parent updates. This has been fixed.
Fixed an issue that would cause the creation of a StaticPartitionsDefinition containing many thousands of partitions could take a significant amount of time.
The run coordinator daemon now uses a fresh request context on each iteration, fixing an issue where stale grpc server references could be used in certain high volume conditions.
Automatically generated data versions for partitioned assets now correctly reflect the data versions of upstream partitions. Previously, they were computed using the data versions from the most recent materializations of upstream assets regardless of partition.
[dagster-airbyte] Previously, attempting to load assets from an Airbyte instance in which some of the tables had hyphens in their name would result in an error. This has been fixed.
[dagster-dbt] Previously, attempting to load assets from a dbt project in which some of the models had hyphens in their name would result in an error. This has been fixed.
[dagstermill] Fixed a bug where known state for executing dagstermill ops was not correctly passed in (Thanks @motuzov!).
[ui] Pressing the up or down arrow key without a selection in the asset graph no longer causes a page error.
When importing a dbt project on the Dagster Cloud setup page, an Unexpected exception error would be raised when scaffolding a pull request on a repository with no profiles.yml. This behavior has been updated to raise a more descriptive error message on the repo selection page.
The running multiple agents guide has been revamped to discuss running agent replicas and zero-downtime deployment of the agent.
The agentReplicas config setting on the helm chart has been renamed to isolatedAgents. In order to use this config setting, your user code dagster version needs to be 1.4.3 or greater.
Previously, it was possible to set max_materializations_per_minute on an AutoMaterializePolicy to a non-positive number. This will now result in an error.
[dagster-dbt] When invokingdagster-dbt project scaffold on a dbt project directory, if a profiles.yml exists in the root of the directory, its contents are used to add dbt adapter packages to the scaffolded setup.py.
The default sentinel value for the multiprocessing executor’s max_concurrent field has been changed from 0 to None to more clearly signal its intent. A value of 0 is still interpreted as the sentinel value which dynamically allocates max_concurrent based on detected CPU count.
The experimental agent config setting agent_replicashas been deprecated in favor of a new name isolated_agents (agentReplicas --> isolatedAgents in the helm chart). Upgrading to the new name requires all code locations to be on 1.4.3 or greater.