Ask AI

Changelog#

0.6.3#

  • Adds a type_check parameter to PythonObjectType, as_dagster_type, and @as_dagster_type to enable custom type checks in place of default isinstance checks. See documentation here: https://dagster.readthedocs.io/en/latest/sections/learn/tutorial/types.html#custom-type-checks

  • Improved the type inference experience by automatically wrapping bare python types as dagster types.

  • Reworked our tutorial (now with more compelling/scary breakfast cereal examples) and public API documentation. See the new tutorial here: https://dagster.readthedocs.io/en/latest/sections/learn/tutorial/index.html

  • New solids explorer in Dagit allows you to browse and search for solids used across the repository.

  • Enabled solid dependency selection in the Dagit search filter.

    • To select a solid and its upstream dependencies, search +{solid_name}.
    • To select a solid and its downstream dependents, search {solid_name}+.
    • For both search +{solid_name}+.

  • Added a terminate button in Dagit to terminate an active run.

  • Added an --output flag to dagster-graphql CLI.

  • Added confirmation step for dagster run wipe and dagster schedule wipe commands (Thanks @shahvineet98).

  • Fixed a wrong title in the dagster-snowflake library README (Thanks @Step2Web).

0.6.2#

  • Changed composition functions @pipeline and @composite_solid to automatically give solids aliases with an incrementing integer suffix when there are conflicts. This removes to the need to manually alias solid definitions that are used multiple times.
  • Add dagster schedule wipe command to delete all schedules and remove all schedule cron jobs
  • execute_solid test util now works on composite solids.
  • Docs and example improvements: https://dagster.readthedocs.io/
  • Added --remote flag to dagster-graphql for querying remote Dagit servers.
  • Fixed issue with duplicate run tag autocomplete suggestions in Dagit (#1839)
  • Fixed Windows 10 / py3.6+ bug causing pipeline execution failures

0.6.1#

  • Fixed an issue where Dagster public images tagged latest on Docker Hub were erroneously published with an older version of Dagster (#1814)
  • Fixed an issue where the most recent scheduled run was not displayed in Dagit (#1815)
  • Fixed a bug with the dagster schedule start --start-all command (#1812)
  • Added a new scheduler command to restart a schedule: dagster schedule restart. Also added a flag to restart all running schedules: dagster schedule restart --restart-all-running.

0.6.0 "Impossible Princess"#

New

This major release includes features for scheduling, operating, and executing pipelines that elevate Dagit and dagster from a local development tool to a deployable service.

  • DagsterInstance introduced as centralized system to control run, event, compute log, and local intermediates storage.
  • A Scheduler abstraction has been introduced along side an initial implementation of SystemCronScheduler in dagster-cron.
  • dagster-aws has been extended with a CLI for deploying dagster to AWS. This can spin up a Dagit node and all the supporting infrastructure—security group, RDS PostgreSQL instance, etc.—without having to touch the AWS console, and for deploying your code to that instance.
  • Dagit
    • Runs: a completely overhauled Runs history page. Includes the ability to Retry, Cancel, and Delete pipeline runs from the new runs page.
    • Scheduler: a page for viewing and interacting with schedules.
    • Compute Logs: stdout and stderr are now viewable on a per execution step basis in each run. This is available in real time for currently executing runs and for historical runs.
    • A Reload button in the top right in Dagit restarts the web-server process and updates the UI to reflect repo changes, including DAG structure, solid names, type names, etc. This replaces the previous file system watching behavior.

Breaking Changes

  • --log and --log-dir no longer supported as CLI args. Existing runs and events stored via these flags are no longer compatible with current storage.
  • raise_on_error moved from in process executor config to argument to arguments in python API methods such as execute_pipeline

0.5.9#

  • Fixes an issue using custom types for fan-in dependencies with intermediate storage.