Declarative Automation

Declarative Automation is a framework that allows you to access information about events that impact the status of your assets, and the dependencies between them, in order to:

• Ensure you're working with the most up-to-date data.
• Optimize resource usage by only materializing assets or executing checks when needed.
• Precisely define when specific assets should be updated based on the state of other assets.

Declarative Automation has two components:

• An automation condition, set on an asset or asset check, which represents when an individual asset or check should be executed.
• An automation condition sensor, which evaluates automation conditions and launches runs in response to their statuses.

Using Declarative Automation

To use Declarative Automation, you must:

• Set automation conditions on assets or asset checks in your code.
• Enable the automation condition sensor in the Dagster UI:
  1. Navigate to Overview > Sensors.
  2. Locate the desired code location.
  3. Toggle on the default_automation_condition_sensor sensor.

Automation conditions

An AutomationCondition on an asset or asset check describe the conditions under which work should be executed.

Dagster provides a few pre-built automation conditions to handle common use cases:

Name	Condition	Useful for
AutomationCondition.on_cron(cron_schedule)	This condition will materialize an asset on a provided cron_schedule, after all of its parents have been updated.	Regularly updating an asset without worrying about the specifics of how its parents update.
AutomationCondition.on_missing()	This condition will materialize an asset if all its dependencies have been updated, but the asset itself has not.	Filling in partitioned assets as soon as upstream data is available.
AutomationCondition.eager()	This condition will materialize an asset: If the asset has never been materialized before, or When the asset's parents update, as long as none of the parents are currently missing or have an update in progress.	Automatically propagating changes through the asset graph. Ensuring assets remain up to date.

Setting automation conditions on assets and asset checks

You can set automation conditions on the @dg.asset decorator or on an AssetSpec object:

import dagster as dg

@dg.asset(automation_condition=dg.AutomationCondition.eager())
def my_eager_asset(): ...

AssetSpec("my_cron_asset", automation_condition=AutomationCondition.on_cron("@daily"))

You can also set automation conditions on the @dg.asset_check decorator or on an AssetCheckSpec object:

@dg.asset_check(asset=dg.AssetKey("orders"), automation_condition=dg.AutomationCondition.on_cron("@daily"))
def my_eager_check() -> dg.AssetCheckResult:
    return dg.AssetCheckResult(passed=True)


dg.AssetCheckSpec(
    "my_cron_check",
    asset=dg.AssetKey("orders"),
    automation_condition=dg.AutomationCondition.on_cron("@daily"),
)

Customizing automation conditions

If the pre-built automation conditions don't fit your needs, you can build your own. For more information, see "Customizing automation conditions".

Automation condition sensors

The default_automation_conditition_sensor monitors all assets and asset checks in a code location in which it is enabled. When automation conditions for an asset or asset check in that code location are met, the sensor will execute a run in response.

The sensor's evaluation history will be visible in the UI:

You can also view a detailed history of each asset's evaluations on the asset's Asset Details page. This allows you to see why an asset was or wasn't materialized at different points in time:

To use multiple sensors or change the properties of the default sensor, see the AutomationConditionSensorDefinition API documentation.