Schedules

@dagster.schedule(cron_schedule, pipeline_name, name=None, tags=None, tags_fn=None, solid_selection=None, mode='default', should_execute=None, environment_vars=None, execution_timezone=None)[source]

Create a schedule.

The decorated function will be called as the run_config_fn of the underlying ScheduleDefinition and should take a ScheduleExecutionContext as its only argument, returning the environment dict for the scheduled execution.

Parameters
  • cron_schedule (str) – A valid cron string specifying when the schedule will run, e.g., '45 23 * * 6' for a schedule that runs at 11:45 PM every Saturday.

  • pipeline_name (str) – The name of the pipeline to execute when the schedule runs.

  • name (Optional[str]) – The name of the schedule to create.

  • tags (Optional[Dict[str, str]]) – A dictionary of tags (string key-value pairs) to attach to the scheduled runs.

  • tags_fn (Optional[Callable[[ScheduleExecutionContext], Optional[Dict[str, str]]]]) – A function that generates tags to attach to the schedules runs. Takes a ScheduleExecutionContext and returns a dictionary of tags (string key-value pairs). You may set only one of tags and tags_fn.

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute when the schedule runs. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The pipeline mode in which to execute this schedule. (Default: ‘default’)

  • should_execute (Optional[Callable[[ScheduleExecutionContext], bool]]) – A function that runs at schedule execution tie to determine whether a schedule should execute or skip. Takes a ScheduleExecutionContext and returns a boolean (True if the schedule should execute). Defaults to a function that always returns True.

  • environment_vars (Optional[Dict[str, str]]) – Any environment variables to set when executing the schedule.

  • execution_timezone (Optional[str]) – Timezone in which the schedule should run. Only works with DagsterCommandLineScheduler, and must be set when using that scheduler.

@dagster.monthly_schedule(pipeline_name, start_date, name=None, execution_day_of_month=1, execution_time=datetime.time(0, 0), tags_fn_for_date=None, solid_selection=None, mode='default', should_execute=None, environment_vars=None, end_date=None, execution_timezone=None)[source]

Create a schedule that runs monthly.

The decorated function will be called as the run_config_fn of the underlying ScheduleDefinition and should take a ScheduleExecutionContext as its only argument, returning the environment dict for the scheduled execution.

Parameters
  • pipeline_name (str) – The name of the pipeline to execute when the schedule runs.

  • start_date (datetime.datetime) – The date from which to run the schedule.

  • name (Optional[str]) – The name of the schedule to create.

  • execution_day_of_month (int) – The day of the month on which to run the schedule (must be between 0 and 31).

  • execution_time (datetime.time) – The time at which to execute the schedule.

  • tags_fn_for_date (Optional[Callable[[datetime.datetime], Optional[Dict[str, str]]]]) – A function that generates tags to attach to the schedules runs. Takes the date of the schedule run and returns a dictionary of tags (string key-value pairs).

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute when the schedule runs. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The pipeline mode in which to execute this schedule. (Default: ‘default’)

  • should_execute (Optional[Callable[ScheduleExecutionContext, bool]]) – A function that runs at schedule execution tie to determine whether a schedule should execute or skip. Takes a ScheduleExecutionContext and returns a boolean (True if the schedule should execute). Defaults to a function that always returns True.

  • environment_vars (Optional[Dict[str, str]]) – Any environment variables to set when executing the schedule.

  • end_date (Optional[datetime.datetime]) – The last time to run the schedule to, defaults to current time.

  • execution_timezone (Optional[str]) – Timezone in which the schedule should run. Only works with DagsterCommandLineScheduler, and must be set when using that scheduler.

@dagster.weekly_schedule(pipeline_name, start_date, name=None, execution_day_of_week=0, execution_time=datetime.time(0, 0), tags_fn_for_date=None, solid_selection=None, mode='default', should_execute=None, environment_vars=None, end_date=None, execution_timezone=None)[source]

Create a schedule that runs weekly.

The decorated function will be called as the run_config_fn of the underlying ScheduleDefinition and should take a ScheduleExecutionContext as its only argument, returning the environment dict for the scheduled execution.

Parameters
  • pipeline_name (str) – The name of the pipeline to execute when the schedule runs.

  • start_date (datetime.datetime) – The date from which to run the schedule.

  • name (Optional[str]) – The name of the schedule to create.

  • execution_day_of_week (int) – The day of the week on which to run the schedule. Must be between 0 (Monday) and 6 (Sunday).

  • execution_time (datetime.time) – The time at which to execute the schedule.

  • tags_fn_for_date (Optional[Callable[[datetime.datetime], Optional[Dict[str, str]]]]) – A function that generates tags to attach to the schedules runs. Takes the date of the schedule run and returns a dictionary of tags (string key-value pairs).

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute when the schedule runs. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The pipeline mode in which to execute this schedule. (Default: ‘default’)

  • should_execute (Optional[Callable[ScheduleExecutionContext, bool]]) – A function that runs at schedule execution tie to determine whether a schedule should execute or skip. Takes a ScheduleExecutionContext and returns a boolean (True if the schedule should execute). Defaults to a function that always returns True.

  • environment_vars (Optional[Dict[str, str]]) – Any environment variables to set when executing the schedule.

  • end_date (Optional[datetime.datetime]) – The last time to run the schedule to, defaults to current time.

  • execution_timezone (Optional[str]) – Timezone in which the schedule should run. Only works with DagsterCommandLineScheduler, and must be set when using that scheduler.

@dagster.hourly_schedule(pipeline_name, start_date, name=None, execution_time=datetime.time(0, 0), tags_fn_for_date=None, solid_selection=None, mode='default', should_execute=None, environment_vars=None, end_date=None, execution_timezone=None)[source]

Create a schedule that runs hourly.

The decorated function will be called as the run_config_fn of the underlying ScheduleDefinition and should take a ScheduleExecutionContext as its only argument, returning the environment dict for the scheduled execution.

Parameters
  • pipeline_name (str) – The name of the pipeline to execute when the schedule runs.

  • start_date (datetime.datetime) – The date from which to run the schedule.

  • name (Optional[str]) – The name of the schedule to create. By default, this will be the name of the decorated function.

  • execution_time (datetime.time) – The time at which to execute the schedule. Only the minutes component will be respected – the hour should be 0, and will be ignored if it is not 0.

  • tags_fn_for_date (Optional[Callable[[datetime.datetime], Optional[Dict[str, str]]]]) – A function that generates tags to attach to the schedules runs. Takes the date of the schedule run and returns a dictionary of tags (string key-value pairs).

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute when the schedule runs. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The pipeline mode in which to execute this schedule. (Default: ‘default’)

  • should_execute (Optional[Callable[ScheduleExecutionContext, bool]]) – A function that runs at schedule execution tie to determine whether a schedule should execute or skip. Takes a ScheduleExecutionContext and returns a boolean (True if the schedule should execute). Defaults to a function that always returns True.

  • environment_vars (Optional[Dict[str, str]]) – Any environment variables to set when executing the schedule.

  • end_date (Optional[datetime.datetime]) – The last time to run the schedule to, defaults to current time.

  • execution_timezone (Optional[str]) – Timezone in which the schedule should run. Only works with DagsterCommandLineScheduler, and must be set when using that scheduler.

@dagster.daily_schedule(pipeline_name, start_date, name=None, execution_time=datetime.time(0, 0), tags_fn_for_date=None, solid_selection=None, mode='default', should_execute=None, environment_vars=None, end_date=None, execution_timezone=None)[source]

Create a schedule that runs daily.

The decorated function will be called as the run_config_fn of the underlying ScheduleDefinition and should take a ScheduleExecutionContext as its only argument, returning the environment dict for the scheduled execution.

Parameters
  • pipeline_name (str) – The name of the pipeline to execute when the schedule runs.

  • start_date (datetime.datetime) – The date from which to run the schedule.

  • name (Optional[str]) – The name of the schedule to create.

  • execution_time (datetime.time) – The time at which to execute the schedule.

  • tags_fn_for_date (Optional[Callable[[datetime.datetime], Optional[Dict[str, str]]]]) – A function that generates tags to attach to the schedules runs. Takes the date of the schedule run and returns a dictionary of tags (string key-value pairs).

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute when the schedule runs. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The pipeline mode in which to execute this schedule. (Default: ‘default’)

  • should_execute (Optional[Callable[ScheduleExecutionContext, bool]]) – A function that runs at schedule execution tie to determine whether a schedule should execute or skip. Takes a ScheduleExecutionContext and returns a boolean (True if the schedule should execute). Defaults to a function that always returns True.

  • environment_vars (Optional[Dict[str, str]]) – Any environment variables to set when executing the schedule.

  • end_date (Optional[datetime.datetime]) – The last time to run the schedule to, defaults to current time.

  • execution_timezone (Optional[str]) – Timezone in which the schedule should run. Only works with DagsterCommandLineScheduler, and must be set when using that scheduler.

class dagster.ScheduleDefinition(name, cron_schedule, pipeline_name, run_config=None, run_config_fn=None, tags=None, tags_fn=None, solid_selection=None, mode='default', should_execute=None, environment_vars=None, execution_timezone=None)[source]

Define a schedule that targets a pipeline

Parameters
  • name (str) – The name of the schedule to create.

  • cron_schedule (str) – A valid cron string specifying when the schedule will run, e.g., ‘45 23 * * 6’ for a schedule that runs at 11:45 PM every Saturday.

  • pipeline_name (str) – The name of the pipeline to execute when the schedule runs.

  • run_config (Optional[Dict]) – The environment config that parameterizes this execution, as a dict.

  • run_config_fn (Callable[[ScheduleExecutionContext], [Dict]]) – A function that takes a ScheduleExecutionContext object and returns the environment configuration that parameterizes this execution, as a dict. You may set only one of run_config and run_config_fn.

  • tags (Optional[Dict[str, str]]) – A dictionary of tags (string key-value pairs) to attach to the scheduled runs.

  • tags_fn (Optional[Callable[[ScheduleExecutionContext], Optional[Dict[str, str]]]]) – A function that generates tags to attach to the schedules runs. Takes a ScheduleExecutionContext and returns a dictionary of tags (string key-value pairs). You may set only one of tags and tags_fn.

  • solid_selection (Optional[List[str]]) – A list of solid subselection (including single solid names) to execute when the schedule runs. e.g. ['*some_solid+', 'other_solid']

  • mode (Optional[str]) – The mode to apply when executing this schedule. (default: ‘default’)

  • should_execute (Optional[Callable[[ScheduleExecutionContext], bool]]) – A function that runs at schedule execution time to determine whether a schedule should execute or skip. Takes a ScheduleExecutionContext and returns a boolean (True if the schedule should execute). Defaults to a function that always returns True.

  • environment_vars (Optional[dict[str, str]]) – The environment variables to set for the schedule

  • execution_timezone (Optional[str]) – Timezone in which the schedule should run. Only works with DagsterCommandLineScheduler, and must be set when using that scheduler.

class dagster.ScheduleExecutionContext[source]

Schedule-specific execution context.

An instance of this class is made available as the first argument to various ScheduleDefinition functions. It is passed as the first argument to run_config_fn, tags_fn, and should_execute.

instance

The instance configured to run the schedule

Type

DagsterInstance

scheduled_execution_time

The time in which the execution was scheduled to happen. May differ slightly from both the actual execution time and the time at which the run config is computed.

Type

datetime