Loggers

Built-in loggers

dagster._loggers.colored_console_logger

Core class for defining loggers.

Loggers are job-scoped logging handlers, which will be automatically invoked whenever dagster messages are logged from within a job.

Parameters:

• logger_fn (Callable[[InitLoggerContext], logging.Logger]) – User-provided function to instantiate the logger. This logger will be automatically invoked whenever the methods on context.log are called from within job compute logic.
• config_schema (Optional[ConfigSchema]) – The schema for the config. Configuration data available in init_context.logger_config. If not set, Dagster will accept any config provided.
• description (Optional[str]) – A human-readable description of this logger.

dagster._loggers.json_console_logger

Core class for defining loggers.

Loggers are job-scoped logging handlers, which will be automatically invoked whenever dagster messages are logged from within a job.

Parameters:

• logger_fn (Callable[[InitLoggerContext], logging.Logger]) – User-provided function to instantiate the logger. This logger will be automatically invoked whenever the methods on context.log are called from within job compute logic.
• config_schema (Optional[ConfigSchema]) – The schema for the config. Configuration data available in init_context.logger_config. If not set, Dagster will accept any config provided.
• description (Optional[str]) – A human-readable description of this logger.

Logging from an @op

class dagster.DagsterLogManager

Centralized dispatch for logging from user code.

Handles the construction of uniform structured log messages and passes them through to the underlying loggers/handlers.

An instance of the log manager is made available to ops as context.log. Users should not initialize instances of the log manager directly. To configure custom loggers, set the logger_defs argument in an @job decorator or when calling the to_job() method on a GraphDefinition.

The log manager inherits standard convenience methods like those exposed by the Python standard library python:logging module (i.e., within the body of an op, context.log.\{debug, info, warning, warn, error, critical, fatal}).

The underlying integer API can also be called directly using, e.g. context.log.log(5, msg), and the log manager will delegate to the log method defined on each of the loggers it manages.

User-defined custom log levels are not supported, and calls to, e.g., context.log.trace or context.log.notice will result in hard exceptions at runtime.

Defining custom loggers

@dagster.logger

Define a logger.

The decorated function should accept an InitLoggerContext and return an instance of python:logging.Logger. This function will become the logger_fn of an underlying LoggerDefinition.

Parameters:

• config_schema (Optional[ConfigSchema]) – The schema for the config. Configuration data available in init_context.logger_config. If not set, Dagster will accept any config provided.
• description (Optional[str]) – A human-readable description of the logger.

class dagster.LoggerDefinition

Core class for defining loggers.

Loggers are job-scoped logging handlers, which will be automatically invoked whenever dagster messages are logged from within a job.

Parameters:

• logger_fn (Callable[[InitLoggerContext], logging.Logger]) – User-provided function to instantiate the logger. This logger will be automatically invoked whenever the methods on context.log are called from within job compute logic.
• config_schema (Optional[ConfigSchema]) – The schema for the config. Configuration data available in init_context.logger_config. If not set, Dagster will accept any config provided.
• description (Optional[str]) – A human-readable description of this logger.

property config_schema

The schema for the logger’s config. Configuration data available in init_context.logger_config.

Type: Any

property description

A human-readable description of the logger.

Type: Optional[str]

property logger_fn

The function that will be invoked to instantiate the logger.

Type: Callable[[InitLoggerContext], logging.Logger]

class dagster.InitLoggerContext

The context object available as the argument to the initialization function of a dagster.LoggerDefinition.

Users should not instantiate this object directly. To construct an InitLoggerContext for testing purposes, use dagster. build_init_logger_context().

Example:

from dagster import logger, InitLoggerContext

@logger
def hello_world(init_context: InitLoggerContext):
    ...

property logger_config

The configuration data provided by the run config. The schema for this data is defined by config_schema on the LoggerDefinition.

property logger_def

The logger definition for the logger being constructed.

property run_id

The ID for this run of the job.

dagster.build_init_logger_context

Builds logger initialization context from provided parameters.

This function can be used to provide the context argument to the invocation of a logger definition.

Note that you may only specify one of pipeline_def and job_def.

Parameters:

• logger_config (Any) – The config to provide during initialization of logger.
• job_def (Optional[JobDefinition]) – The job definition that the logger will be used with.

Examples:

context = build_init_logger_context()
logger_to_init(context)