Learn about Dagster's core concepts and how to use them in your data platform.
An asset is an object in persistent storage, such as a table, file, or persisted machine learning model. A Software-defined Asset is a Dagster object that couples an asset to the function and upstream assets used to produce its contents.
Schedules launch runs on a fixed interval, while sensors allow you to do so based on an external state change.
A software-defined asset or job can represent a collection of partitions that can be tracked and executed independently.
Resources enable you to separate logic from external dependencies, making developing and testing possible in multiple environments.
Additionally, Dagster provides a configuration system that allows you to document, schematize, and error-check your configuration.
A code location is a collection of Dagster definitions, including assets, jobs, schedules, sensors, and resources. Dagster tools like the Dagster webserver/UI and CLI use code locations to load your code.
The Dagster UI is a web-based interface for viewing and interacting with Dagster objects.
Dagster enables you to build testable and maintainable data applications. It provides ways to allow you unit-test your data applications, separate business logic from environments, and set explicit expectations on uncontrollable inputs.
Ops typically perform relatively simple tasks, such as executing a database query or sending a Slack message.
An op graph is a set of interconnected ops or sub-graphs. While individual ops typically perform simple tasks, ops can be assembled into a graph or job to accomplish complex tasks.
Learn to execute op and asset jobs, and apply metadata and tags.
I/O managers are user-provided objects that store asset and op outputs and load them as inputs to downstream assets and ops.
The Dagster Type system provides gradual, opt-in typing for the inputs and outputs of assets and ops.
A rich, extensible logging system, Dagster's built-in logger tracks all execution events. Loggers can also be customized to fit your infrastructure.
The GraphQL API allows you to interact programmatically with Dagster.