Using resources in dg projects

Assets, asset checks, and sensors in Dagster frequently require resources that are instantiated elsewhere in the project.

For example you have an asset:

# in defs/asset_one.py
from my_project.resources import AResource

import dagster as dg


@dg.asset
def asset_one(a_resource: AResource): ...

And a separately defined resource at the root of the project:

# src/resources.py
import dagster as dg


class AResource(dg.ConfigurableResource): ...

Resources can be added at any level in the defs hierarchy by creating a Definitions object.

from my_project.resources import AResource

import dagster as dg


@dg.definitions
def defs() -> dg.Definitions:
    return dg.Definitions(
        resources={"a_resource": AResource(name="foo")},
    )

Resource binding can happen at any level of the hierarchy. Which means that if you moved asset_one in this example to be in a subdirectory, you could leave the existing resources.py file at src/defs/resources.py.

src
└── resource_docs
    ├── definitions.py
    ├── defs
    │   ├── assets
    │   │   └── asset_one.py # contains def asset_one():
    │   └── resources.py # contains AResource()
    └── resources.py # contains class AResource

We have a scaffold command that makes this straightforward.

You can run

dg scaffold defs dagster.resources path/to/resources.py

which will create

import dagster as dg

@dg.definitions
def resources() -> dg.Definitions:
    return dg.Definitions(resources={})

and you can fill out the resource dictionary as needed.