dbt Examples

You can find the code for this example on Github.

Here provides examples of how to use dagster-dbt to integrate your existing dbt project with the Dagster platform.

These examples illustrate how custom solids can be combined with dbt commands in a Dagster pipeline.

Requirements

  1. Make sure you have the necessary Python libraries installed. Running inside a Python virtualenv is recommended.

    pip install -e .

  2. The example dbt project requires a running Postgres database. By default, the project will attempt to connect to postgresql://dbt_example:dbt_example@localhost:5432/dbt_example. If you are running Docker, have Docker Compose installed, and aren't running any other process bound to port 5432, you can bring up a default database with this address by running:

    docker-compose up -d

    If you'd like to run the project against a different running database, set the environment variables DAGSTER_DBT_EXAMPLE_PGHOST, DAGSTER_DBT_EXAMPLE_PGPORT, DAGSTER_DBT_EXAMPLE_PGUSER, and DAGSTER_DBT_EXAMPLE_PGPASSWORD as appropriate.

  3. Add the profile for the 'dbt_example_project' to your dbt 'profiles.yml' file.

    mkdir -p ~/.dbt/
touch ~/.dbt/profiles.yml
cat dbt_example_project/profiles.yml >> ~/.dbt/profiles.yml

    Test that this is correctly setup by running dbt ls:

    dbt ls --project-dir dbt_example_project

  4. Run Dagit

    $ dagit -m dbt_example