DagsterDocs

Snowflake (dagster-snowflake)

This library provides an integration with the Snowflake data warehouse.

Presently, it provides a snowflake_resource, which is a Dagster resource for configuring Snowflake connections and issuing queries.

To use this library, you should first ensure that you have an appropriate Snowflake user configured to access your data warehouse.

dagster_snowflake.snowflake_resource ResourceDefinition[source]

A resource for connecting to the Snowflake data warehouse.

A simple example of loading data into Snowflake and subsequently querying that data is shown below:

Examples:

from dagster import execute_pipeline, pipeline, DependencyDefinition, ModeDefinition
from dagster_snowflake import snowflake_resource

@op(required_resource_keys={'snowflake'})
def get_one(context):
    context.resources.snowflake.execute_query('SELECT 1')

@graph
def my_snowflake_graph():
    get_one()

my_snowflake_graph.to_job(
    resources={'snowflake': snowflake_resource}
).execute_in_process(
    run_config={
        'resources': {
            'snowflake': {
                'config': {
                    'account': {'env': 'SNOWFLAKE_ACCOUNT'},
                    'user': {'env': 'SNOWFLAKE_USER'},
                    'password': {'env': 'SNOWFLAKE_PASSWORD'},
                    'database': {'env': 'SNOWFLAKE_DATABASE'},
                    'schema': {'env': 'SNOWFLAKE_SCHEMA'},
                    'warehouse': {'env': 'SNOWFLAKE_WAREHOUSE'},
                }
            }
        }
    }
)
class dagster_snowflake.SnowflakeConnection(context)[source]
execute_queries(sql_queries, parameters=None, fetch_results=False)[source]
execute_query(sql, parameters=None, fetch_results=False)[source]
get_connection(raw_conn=True)[source]
load_table_from_local_parquet(src, table)[source]