Skip to main content

Dagster & GCP BigQuery

warning

This feature is considered in a beta stage. It is still being tested and may change.

The Google Cloud Platform BigQuery integration allows data engineers to easily query and store data in the BigQuery data warehouse through the use of the BigQueryResource.

Installation​

pip install dagster-gcp

Examples​

from dagster_gcp import BigQueryResource

import dagster as dg


@dg.asset
def my_table(bigquery: BigQueryResource):
    with bigquery.get_client() as client:
        client.query("SELECT * FROM my_dataset.my_table")


defs = dg.Definitions(
    assets=[my_table], resources={"bigquery": BigQueryResource(project="my-project")}
)

About Google Cloud Platform BigQuery​

The Google Cloud Platform BigQuery service, offers a fully managed enterprise data warehouse that enables fast SQL queries using the processing power of Google's infrastructure.