Census (dagster-census)

This library provides an integration with Census.

dagster_census.census_trigger_sync_op OpDefinition

Executes a Census sync for a given sync_id and polls until that sync completes, raising an error if it is unsuccessful.

It outputs a CensusOutput which contains the details of the Census sync after it successfully completes.

It requires the use of the census_resource, which allows it to communicate with the Census API.

Examples:

from dagster import job
from dagster_census import census_resource, census_sync_op

my_census_resource = census_resource.configured(
    {
        "api_key": {"env": "CENSUS_API_KEY"},
    }
)

sync_foobar = census_sync_op.configured({"sync_id": "foobar"}, name="sync_foobar")

@job(resource_defs={"census": my_census_resource})
def my_simple_census_job():
    sync_foobar()

dagster_census.census_resource ResourceDefinition

This resource allows users to programatically interface with the Census REST API to launch syncs and monitor their progress. This currently implements only a subset of the functionality exposed by the API.

Examples:

from dagster import job
from dagster_census import census_resource

my_census_resource = census_resource.configured(
    {
        "api_key": {"env": "CENSUS_API_KEY"},
    }
)

@job(resource_defs={"census":my_census_resource})
def my_census_job():
    ...

class dagster_census.CensusResource: This class exposes methods on top of the Census REST API.

class dagster_census.CensusOutput

Contains recorded information about the state of a Census sync after a sync completes.

Parameters:

sync_run (Dict[str, Any]) – The details of the specific sync run.
source (Dict[str, Any]) – Information about the source for the Census sync.
destination (Dict[str, Any]) – Information about the destination for the Census sync.