Build pipelines with AWS Lambda
note
This article focuses on using an out-of-the-box Amazon Web Services (AWS) Lambda resource. For further customization, use the open_pipes_session
instead.
This article covers how to use Dagster Pipes with Dagster's AWS Lambda integration to invoke a Lambda function and execute external code.
Dagster Pipes allows your code to interact with Dagster outside of a full Dagster environment. The environment only needs to contain dagster-pipes
, a single-file Python package with no dependencies that can be installed from PyPI or easily vendored. dagster-pipes
handles streaming stdout
/stderr
and Dagster events back to the orchestration process.