GraphQL API
Dagster exposes a GraphQL API that allows clients to programmatically interact with Dagster. The API allows users to:
- Query information about pipeline runs (both historical and currently executing)
- Retrieve metadata about repositories, pipelines, and solids (such as pipeline structure and config schemas)
- Launch pipeline executions and re-executions, allowing users to trigger pipeline executions on custom events
Note about breaking changes¶
The GraphQL API is still evolving and is subject to breaking changes. A large portion of the API is primarily for internal use by Dagit. For any of the queries below, we will be clear about breaking changes in release notes.
Using the GraphQL API¶
The GraphQL API is served alongside the Dagit web server. To start the server, simply run dagit
in
your Dagster workspace.
dagit
Now the GraphQL API will be served at the /graphql
endpoint. If you are running Dagit locally on
port 3000, you can access the API at localhost:3000/graphql
Using the GraphQL Playground¶
You can access the GraphQL Playground by navigating to the /graphql
route in your browser. The
GraphQL Playground contains the full GraphQL Schema and an interactive playground to write and test
queries and mutations.
Exploring the GraphQL Schema and Documentation¶
Clicking on the Docs
tab on the right edge of the playground will open up interactive documentation
for the GraphQL API. This is the best way to explore the API and get information about which fields are available
on the queries and mutations.
Example Queries¶
Get a list of pipelines runs¶
To retrieve a list of all pipeline runs, use the pipelineRunsOrError
query.
query PaginatedPipelineRuns {
pipelineRunsOrError {
__typename
... on PipelineRuns {
results {
runId
pipelineName
status
runConfigYaml
stats {
... on PipelineRunStatsSnapshot {
startTime
endTime
stepsFailed
}
}
}
}
}
}
Pagination¶
You may eventually accumulate too many runs to return in one query. The pipelineRunsOrError
query
takes in optional cursor
and limit
arguments for pagination:
query PaginatedPipelineRuns {
pipelineRunsOrError(
cursor: "7fd2e5ef-5591-43db-be15-1ebbbbed8bb5"
limit: 10
) {
__typename
... on PipelineRuns {
results {
runId
pipelineName
status
runConfigYaml
stats {
... on PipelineRunStatsSnapshot {
startTime
endTime
stepsFailed
}
}
}
}
}
}
Filters¶
The pipelineRunsOrError
query also takes in an optional filter argument, of type PipelineRunsFilter
.
This allows you to filter runs by:
- run ID
- pipeline name
- tags
- statuses
For example, the following query will return all failed runs:
query FilteredPipelineRuns {
pipelineRunsOrError(filter: { status: FAILURE }) {
__typename
... on PipelineRuns {
results {
runId
pipelineName
status
runConfigYaml
stats {
... on PipelineRunStatsSnapshot {
startTime
endTime
stepsFailed
}
}
}
}
}
}
Get a list of repositories¶
This query will retrieve the names and location names of all the repositories currently loaded.
query RepositoriesQuery {
repositoriesOrError {
... on RepositoryConnection {
nodes {
name
location {
name
}
}
}
}
}
Get a list of pipelines within a repository:¶
Given a repository, this query will retrieve the names of all the pipelines in the repository.
This query takes a selector
, which is of type RepositorySelector
. This consists of both the
repository location name and repository name.
query PipelinesQuery(
$repositoryLocationName: String!
$repositoryName: String!
) {
repositoryOrError(
repositorySelector: {
repositoryLocationName: $repositoryLocationName
repositoryName: $repositoryName
}
) {
... on Repository {
pipelines {
name
}
}
}
}
Launch a pipeline run¶
To execute a pipeline, we can use the launchPipelineExecution
mutation. Here, we define
ExecutePipeline
to wrap our mutation and allow us pass in the required arguments as query
variables.
For this query, the required arguments are:
selector
:
- This is of type
PipelineSelector
, and consists of the repository location name, repository name, and pipeline name - You can also add a
solidSelection
argument here, to execute a subset of the pipeline
runConfigData
:
- This is where you define the run config to execute the pipeline with
- Note that
runConfigData
is of typeRunConfigData
. This type is used when passing in an arbitrary object for run config. This is any-typed in the GraphQL type system but must conform to the constraints of the config schema for this pipeline. If it does not, the mutation will appropriately return aPipelineConfigValidationInvalid
response
mode
:
- The mode to run the pipeline with.
- If you have not defined any custom modes for your pipeline, the default mode is
default
mutation ExecutePipeline(
$repositoryLocationName: String!
$repositoryName: String!
$pipelineName: String!
$runConfigData: RunConfigData!
$mode: String!
) {
launchPipelineExecution(
executionParams: {
selector: {
repositoryLocationName: $repositoryLocationName
repositoryName: $repositoryName
pipelineName: $pipelineName
}
runConfigData: $runConfigData
mode: $mode
}
) {
__typename
... on LaunchPipelineRunSuccess {
run {
runId
}
}
... on PipelineConfigValidationInvalid {
errors {
message
reason
}
}
... on PythonError {
message
}
}
}
Launch a pipeline run with preset¶
If you want to use a preset instead of defining the run config, use the preset
argument instead.:
mutation ExecutePipeline(
$repositoryLocationName: String!
$repositoryName: String!
$pipelineName: String!
$presetName: String!
) {
launchPipelineExecution(
executionParams: {
selector: {
repositoryLocationName: $repositoryLocationName
repositoryName: $repositoryName
pipelineName: $pipelineName
}
preset: $presetName
}
) {
__typename
... on LaunchPipelineRunSuccess {
run {
runId
}
}
}
}