GitHub (dagster-github)

This library provides an integration with GitHub Apps, to support performing various automation operations within your github repositories and with the tighter permissions scopes that github apps allow for vs using a personal token.

Presently, it provides a thin wrapper on the github v4 graphql API.

To use this integration, you’ll first need to create a GitHub App for it.

1. Create App: Follow the instructions in https://developer.github.com/apps/quickstart-guides/setting-up-your-development-environment/, You will end up with a private key and App ID, which will be used when configuring the dagster-github resource. Note you will need to grant your app the relevent permissions for the API requests you want to make, for example to post issues it will need read/write access for the issues repository permission, more info on GitHub application permissions can be found here
2. Install App: Follow the instructions in https://developer.github.com/apps/quickstart-guides/setting-up-your-development-environment/#step-7-install-the-app-on-your-account
3. Find your installation_id: You can pull this from the GitHub app administration page, https://github.com/apps/<app-name>/installations/<installation_id>. Note if your app is installed more than once you can also programatically retrieve these IDs. Sharing your App ID and Installation ID is fine, but make sure that the Private Key for your app is stored securily.

Posting Issues

Now, you can create issues in GitHub from Dagster with the GitHub resource:

import os

from dagster import job, op
from dagster_github import GithubResource


@op
def github_op(github: GithubResource):
    github.get_client().create_issue(
        repo_name='dagster',
        repo_owner='dagster-io',
        title='Dagster\'s first github issue',
        body='this open source thing seems like a pretty good idea',
    )

@job(resource_defs={
     'github': GithubResource(
         github_app_id=os.getenv('GITHUB_APP_ID'),
         github_app_private_rsa_key=os.getenv('GITHUB_PRIVATE_KEY'),
         github_installation_id=os.getenv('GITHUB_INSTALLATION_ID')
 )})
def github_job():
    github_op()

github_job.execute_in_process()

Run the above code, and you’ll see the issue appear in GitHub:

GitHub enterprise users can provide their hostname in the run config. Provide github_hostname as part of your github config like below.

GithubResource(
    github_app_id=os.getenv('GITHUB_APP_ID'),
    github_app_private_rsa_key=os.getenv('GITHUB_PRIVATE_KEY'),
    github_installation_id=os.getenv('GITHUB_INSTALLATION_ID'),
    github_hostname=os.getenv('GITHUB_HOSTNAME'),
)

By provisioning GithubResource as a Dagster resource, you can post to GitHub from within any asset or op execution.

Executing GraphQL queries

import os

from dagster import job, op
from dagster_github import github_resource


@op
def github_op(github: GithubResource):
    github.get_client().execute(
        query="""
        query get_repo_id($repo_name: String!, $repo_owner: String!) {
            repository(name: $repo_name, owner: $repo_owner) {
                id
            }
        }
        """,
        variables={"repo_name": repo_name, "repo_owner": repo_owner},
    )

@job(resource_defs={
     'github': GithubResource(
         github_app_id=os.getenv('GITHUB_APP_ID'),
         github_app_private_rsa_key=os.getenv('GITHUB_PRIVATE_KEY'),
         github_installation_id=os.getenv('GITHUB_INSTALLATION_ID')
 )})
def github_job():
    github_op()

github_job.execute_in_process()

Resources

class dagster_github.resources.GithubClient

A client for interacting with the GitHub API.

This client handles authentication and provides methods for making requests to the GitHub API using an authenticated session.

client

The HTTP session used for making requests.

Type: requests.Session

app_id

The GitHub App ID.

Type: int

app_private_rsa_key

The private RSA key for the GitHub App.

Type: str

default_installation_id

The default installation ID for the GitHub App.

Type: Optional[int]

hostname

The GitHub hostname, defaults to None.

Type: Optional[str]

installation_tokens

A dictionary to store installation tokens.

Type: Dict[Any, Any]

app_token

A dictionary to store the app token.

Type: Dict[str, Any]

create_issue

Create a new issue in the specified GitHub repository.

This method first retrieves the repository ID using the provided repository name and owner, then creates a new issue in that repository with the given title and body.

Parameters:

• repo_name (str) – The name of the repository where the issue will be created.
• repo_owner (str) – The owner of the repository where the issue will be created.
• title (str) – The title of the issue.
• body (str) – The body content of the issue.
• installation_id (Optional[int]) – The installation ID to use for authentication.

Returns: The response data from the GitHub API containing the created issue details.Return type: Dict[str, Any]Raises: RuntimeError – If there are errors in the response from the GitHub API.

create_pull_request

Create a new pull request in the specified GitHub repository.

This method creates a pull request from the head reference (branch) to the base reference (branch) in the specified repositories. It uses the provided title and body for the pull request description.

Parameters:

• base_repo_name (str) – The name of the base repository where the pull request will be created.
• base_repo_owner (str) – The owner of the base repository.
• base_ref_name (str) – The name of the base reference (branch) to which the changes will be merged.
• head_repo_name (str) – The name of the head repository from which the changes will be taken.
• head_repo_owner (str) – The owner of the head repository.
• head_ref_name (str) – The name of the head reference (branch) from which the changes will be taken.
• title (str) – The title of the pull request.
• body (Optional[str]) – The body content of the pull request. Defaults to None.
• maintainer_can_modify (Optional[bool]) – Whether maintainers can modify the pull request. Defaults to None.
• draft (Optional[bool]) – Whether the pull request is a draft. Defaults to None.
• installation_id (Optional[int]) – The installation ID to use for authentication.

Returns: The response data from the GitHub API containing the created pull request details.Return type: Dict[str, Any]Raises: RuntimeError – If there are errors in the response from the GitHub API.

create_ref

Create a new reference (branch) in the specified GitHub repository.

This method first retrieves the repository ID and the source reference (branch or tag) using the provided repository name, owner, and source reference. It then creates a new reference (branch) in that repository with the given target name.

Parameters:

• repo_name (str) – The name of the repository where the reference will be created.
• repo_owner (str) – The owner of the repository where the reference will be created.
• source (str) – The source reference (branch or tag) from which the new reference will be created.
• target (str) – The name of the new reference (branch) to be created.
• installation_id (Optional[int]) – The installation ID to use for authentication.

Returns: The response data from the GitHub API containing the created reference details.Return type: Dict[str, Any]Raises: RuntimeError – If there are errors in the response from the GitHub API.

execute

Execute a GraphQL query against the GitHub API.

This method sends a POST request to the GitHub API with the provided GraphQL query and optional variables. It ensures that the appropriate installation token is included in the request headers.

Parameters:

• query (str) – The GraphQL query string to be executed.
• variables (Optional[Dict[str, Any]]) – Optional variables to include in the query.
• headers (Optional[Dict[str, Any]]) – Optional headers to include in the request.
• installation_id (Optional[int]) – The installation ID to use for authentication.

Returns: The response data from the GitHub API.Return type: Dict[str, Any]Raises:

• RuntimeError – If no installation ID is provided and no default installation ID is set.
• requests.exceptions.HTTPError – If the request to the GitHub API fails.

get_installations

Retrieve the list of installations for the authenticated GitHub App.

This method makes a GET request to the GitHub API to fetch the installations associated with the authenticated GitHub App. It ensures that the app token is valid and includes it in the request headers.

Parameters: headers (Optional[Dict[str, Any]]) – Optional headers to include in the request.Returns: A dictionary containing the installations data.Return type: Dict[str, Any]Raises: requests.exceptions.HTTPError – If the request to the GitHub API fails.

dagster_github.resources.GithubResource ResourceDefinition

A resource configuration class for GitHub integration.

This class provides configuration fields for setting up a GitHub Application, including the application ID, private RSA key, installation ID, and hostname.

dagster_github.resources.github_app_id

The GitHub Application ID. For more information, see https://developer.github.com/apps/.

Type: int

dagster_github.resources.github_app_private_rsa_key

The private RSA key text for the GitHub Application. For more information, see https://developer.github.com/apps/.

Type: str

dagster_github.resources.github_installation_id

The GitHub Application Installation ID. Defaults to None. For more information, see https://developer.github.com/apps/.

Type: Optional[int]

dagster_github.resources.github_hostname

The GitHub hostname. Defaults to api.github.com. For more information, see https://developer.github.com/apps/.

Type: Optional[str]

Legacy

dagster_github.resources.github_resource ResourceDefinition