Ask AI

Configuring and running a Kubernetes agent#

This guide is applicable to Dagster+.

In this guide, you'll configure and run a Kubernetes agent. Kubernetes agents are used to launch your code in Kubernetes Jobs and Services.


To complete the steps in this guide, you'll need:

Step 1: Generate a Dagster+ agent token#

In this step, you'll generate a token for the Dagster+ agent. The Dagster+ agent will use this to authenticate to the agent API.

  1. Sign in to your Dagster+ instance.
  2. Click the user menu (your icon) > Organization Settings.
  3. In the Organization Settings page, click the Tokens tab.
  4. Click the + Create agent token button.
  5. After the token has been created, click Reveal token.

Keep the token somewhere handy - you'll need it to complete the setup.

Step 2: Configure the agent#

In this step, you'll create a Kubernetes namespace for your Dagster+ resources. You'll also add the agent token to the Kubernetes cluster.

  1. Create a Kubernetes namespace for your Dagster+ resources:

    kubectl create namespace <namespace>

    For example, if the namespace is dagster-plus:

    kubectl create namespace dagster-plus
  2. Add the agent token you created in Step 1 as a secret in the Kubernetes cluster:

    kubectl create secret generic dagster-plus-agent-token \
        --from-literal=DAGSTER_CLOUD_AGENT_TOKEN=<your_agent_token> \
        --namespace dagster-plus

Step 3: Start the agent#

In this step, you'll spin up the agent with Helm.

  1. Add the agent chart repository:

    helm repo add dagster-plus
    helm repo update
  2. Install the agent chart, specifying:

    • The namespace into which to install the chart
    • The Dagster+ deployment the agent will be responsible for. Note: When your Dagster+ organization is first created, the default deployment is prod:
    helm upgrade \
       --install user-cloud dagster-cloud/dagster-cloud-agent \
       --namespace dagster-plus \
       --set dagsterCloud.deployment=prod

After the Helm chart is installed into the Kubernetes cluster, you can view the agent in Dagster+.

In Dagster+, navigate to the Status page and click the Agents tab. You should see the agent running in the Agent statuses section:

Instance Status

Next steps#

Now that you've got your agent running, what's next?

You can also further configure the jobs the agent runs, including adding secrets, mounting volumes, and setting resource limits.