This guide is applicable to Dagster Cloud.
In this guide, you'll configure and run a Docker agent. Docker agents are used to launch your code in Docker containers.
To complete the steps in this guide, you'll need:
Permissions in Dagster Cloud that allow you to manage agent tokens. Refer to the User permissions documentation for more info.
To have Docker installed
Access to a container registry to which you can push images with Dagster code. Additionally, your Docker agent must have the permissions required to pull images from the registry.
This can be:
In this step, you'll generate a token for the Dagster Cloud agent. The Dagster Cloud agent will use this to authenticate to the agent API.
Keep the token somewhere handy - you'll need it to complete the setup.
Create a Docker network for your agent:
docker network create dagster_cloud_agent
instance_class: module: dagster_cloud.instance class: DagsterCloudAgentInstance dagster_cloud_api: agent_token: <YOUR_AGENT_TOKEN> deployment: prod user_code_launcher: module: dagster_cloud.workspace.docker class: DockerUserCodeLauncher config: networks: - dagster_cloud_agent
In the file, fill in the following:
agent_token - Add the agent token you created in Step 1
deployment - Enter the deployment associated with this instance of the agent.
In the above example, we specified
prod as the deployment. This is present when Dagster Cloud organizations are first created.
Save the file.
Next, you'll start the agent as a container. Run the following command in the same folder as your
docker run \ --network=dagster_cloud_agent \ --volume $PWD/dagster.yaml:/opt/dagster/app/dagster.yaml:ro \ --volume /var/run/docker.sock:/var/run/docker.sock \ --restart on-failure \ -it docker.io/dagster/dagster-cloud-agent:latest \ dagster-cloud agent run /opt/dagster/app
dagster.yamlmounted as a volume
--restart-on-failureargument will ensure that if it fails, Docker will restart it.
To view the agent in Dagster Cloud, navigate to the Status page and click the Agents tab. You should see the agent running in the Agent statuses section:
Now that you've got your agent running, what's next?