Skip to main content

Third-party Docker registries


This feature is only supported on AWS, GCP support is currently in progress. Contact Anyscale support for more details.

Bring your own Docker environments can be configured to pull from private Docker registries that are hosted outside of your cloud provider, such as JFrog Artifactory and Docker Hub. In order to authenticate, Anyscale managed nodes need access to Docker login credentials to authenticate to the registry hosting the image.

The following diagram illustrates the relationship between the Anyscale control plane, the data plane, and the third-party registry:

The credentials for the third-party registry never enter the Anyscale control plane. The Anyscale control plane stores a reference to a secret in your cloud provider's secret manager and nodes in your account fetch the contents of the secret to authenticate at runtime.

Step 1. Obtain access credentials for your registry

This step will vary depending on your Docker registry provider. The credentials only need to allow read access to the registry containing your image, write and delete permissions are not needed. If you have Docker setup locally, you can test the credentials with docker login:

docker login # This should prompt you for a username and password
docker pull # This should succeed

Step 2. Store credentials in your cloud provider's secret manager

Once you have obtained credentials to authenticate to your Docker registry provider, they should be stored in your cloud provider's secret manager in the following JSON format:

"username": "myusername",
"password": "mypassword",
"server": ""

The server field should match the server in the image you want to pull. For example, for the image the server field should be set to For images hosted on Docker Hub, use the server At runtime, Anyscale managed nodes will check if the server field in the secret matches the server hosting the image. If there is a mismatch, the credentials will not be used to authenticate to avoid leaking credentials to the wrong registry provider.

Step 3. Grant read access to managed nodes

See this guide for granting a role access to a secret. Make sure to grant access to the role used by Anyscale managed nodes in your account. In most cases, this should be <cloud_id>-cluster_node_role role. If your nodes are being launched with the role ray-autoscaler-v1, or if you are using a custom AWS IAM role then you can apply the same steps to that role instead to grant access.

Step 4. Register the secret with Anyscale

When creating a cluster environment for your image, use the registry_login_secret field to configure your cluster to pull. On AWS, this should be the full ARN of the secret.

Create a YAML configuration file like the following:

docker_image: my-registry/my-image:tag
ray_version: 2.7.0 # Replace this with the version of Ray in your image
env_vars: # Optionally, specify environment variables
MY_VAR: value
registry_login_secret: mysecretid

Then, run the following:

anyscale cluster-env build -n <cluster-env-name> my_cluster_env.yaml