Skip to main content

Iterating on dependencies

The challenge of managing dependencies is harder in the distributed setting because the relevant packages, code, environment variables, and other dependencies must exist across a cluster of machines.

Mental model: containers​

The heart of dependency management on Anyscale is a container image. Fundamentally, Anyscale runs your Ray workloads, namely the Ray driver and worker processes, inside of containers.

As an Anyscale user, you're responsible for specifying the container image. You can base that container image on an Anyscale-provided base image or on a user-provided base image, as long as it conforms to certain specs.

Anyscale also provides a build-farm for easily building container images, but many Anyscale users choose to use their own image-building processes.

When running a Ray workload, the primary way to specify a container image is through the image_uri parameter. The following is a simple job specification from this example's job.yaml file:

name: my-first-job

# When empty, use the default image. This can be an Anyscale-provided base image
# like anyscale/ray:2.43.0-slim-py312-cu125, a user-provided base image (provided
# that it meets certain specs), or you can build new images using the Anyscale
# image builder at https://console.anyscale.com/v2/container-images.
image_uri: anyscale/ray:2.44.0-slim-py312-cu125

# Path to a local directory or a remote URI to a .zip file (S3, GS, HTTP) that
# is the working directory for the job. The files in the directory are
# automatically uploaded to the job environment in Anyscale.
working_dir: .

env_vars:
  EXAMPLE_ENV_VAR: EXAMPLE_VAL

# The script to run in your job. You can also do "uv run main.py" if you have a
# pyproject.toml file in your working_dir.
entrypoint: python main.py

Iteration during development​

During development, it's natural to iterate on your codebase as well as on other dependencies.

Anyscale provides a few building blocks for iterating on code and dependencies. In particular, you can specify the following when configuring a Ray workload:

  • image_uri: The base image to use for the Ray cluster.
  • working_dir: A path to a local directory or a remote URI to a zip file that is the working directory for the job. This directory often contains your codebase. The files in the directory are automatically uploaded to your Anyscale environment. You can also specify this parameter from the CLI with --working-dir=..
  • env_vars: Environment variable settings. You can also specify an environment variable from the CLI with --env EXAMPLE_ENV_VAR=EXAMPLE_VAL.
  • containerfile: (alpha and only available on VMs) An alternative to image_uri. Instead of specifying the base image, this parameter is a containerfile that can specify a base image plus additional dependencies. This parameter is useful for rapid iteration in scenarios where you want a lightweight way to add additional dependencies on top of a base image. Anyscale builds the additional dependencies on the fly and uses them throughout the cluster. See the containerfile section for more details.

Iterating on code​

During development, your codebase changes all the time. The most straightforward way to iterate on your code and to make sure Anyscale propagates the code changes throughout the entire cluster is to put the codebase into the working directory.

In an Anyscale Job or Service, you specify this working directory in the job or service YAML or in the Anyscale CLI. In a workspace, the working directory is /home/ray/default/.

If the codebase must be installed, for example if it's a Python package, or requires commands to be executed, you can do that in the containerfile as described below.

Integration with uv (alpha)​

uv offers very powerful tools for Python dependency management and is rapidly emerging as a standard. To use uv along with Ray, run your script with uv run. You need to install uv in the base image.

Normally, when running a script with uv run, uv pulls in all of the relevant Python dependencies and runs the script in the appropriate environment. You can define that environment in a pyproject.toml file, in-line in the uv run command, or in a variety of other manners.

Ray integrates with uv to ensure that it not only runs the Ray driver script with the correct uv run command, but also runs all of the Ray worker processes with the correct uv run command.

Because this feature is in alpha, you also have to set the following environment variable.

RUN echo "export RAY_RUNTIME_ENV_HOOK=ray._private.runtime_env.uv_runtime_env_hook.hook" >> /home/ray/.bashrc

In a workspace, you can run uv run --with emoji main.py from a terminal to run your script with uv. In a job, you can define your entrypoint to be a command like uv run --with emoji main.py.

Iterating on the containerfile (alpha)​

However, not all dependencies are Python dependencies. For more general dependencies, the natural approach is to build them into the underlying image. For situations where the container building process is cumbersome, Anyscale provides the ability to provide a containerfile (using a subset of Dockerfile instructions), which specifies some additional dependencies on top of a base image. The additional dependencies are installed and cached on the fly in order to create the container image used to run the Ray workload. Here is a simple example which installs uv, ffmpeg, and vllm.

For jobs and services, you can specify the containerfile using the containerfile parameter in the job or service YAML file. For workspaces, you can specify the containerfile in the Dependencies tab in the UI.

FROM anyscale/ray:2.44.1-slim-py312-cu128

RUN curl -LsSf https://astral.sh/uv/install.sh | sh
RUN echo "export RAY_RUNTIME_ENV_HOOK=ray._private.runtime_env.uv_runtime_env_hook.hook" >> /home/ray/.bashrc

RUN sudo apt-get update -y \
   && sudo apt-get install --no-install-recommends -y ffmpeg \
   && sudo rm -f /etc/apt/sources.list.d/*

RUN uv pip install --system vllm

You can modify the containerfile during development to iterate rapidly during development without going through a cumbersome image building process. This feature is only available when deploying Anyscale on VMs.

Building dependencies inside your working directory​

If you're iterating rapidly on code that needs to be installed (editable packages) or has some build process, you have a couple ways to handle this.

One option is to put the codebase in your working directory, mount the working directory in the containerfile, and install the codebase in the containerfile. The following is an example assuming your codebase is called my_module:

FROM anyscale/ray:2.44.1-slim-py312-cu128
COPY --chown=ray:ray working_dir /home/ray/default
RUN cd /home/ray/default/my_module && pip install -e .

This example mounts the working directory as uv builds the image so that within the containerfile, you can build the codebase in your working directory.

But if you need to clone a repository directly in the containerfile, be sure to check that it doesn’t already exist:

COPY --chown=ray:ray working_dir /home/ray/default/
RUN cd /home/ray/default && \
    [ -d sampleproject ] || git clone https://github.com/pypa/sampleproject
RUN cd /home/ray/default/sampleproject && pip install -e .

Modifying the containerfile in workspaces​

When working with Anyscale Workspaces, you can access the container file in the Dependencies tab. This tab shows the base image along with additional dependencies that Ray builds on top of the base image.

To modify dependencies on the head node, run commands in a terminal on the workspace.

To modify dependencies on the worker nodes, edit the containerfile in the Dependencies tab and rebuild the container. If the containerfile builds successfully, then all new Ray applications that you run in the workspace use the new image for their worker processes. The driver process runs on the head node and uses the head node image.

In order to keep the head node image in sync with the image for the worker nodes, restart the workspace. Whenever you restart the workspace, it uses the last image successfully built from the containerfile in the Dependencies tab.

Other details​

There are some other important details about providing container files.

  • The additional dependencies on top of the base image are built in the data plane (essentially in the same place that the Ray workload runs), and so images built in this way have the same credentials as the underlying Ray workload. This typically means they have the ability to clone private repositories.