Container-driven development on Anyscale

This article provides an overview of developing in containers and provides recommendations for each stage in your development lifecycle.

How does Anyscale use containers?

Anyscale uses containers for deploying nodes in Ray clusters for workspaces, jobs, and services.

Each container is replica of an immutable container image specified as part of the cluster definition. See Define a Ray cluster and What is a container image?.

Anyscale recommends using managed base images when beginning development on a new workload. Users with a good grasp of containers might choose to build a custom image that extends a base image with their own dependencies, environmental variables, or init scripts before starting development.

For users new to the Anyscale platform or developing in containers, you can launch an Anyscale workspace using a base image and then iterate on this image using the Anyscale console. See Runtime environments on Anyscale.

When you prepare your workload for production, you create a custom image that extends an Anyscale base image to add additional dependencies, code assets, and environmental variables. See Custom images on Anyscale.

What are the primary components of an Anyscale application?

Anyscale launches Ray clusters as workspaces, jobs, or services.

Despite the differences in use cases for workspaces, jobs, and services, they all use the same two primary constructs for configuring and defining the Ray cluster and application: compute configurations and container images. See Define a Ray cluster.

When you are developing a new application on Anyscale, you are iterating on the code that you eventually package as a container image to submit to a Ray cluster. The following table provides an overview of the key components you need to iterate on during development:

Application component	Description	Examples
Codebase	The code and configuration files that define your workload or application.	Python Ray scripts, Python modules, Bash scripts, YAML configurations.
Python dependencies	Python packages from a public or private package index.	PyPi packages, Python libraries in index mirrors or caches, Python libraries from custom repositories.
System packages	Packages installed on the base Ubuntu operating system.	APT packages, programs downloaded and installed from the terminal, init scripts.
Environment variables	Variables set at the system level accessible to all Ray nodes.	Default environment variables, variables set in init scripts, using secrets, runtime environment variables.

Develop in a container

Anyscale recommends starting development using an Anyscale base image with a recent version of Ray. Use a slim image to reduce the size of containers. If you need access to a library excluded from slim images by default, you can install it using runtime environments. You can choose Python and CUDA versions based on other dependencies in your environment, but Anyscale recommends you use recent versions to access the latest features. See Anyscale base images.

The following table provides an overview of the recommended approaches to developing on Anyscale. See Develop Anyscale applications.

Approach	Description
Workspace development	Develop code in hosted IDEs or SSH to your workspace. Run Ray code interactively using Jupyter notebooks or Python scripts. Manage dependencies using tools in the Anyscale console. Launch jobs or services using dependencies injected from the workspace environment.
Local development	Develop in your preferred IDE on your personal machine. Launch services or submit jobs to Anyscale using the CLI or SDK, using built-in options for additional dependency management. Use the Anyscale console for monitoring and observability.

note

You can optionally develop new workloads using a local Ray cluster or on-prem infrastructure such as KubeRay, and then deploy to Anyscale. Because most Anyscale features extend Ray functionality, many of the same patterns apply, but you might need to refactor code and change configurations as you move to deploy on Anyscale.

If you need to use package managers such as poetry that aren't supported by runtime environments or uv, you need to install and configure them in a containerfile and build a custom image.

Refactor development patterns to define custom images

Anyscale recommends refactoring development code and configurations to solidify dependencies in a custom image before moving to production. See Custom images on Anyscale.

The following table provides an overview of job and service configuration options used during development that Anyscale recommends refactoring when building a production image:

note

Anyscale configures many of these options by default when you launch jobs or services from an Anyscale workspace. See Launch jobs and services from an Anyscale workspace.

Option	How to include in image
working_dir	Include instructions in your containerfile to copy your codebase to the path /home/ray/default/ in your container. Depending on how you build your container, this might be adding all files from your local directory, pushing artifacts from a CI/CD tool, or cloning code from one or more Git repositories.
excludes	Exclude files or directories from your image by not including them in the artifacts copied to /home/ray/default/.
requirements	Install Python dependencies in your containerfile.
env_vars	Define environment variables in your containerfile.
py_modules	Install custom Python modules to /home/ray/default/ or from a custom package repository.

Test code and package a container for production

You can build a custom image on Anyscale using standard containerfile syntax. Anyscale provides tools to build container images using the console, CLI, or SDK. See Build a custom image on Anyscale.

You can configure Anyscale to use container images stored in external image registries. This means you can use external tooling for CI/CD automation to test code and build container images.

Anyscale recommends testing custom images before deploying new workloads to production. Each application has different testing requirements, and the way you implement testing varies based on the tools you use and how the application interacts with the rest of your ecosystem. The following list provides a general overview of testing:

• Write unit tests to validate functions and modules in your Ray application.
• Write integration tests to ensure that modules and scripts in your Ray application interact as required.
• Write system tests to confirm the application meets the specified requirements for the end-to-end application.
• Gather feedback from users for acceptance testing to ensure that the results of the submitted Anyscale job or launched Anyscale service behave as expected.

As an example, you might run unit tests for each custom Python module in your application using lightweight Python testing before building a custom image, and then run integration and system tests using in a Ray cluster launched with your custom image. If you choose to build an image as the first step or start from a manually defined custom image, you run unit tests and integration tests in a Ray cluster launched with your custom image.

Depending on the maturity of your testing infrastructure and the frequency of changes to your applications, you might choose to have robust automation for testing or instead rely primarily on human quality assurance testing.

Example: Containerize Ray code and submit a job

The following example Python script uses the SDK to complete the following:

• Define a containerfile that extends a base image by adding a Python package and copying all content from the current working directory.
• Build an image on Anyscale based on that containerfile.
• Submit a job using the built image.
• Wait for the job to complete.
• Get the logs for the job.

You could use this pattern with your preferred CI/CD tooling to run final tests on Anyscale after passing unit tests at an earlier step. You could also use this pattern for lightweight manual validation testing from your local machine.

import anyscale
from anyscale.job.models import JobConfig

# Define the containerfile.
containerfile = '''
# Specify base image.
FROM anyscale/ray:2.47.1-slim-py312

# Install Python package.
RUN pip install --no-cache-dir --upgrade emoji

# Add the local working directory to the default entrypoint.
COPY --chown=ray:ray . /home/ray/default/
'''

# Build the image.
image_uri = anyscale.image.build(containerfile, name="image-name")

# Deploy the image as an Anyscale job.
anyscale.job.submit(
    JobConfig(
        name="job-name",
        entrypoint="python main.py",
        image_uri=image_uri
    ),
)

# Wait for the job to succeed.
anyscale.job.wait(name="job-name")

# Get the logs for the job.
anyscale.job.get_logs(name="my-job")

Deploy a container in production

You deploy Ray applications to production by submitting an Anyscale job or launching an Anyscale service. In addition to a custom image, you must define a compute configuration to deploy an Anyscale cluster. See Define a Ray cluster.

Container images are immutable by definition. Compute configurations define the resources used by your cluster. Cluster size scales independently of your image.

If you have included all dependencies in your custom image, you can deploy your application without specifying options that install dependencies, define environment variables, or upload files. See Refactor development patterns to define custom images.

See the following sections in the CLI and SDK reference for a full list of options for Anyscale jobs and services:

Cluster type	CLI	SDK	Config YAML
Job	anyscale job submit	anyscale.job.submit	JobConfig
Service	anyscale service deploy	anyscale.service.deploy	ServiceConfig

note

Anyscale recommends avoiding building dependencies on external services when possible.