Skip to main content

Concepts

Anyscale simplifies building, running and managing Ray applications.

With Anyscale you can:

  • Automate the creation, scaling and termination of cloud resources (e.g., AWS, GCP) required to run Ray workloads.
  • Monitor and manage the resources to ensure that your workloads complete successfully.

Anyscale provides these functionalities through a managed control plane. Workloads run on a data plane, which can run in Anyscale's account or in your own. The following diagram summarizes the Anyscale architecture:

Running a Script on Anyscale

As an Anyscale user, you have to be registered as part of an organization. In general, users within the organization will be able see workloads resources launched by others. Optionally, you can use projects to organize resources, or if you want to only provide access to certain users.

Clusters

Any workload started by Anyscale runs on a Ray cluster. An important advantage of Anyscale over open-source Ray clusters is that you will not need to manage the cluster lifecycle. As we saw in the Get Started tutorial, you can specify anyscale:// as the Ray address. This will start a Ray cluster for you, and Anyscale will also automatically terminate it when it idles for a period of time.

Additionally, Ray clusters on Anyscale automatically scale to accomodate your workload. Anyscale has a set of defaults that include support for CPU and GPU workloads, so you don't need to configure the compute resources that power your application. If you want to customize, e.g., the instance types or the maximum cluster size, you can do so by specifying a cluster compute.

By default, Anyscale clusters are fully-managed, meaning that both the control and data plane run on a single AWS region of your choice, within Anyscale's cloud account. Anyscale clouds provide support for adding more regions within Anyscale's cloud account. You can create clouds that include an AWS or GCP identity, which will be used to run the data plane on your own cloud account. If you're using multiple Anyscale clouds, you can change your preferred default and also override the default within your cluster compute.

Finally, Anyscale provides multiple ways to manage dependencies. Anyscale allows you to build cluster environments, creating Docker images that package all your dependencies ahead of runtime. This way, cluster startup can be fast and reliable. Anyscale also supports runtime environments which, among other things, enable you to upload code and data from your machine onto the cluster, and make it possible to use different dependencies within different parts of your application.

Applications

The most convenient way to scale up your application depends on whether you're in the process of developing your application or you're trying to deploy your application in production.

Development

During development, it is usually ideal to run things locally on your machine. Using Ray client, your application can seamlessly start an interactive session with an Anyscale cluster to tap into cloud resources for compute-intensive operations. This effectively allows you to scale your laptop to meet the demands of any application without modifying your code or setting up any infrastructure.

Production

In production, it is preferable to use the cloud to run not just compute-intensive operations, but also the application itself. Additionally, you want to make sure that the application is fault-tolerant to avoid incidents and guarantee availability.

Anyscale can support production workloads, which fall under two types:

  • If your application has a bounded amount of computation to do, e.g., an RLlib application, it is considered a job. When you start a production job, Anyscale will provision a Ray cluster, deploy the application and run it to completion, restarting it in the event of failure.
  • If your application needs to run indefinitely, e.g., a Ray Serve application, it is considered a service. When you start a service, Anyscale will provision a Ray cluster that provides high availability, including rolling upgrades.

Production APIs for both jobs and services are declarative. They allow you to specify a working directory as part of the runtime environment, which can contain both code and data that the application will use, along with an entrypoint to start the application. Jobs and services leverage Anyscale Ray clusters, which means that they also support the usage of cluster computes and cluster environments.

Interfaces

Throughout this documentation, you'll see references to the four interfaces that Anyscale supports:

  • a Web UI, accessible at console.anyscale.com,
  • a CLI, which can be installed via pip install anyscale,
  • a Python SDK, which will also be installed with pip install anyscale, and
  • an HTTP API, which can be used for programmatic access in any language.