Deploy Anyscale on Kubernetes

This page provides an overview of the permissions and resources required to deploy an Anyscale cloud on Kubernetes.

You can deploy Anyscale to an existing Kubernetes cluster or deploy Anyscale alongside a new Kubernetes cluster. Supported Kubernetes clusters include Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), Oracle Kubernetes Engine (OKE), CoreWeave, or other Kubernetes clusters running in the cloud or on-prem.

important

Deploying an Anyscale cloud on Kubernetes requires customization for your environment. Anyscale works with customers to customize Terraform modules that deploy resources in your cloud provider account or other Kubernetes environment.

Contact Anyscale support for assistance customizing Terraform modules and deploying Anyscale to Kubernetes.

You can review the Anyscale Terraform modules for Kubernetes, but Anyscale doesn't recommend attempting to deploy an Anyscale cloud on Kubernetes independently.

For self-service onboarding, Anyscale recommends deploying Anyscale using virtual machines on AWS or Google Cloud.

What is the Anyscale operator for Kubernetes?

The Anyscale operator for Kubernetes manages the relationship between the Anyscale control plane and your Kubernetes cluster. When you deploy Anyscale on Kubernetes, you configure a control plane role, networking, and security to allow the Anyscale control plane to use the Anyscale operator to manage resources in your Kubernetes cluster.

When you deploy an Anyscale cloud on Kubernetes, Anyscale installs the operator to your Kubernetes cluster. You interact with the Anyscale control plane to configure workspace, jobs, and services. The control plane sends instructions to the Anyscale operator to deploy Ray nodes using pods in your Kubernetes cluster.

The following diagram provides a high-level overview of the architecture of an Anyscale cloud on Kubernetes:

Namespace resources used by the Anyscale operator

The Anyscale operator uses the following namespace resources in your Kubernetes cluster:

• Pods: Each Anyscale node maps to a single pod.
• Services and Ingresses: Used for head node connectivity and for exposing Anyscale services. Ingresses may be either private or public.
• Secrets: Used to hold secrets used by the Anyscale operator.
• ConfigMaps: Used to store configuration options for the Anyscale operator.
• Events: Used to enhance workload observability.

Global resources used by the Anyscale operator

The Anyscale operator uses the following global resources in your Kubernetes cluster:

• TokenReview: On the startup of an Anyscale node in an Anyscale workload, Anyscale uses the Kubernetes TokenReview API to verify a pod's identity when the pod bootstraps itself to the Anyscale control plane.
• Nodes: The operator periodically reads node information to enhance workload observability.

Installing the Helm chart for the Anyscale operator requires permissions to create cluster roles and cluster role bindings, which grant the Anyscale operator the necessary permissions to manage these global resources. If you don't have these permissions, consider deploying Anyscale inside of vCluster in a Namespace of your choice.

Features missing from Anyscale clouds on Kubernetes

Most Anyscale features have full support for clouds deployed on Kubernetes, with the following exceptions:

• Some optimization features for accelerated cluster startup aren't available.
• For zero downtime upgrades to Anyscale services, you must use an ingress control that Anyscale can patch.

note

If you don't have access to desired Anyscale features, your admin might have deployed your Anyscale cloud without required resources, networking, or permissions. Contact Anyscale support to troubleshoot your deployment.

Object storage and IAM roles for Kubernetes deployments

All Anyscale clouds deployed on Kubernetes require access to a cloud object storage location to persist production artifacts, which include cluster logs, workspace snapshots, workload checkpoints, and cached container images. All Pods, the Anyscale operator, and the Anyscale control plane must have permissions to read and write files to this storage location.

If you use a managed Kubernetes service, you should configure the default object storage location using resources in the same account. See Deploy Anyscale to managed Kubernetes services.

If you're deploying to a custom Kubernetes cluster such as on-prem, you can choose an object storage location in any cloud provider. See the following docs for details:

• Extend AWS IAM roles to workloads outside of AWS with IAM Roles Anywhere
• Workload Identity Federation
• Authorize access to data in Azure Storage

Deploy Anyscale to managed Kubernetes services

You can deploy Anyscale to Kubernetes services managed by AWS, Azure, or Google Cloud.

You must configure IAM permissions and a storage location in your cloud provider account.

Cloud provider managed Kubernetes service	Default storage location	IAM requirements
Amazon Elastic Kubernetes Service (EKS)	An S3 bucket, ideally in the same region and account for simplified setup and reduced ingress and egress costs.	An IAM role that the Anyscale operator and control plane can use to manage infrastructure in EKS and connect to your S3 bucket. An instance profile that nodes can assume to grant access to your S3 bucket. See Learn how EKS Pod Identity grants pods access to AWS services.
Google Kubernetes Engine (GKE)	A Google Cloud Storage (GCS) bucket, ideally in the same region and project for simplified setup and reduced ingress and egress costs.	A service account that the Anyscale operator and control plane can use to manage infrastructure in GKE and connect to your GCS bucket. The Anyscale operator must have the Service Account Token Creator role to generate presigned URLs for objects in the storage bucket. A service account that nodes can assume to grant access to your GCS bucket. See About Workload Identity Federation for GKE
Azure Kubernetes Service (AKS)	A blob storage container, ideally in the same region and account for simplified setup and reduced ingress and egress costs.	A Microsoft Entra ID that the Anyscale operator and control plane can use to manage infastructure in AKS and connect to your blob container. A Microsoft Entra ID that nodes can assume to grant access to your blob container. See Authorize access to blobs using Microsoft Entra ID

Deploy Anyscale to other Kubernetes clusters

Anyscale supports deploying clouds to most Kubernetes clusters, including on-prem.

To get full access to Anyscale platform features, you must configure a default storage account and IAM permissions in AWS, Azure, or Google Cloud alongside your custom or on-prem Kubernetes cluster. See Object storage and IAM roles for Kubernetes deployments.

General requirements for Anyscale clouds on Kubernetes

Each Anyscale cloud deployed to Kubernetes is a custom deployment based on how you configure and run Kubernetes.

Contact Anyscale support to get started deploying on Kubernetes. Consider the following requirements and recommendations when choosing to deploy to Kubernetes:

• Use a Kubernetes cluster v1.28 or later when possible.
• Grant Anyscale permissions to deploy a Helm chart into the Kubernetes cluster.
• Permissions on a Kubernetes Service Account that has permissions to operate core Kubernetes resources.
• Identify the name of target Kubernetes Namespace to deploy the Anyscale operator.
• Configure the Ingress-NGINX controller and set the following properties:
  • allow-snippet-annotations set to true.
  • annotations-risk-level set to Critical.
  • enable-underscores-in-headers set to true.
• Egress to the internet from Anyscale pods deployed into the Kubernetes cluster. This is a requirement of all Anyscale deployments.
• If using GPUs with EKS or AKS, configure the k8s-device-plugin.
  • This isn't required to use GPUs with GKE.
• You must decide how to configure your load balancer and networking rules:
  • If you choose direct networking, configure an internet-facing load balancer that opens port 443 access to the head pod.
  • For custom networking, configure an internal load balancer.
    • In some cases, you can apply an annotation on the LoadBalancer service in front of the NGINX pods to configure internal load balancing.

note

Anyscale also supports using Kubernetes Gateway and Istio as ingress controllers. Other ingress controllers might work, but Anyscale hasn't tested them and doesn't guarantee support.

Anyscale hasn't tested earlier versions of Kubernetes and doesn't guarantee support.