Skip to main content

Deploy Anyscale on non-managed Kubernetes (cloud register)

View Markdown

Deploy Anyscale on non-managed Kubernetes (cloud register)

This page describes how to deploy Anyscale on a Kubernetes cluster that doesn't have a dedicated Anyscale deployment guide. You provision and configure all resources yourself, then register the cloud with Anyscale using --provider generic.

tip

Dedicated deployment guides are available for the following providers. Use those instead of this page:

Prerequisites​

Before you start, ensure the following:

  • You are an Anyscale organization owner.
  • You have an existing Kubernetes cluster. Anyscale recommends v1.28 or later.
  • You have an object storage bucket accessible from your cluster. Anyscale supports S3-compatible storage (s3://), Google Cloud Storage (gs://), or Azure Blob Storage (abfss://). The bucket must be readable and writable by all pods and the Anyscale control plane.
  • You have a CLI token for an Anyscale service account. Anyscale uses this token to authenticate the operator instead of cloud-provider IAM. Generate one in the Anyscale console under User settings > API keys.
  • The following CLI tools are installed: Anyscale CLI, kubectl, helm.
  • The Nginx ingress controller is installed and configured on your cluster. Set the following properties:
warning

Anyscale services require allow-snippet-annotations: true and annotations-risk-level: "Critical", which override ingress-nginx's default protections against CVE-2021-25742. Restrict ingress resource permissions to trusted users only.

1. Install the Anyscale CLI​

pip install -U anyscale
anyscale login

2. Register the Anyscale cloud​

Run the following command to register your Anyscale cloud. Replace the placeholders with values for your environment.

anyscale cloud register \
  --name <cloud-name> \
  --provider generic \
  --region <region> \
  --compute-stack k8s \
  --cloud-storage-bucket-name <storage-bucket-uri> \
  --cloud-storage-bucket-endpoint <storage-endpoint-url>

Set the following values:

  • <cloud-name>: A unique name for your Anyscale cloud.
  • <region>: The region for your cluster. This is used for labeling and routing and doesn't need to match a specific cloud provider region.
  • <storage-bucket-uri>: The URI for your object storage bucket. Use the prefix appropriate for your storage type:
    • S3-compatible: s3://<bucket-name>
    • Google Cloud Storage: gs://<bucket-name>
    • Azure Blob Storage: abfss://<container-name>@<storage-account-name>.dfs.core.windows.net
  • <storage-endpoint-url>: The HTTPS endpoint URL for your storage provider. For example, https://<storage-account-name>.blob.core.windows.net for Azure Blob Storage.

The command outputs a cloud resource ID in the format cldrsrc_xxx. Record this ID for the next step.

3. Install the Anyscale operator​

Add the Helm chart​

helm repo add anyscale https://anyscale.github.io/helm-charts
helm repo update anyscale

Install the operator​

Set your Anyscale CLI token as an environment variable, then install the operator:

export ANYSCALE_CLI_TOKEN=<cli-token>

helm upgrade <release-name> anyscale/anyscale-operator \
  --set-string global.cloudDeploymentId=<cloud-resource-id> \
  --set-string global.cloudProvider=generic \
  --set-string global.auth.anyscaleCliToken=$ANYSCALE_CLI_TOKEN \
  --set-string workloads.serviceAccount.name=anyscale-operator \
  --namespace <namespace> \
  --create-namespace \
  --wait \
  -i

Set the following values:

  • <release-name>: The Helm release name, such as anyscale-operator.
  • <cloud-resource-id>: The cloud resource ID from the previous step.
  • <cli-token>: Your Anyscale service account API key.
  • <namespace>: The Kubernetes namespace for the operator. Anyscale recommends a dedicated namespace per Anyscale cloud.

To customize the Helm chart with custom patches or additional pod shapes, see Configure the Helm chart for the Anyscale operator.

4. Verify your cloud​

anyscale cloud verify --name <cloud-name>

Next steps​