Skip to main content

Deploy Anyscale on CoreWeave

View Markdown

Deploy Anyscale on CoreWeave

This page provides an overview of deploying a new Anyscale cloud on CoreWeave Kubernetes Service (CKS).

To learn more about CKS, see Introduction to CoreWeave Kubernetes Service.

Requirements​

Before you start, ensure the following:

  • You are an Anyscale organization owner.
  • The following CLI tools are installed: Anyscale CLI, aws CLI, kubectl, helm.

Step 1: Create and configure CoreWeave resources​

Anyscale requires the following CoreWeave resources and configurations.

  • A CKS cluster.
  • A node pool.
  • A CoreWeave AI object storage bucket with CORS enabled.
  • A kubeconfig file with a valid CoreWeave API access token for your CKS cluster.

Deploying and configuring these resources requires admin permissions in your CoreWeave account.

Deploy a CKS cluster​

Follow the CoreWeave docs to create a CKS cluster.

Anyscale recommends that you follow the steps to configure a Virtual Private Cloud (VPC).

Create a cluster node pool​

Configure a node pool to add nodes to your CKS cluster and run workloads on Anyscale. You can optionally configure labels and taints on the node pool to control pod scheduling, and you can add or edit them after pool creation from the CoreWeave console. You can also apply labels and taints directly to individual nodes. See Create a Node Pool.

Create and configure an AI object storage bucket​

Create a dedicated CoreWeave AI object storage bucket to use as the default system storage for your Anyscale deployment. Capture the following values as you create your bucket:

Create a permanent access key using an identity that has the Object Storage Admin IAM role. You use this access key in the next step to enable CORS. Anyscale uses this access key and secret to mount your bucket to your CKS cluster.

Follow the CoreWeave docs to Get Started with AI Object Storage.

Enable CORS for your bucket​

Anyscale requires that you update your bucket to enable CORS. CoreWeave AI object storage buckets are S3 compatible.

Add the following to your ~/.aws/config file to create a new profile for CoreWeave with the aws CLI:

[profile coreweave]
aws_access_key_id=<your_access_key_id>
aws_secret_access_key=<your_access_key_secret>
region=<your_bucket_availability_zone>
endpoint_url=https://cwobject.com
s3 =
    addressing_style = virtual

Run the following command to confirm your profile is configured correctly:

aws s3 ls s3://<your_bucket_name> --profile coreweave

Create a file named cors.json with the following contents:

{
    "CORSRules": [
        {
        "AllowedOrigins": ["https://*.anyscale.com"],
        "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"],
        "AllowedHeaders": ["*"],
        "ExposeHeaders": ["Accept-Ranges", "Content-Range", "Content-Length"],
        "MaxAgeSeconds": 3600
        }
    ]
}

Run the following command to add these rules to your bucket:

aws s3api put-bucket-cors \
    --bucket <your_bucket_name> \
    --cors-configuration file://cors.json \
    --profile coreweave

Create an API access token and kubeconfig file​

Follow the CoreWeave docs to Create a new API Access Token. Create and download a kubeconfig file and store it on your local machine.

Step 2: Configure ingress for CKS​

Anyscale requires ingress for core platform features such as services and workspaces. Complete the following steps to install nginx-ingress for your CKS cluster using Helm.

Create an ingress configuration YAML​

Create a file named values-nginx.yaml with the following contents:

controller:
    annotations:
        service.beta.kubernetes.io/coreweave-load-balancer-type: "public"
    service:
        type: LoadBalancer
    allowSnippetAnnotations: true
    config:
        enable-underscores-in-headers: "true"
        annotations-risk-level: "Critical"
warning

Anyscale services require allowSnippetAnnotations: true and annotations-risk-level: "Critical", which override ingress-nginx's default protections against CVE-2021-25742. Restrict ingress resource permissions to trusted users only.

Create the ingress controller​

Run the following commands to create your ingress controller:

helm repo add nginx https://kubernetes.github.io/ingress-nginx
helm upgrade ingress-nginx nginx/ingress-nginx \
    --version 4.12.1 \
    --namespace ingress-nginx \
    --values values-nginx.yaml \
    --create-namespace \
    --install

Step 3: Create an Anyscale service account API key​

Anyscale uses a service account API key to establish a trust relationship between the Anyscale operator and the control plane.

Run the following command to log in to your Anyscale organization:

anyscale login

Provide a human-readable name for the service account. Anyscale recommends that you use a unique service account and API key for this configuration.

anyscale service-account create --name <your_service_account_name>

Anyscale returns an API key for your service account. Copy the value to the following field:

note

Anyscale service account API keys don't expire. Delete an API key or service account to revoke access. See Anyscale service accounts and Manage API keys.

Step 4: Create your Anyscale cloud​

Provide a human-readable name for your Anyscale cloud. You use this name to identify the cloud when using the Anyscale console, CLI, and SDK. This name also appears in the URL for Anyscale service API endpoints.

Run the following command to create an Anyscale cloud for your CKS cluster:

anyscale cloud register --name <your_cloud_name> \
     --provider generic \
     --region <your_bucket_availability_zone> \
     --compute-stack k8s \
     --cloud-storage-bucket-name s3://<your_bucket_name> \
     --cloud-storage-bucket-endpoint https://cwobject.com

The output includes the ID for the configured cloud resource in your newly created cloud in the field CloudDeploymentId. Copy the value to the following field:

Step 5: Configure and install the Anyscale operator​

Create a values.yaml file that describes your cloud and CoreWeave resources and use Helm to install the Anyscale operator to your CKS cluster.

Create a values YAML file​

Create a values.yaml file. The following example uses the values you provided throughout this tutorial to create a minimum viable configuration for the Anyscale operator. Anyscale recommends that you start with a minimal spec and then customize once you've verified functional behavior.

global:
  cloudProvider: generic
  cloudDeploymentId: <your_cloud_resource_id>
  auth:
    anyscaleCliToken: <your_api_key>

workloads:
  serviceAccount:
    name: anyscale-operator

credentialMount:
  aws:
    enabled: true
    createSecret:
      create: true
      accessKeyId: <your_access_key_id>
      secretAccessKey: <your_access_key_secret>
      endpointUrl: https://cwobject.com
      addressingStyle: virtual
      signatureVersion: s3v4

Install the Anyscale operator on CKS​

Run the following commands to install the Anyscale operator with Helm using your values.yaml file:

helm repo add anyscale https://anyscale.github.io/helm-charts
helm repo update anyscale
helm upgrade anyscale-operator anyscale/anyscale-operator \
    --namespace anyscale-operator \
    -f values.yaml \
    --create-namespace \
    -i

Step 6: Verify your Anyscale cloud​

It might take several minutes for your Anyscale cloud to be ready to use. Run the following command to verify that your newly created Anyscale cloud is fully functional and ready to use.

anyscale job submit --cloud <your_cloud_name> --working-dir https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip -- python hello_world.py