--- title: "Deploy Anyscale on CoreWeave" description: "Learn how to deploy Anyscale using the Kubernetes stack on CoreWeave." --- # Deploy Anyscale on CoreWeave This page provides an overview of deploying a new Anyscale cloud on CoreWeave Kubernetes Service (CKS). To learn more about CKS, see [Introduction to CoreWeave Kubernetes Service](https://docs.coreweave.com/products/cks). ## Requirements Before you start, ensure the following: - You are an Anyscale organization owner. - The following CLI tools are installed: Anyscale CLI, `aws` CLI, `kubectl`, `helm`. ## Step 1: Create and configure CoreWeave resources Anyscale requires the following CoreWeave resources and configurations. - A CKS cluster. - A node pool. - A CoreWeave AI object storage bucket with CORS enabled. - A kubeconfig file with a valid CoreWeave API access token for your CKS cluster. Deploying and configuring these resources requires admin permissions in your CoreWeave account. ### Deploy a CKS cluster Follow the CoreWeave docs to [create a CKS cluster](https://docs.coreweave.com/products/cks/clusters/create). Anyscale recommends that you follow the steps to configure a Virtual Private Cloud (VPC). ### Create a cluster node pool Configure a node pool to add nodes to your CKS cluster and run workloads on Anyscale. You can optionally configure labels and taints on the node pool to control pod scheduling, and you can add or edit them after pool creation from the CoreWeave console. You can also apply labels and taints directly to individual nodes. See [Create a Node Pool](https://docs.coreweave.com/products/cks/nodes/create). ### Create and configure an AI object storage bucket Create a dedicated CoreWeave AI object storage bucket to use as the default system storage for your Anyscale deployment. Capture the following values as you create your bucket: CoreWeave bucket name `my-bucket` CoreWeave bucket availability zone `US-WEST-01A` CoreWeave access key ID `xxxx-xxxx-xxxx` CoreWeave access key secret `xxxx-xxxx-xxxx` Create a permanent access key using an identity that has the Object Storage Admin IAM role. You use this access key in the next step to enable CORS. Anyscale uses this access key and secret to mount your bucket to your CKS cluster. Follow the CoreWeave docs to [Get Started with AI Object Storage](https://docs.coreweave.com/products/storage/object-storage/get-started-caios). #### Enable CORS for your bucket Anyscale requires that you update your bucket to enable CORS. CoreWeave AI object storage buckets are S3 compatible. Add the following to your `~/.aws/config` file to create a new profile for CoreWeave with the `aws` CLI: ```bash [profile coreweave] aws_access_key_id= aws_secret_access_key= region= endpoint_url=https://cwobject.com s3 = addressing_style = virtual ``` Run the following command to confirm your profile is configured correctly: ```bash aws s3 ls s3:// --profile coreweave ``` Create a file named `cors.json` with the following contents: ```json { "CORSRules": [ { "AllowedOrigins": ["https://*.anyscale.com"], "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"], "AllowedHeaders": ["*"], "ExposeHeaders": ["Accept-Ranges", "Content-Range", "Content-Length"], "MaxAgeSeconds": 3600 } ] } ``` Run the following command to add these rules to your bucket: ```bash aws s3api put-bucket-cors \ --bucket \ --cors-configuration file://cors.json \ --profile coreweave ``` ### Create an API access token and kubeconfig file Follow the CoreWeave docs to [Create a new API Access Token](https://docs.coreweave.com/products/cks/auth-access/manage-api-access-tokens#create-a-new-api-access-token). Create and download a `kubeconfig` file and store it on your local machine. ## Step 2: Configure Envoy Gateway for CKS Anyscale requires an ingress controller for core platform features such as services and workspaces. These steps install Envoy Gateway v1.7.0 on your CKS cluster. :::note Other gateway and ingress controllers are supported. See [Ingress and gateway controllers](/clouds/kubernetes.md#ingress-support) for all supported options. ::: ### Install Envoy Gateway Run the following commands to install Envoy Gateway v1.7.0: ```bash helm install eg oci://docker.io/envoyproxy/gateway-helm \ --version v1.7.0 \ --namespace envoy-gateway-system \ --create-namespace ``` ```bash kubectl wait --for=condition=available deployment/envoy-gateway \ -n envoy-gateway-system --timeout=120s ``` ### Create an EnvoyProxy resource Create a file named `envoyproxy.yaml` with the following contents: ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: envoy-proxy namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: LoadBalancer annotations: service.beta.kubernetes.io/coreweave-load-balancer-type: "public" ``` Apply the resource: ```bash kubectl apply -f envoyproxy.yaml ``` ### Create a GatewayClass Create a file named `gatewayclass.yaml` with the following contents: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: eg spec: controllerName: gateway.envoyproxy.io/gatewayclass-controller parametersRef: group: gateway.envoyproxy.io kind: EnvoyProxy name: envoy-proxy namespace: envoy-gateway-system ``` Apply the resource: ```bash kubectl apply -f gatewayclass.yaml ``` ## Step 3: Create an Anyscale service account API key Anyscale uses a service account API key to establish a trust relationship between the Anyscale operator and the control plane. Run the following command to log in to your Anyscale organization: ```bash anyscale login ``` Run the following command to confirm your identity and permissions. You must have the `owner` role to register a new cloud. ```bash anyscale auth show ``` Copy your organization ID from the output: Anyscale organization ID `org_xxxxx` Provide a human-readable name for the service account. Anyscale recommends that you use a unique service account and API key for this configuration. Anyscale service account name `cw-operator-sa` ```bash anyscale service-account create --name ``` Anyscale returns an API key for your service account. Copy the value to the following field: Anyscale API key `aph0_xxxxx` :::note Anyscale service account API keys don't expire. Delete an API key or service account to revoke access. See [Anyscale service accounts](/auth/service-accounts.md) and [Manage API keys](/auth/api-keys.md). ::: ## Step 4: Create your Anyscale cloud Provide a human-readable name for your Anyscale cloud. You use this name to identify the cloud when using the Anyscale console, CLI, and SDK. This name also appears in the URL for Anyscale service API endpoints. Anyscale cloud name `cks-dev` Run the following command to create an Anyscale cloud for your CKS cluster: ```bash anyscale cloud register --name \ --provider generic \ --region \ --compute-stack k8s \ --cloud-storage-bucket-name s3:// \ --cloud-storage-bucket-endpoint https://cwobject.com ``` The output includes the ID for the configured cloud resource in your newly created cloud in the field `CloudDeploymentId`. Copy the value to the following field: Cloud resource ID `cldrsrc_xxxxx` ### Grant the service account access to your cloud The Anyscale operator authenticates as the service account you created in the previous step. Grant the service account collaborator access to your cloud so the operator can manage resources without a 403 error. Create a file named `collaborators.yaml` with the following contents: ```yaml collaborators: - email: "@org-.serviceaccount.com" permission_level: "collaborator" ``` Run the following command to grant access: ```bash anyscale cloud add-collaborators --cloud --users-file collaborators.yaml ``` ## Step 5: Configure and install the Anyscale operator Create a `values.yaml` file that describes your cloud and CoreWeave resources and use Helm to install the Anyscale operator to your CKS cluster. ### Create the Gateway The `anyscale-operator` namespace must exist before you create the Gateway. Create it with the following command. If the namespace already exists, this command returns an error; that's expected and you can proceed. ```bash kubectl create namespace anyscale-operator ``` Create a file named `gateway.yaml`. Enter your cloud deployment ID from step 4. Hyphens are applied automatically for the certificate name: Cloud resource ID (for certificates) `cldrsrc_1234` ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: gateway namespace: anyscale-operator spec: gatewayClassName: eg listeners: - name: http port: 80 protocol: HTTP allowedRoutes: namespaces: from: All - name: https port: 443 protocol: HTTPS hostname: '*.i.anyscaleuserdata.com' tls: mode: Terminate certificateRefs: - kind: Secret name: anyscale--certificate allowedRoutes: namespaces: from: All - name: https-session port: 443 protocol: HTTPS hostname: '*.s.anyscaleuserdata.com' tls: mode: Terminate certificateRefs: - kind: Secret name: anyscale-svc--certificate allowedRoutes: namespaces: from: All ``` Apply the Gateway and retrieve its external address: ```bash kubectl apply -f gateway.yaml ``` ```bash kubectl get gateway gateway -n anyscale-operator \ -o jsonpath='{.status.addresses[0].value}' ``` Record the Gateway address in the following field: Gateway address `lb.example.com` ### Create a values YAML file Create a `values.yaml` file. The following example uses the values you provided throughout this tutorial to create a minimum viable configuration for the Anyscale operator. Anyscale recommends that you start with a minimal spec and then customize once you've verified functional behavior. ```yaml global: cloudProvider: generic cloudDeploymentId: auth: anyscaleCliToken: workloads: serviceAccount: name: anyscale-operator credentialMount: aws: enabled: true createSecret: create: true accessKeyId: secretAccessKey: endpointUrl: https://cwobject.com addressingStyle: virtual signatureVersion: s3v4 networking: gateway: enabled: true name: "gateway" namespace: "anyscale-operator" apiVersion: "gateway.networking.k8s.io/v1" hostname: "" # ip: "" # Use this instead of hostname if the LB provides an IP address ``` ### Install the Anyscale operator on CKS Run the following commands to install the Anyscale operator with Helm using your `values.yaml` file: ```bash helm repo add anyscale https://anyscale.github.io/helm-charts helm repo update anyscale helm upgrade anyscale-operator anyscale/anyscale-operator \ --namespace anyscale-operator \ -f values.yaml \ --create-namespace \ -i ``` ## Step 6: Verify your Anyscale cloud It might take several minutes for your Anyscale cloud to be ready to use. Run the following command to verify that your newly created Anyscale cloud is fully functional and ready to use. ```bash anyscale job submit --cloud --working-dir https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip -- python hello_world.py ``` --- Previous: [Anyscale on Kubernetes](/clouds/kubernetes.md) | Next: [Deploy Anyscale on Nebius](/clouds/kubernetes/nebius.md)