--- title: "Deploy Anyscale on Nebius" description: "Learn how to deploy Anyscale using the Kubernetes stack on Nebius AI Cloud." --- # Deploy Anyscale on Nebius This page describes how to deploy a new Anyscale cloud on [Managed Service for Kubernetes](https://docs.nebius.com/kubernetes) on Nebius AI Cloud, using Nebius [Object Storage](https://docs.nebius.com/object-storage/quickstart) as S3-compatible storage for cluster logs, snapshots, and artifacts. ## Requirements Before you start, ensure that you meet the following requirements: - You're an Anyscale organization owner. - You have an Anyscale account and a [Nebius AI Cloud](https://console.nebius.com/) account. - The following CLI tools are installed: Anyscale CLI, [Nebius AI Cloud CLI](https://docs.nebius.com/cli/install), AWS CLI, `kubectl`, `helm`, and `jq`. ## Step 1: Create and configure Nebius resources Anyscale requires the following Nebius resources and configurations: - A Managed Service for Kubernetes cluster. - A CPU node group. - A GPU node group (optional, for GPU workloads). - An Object Storage bucket (S3-compatible) for Anyscale system storage. - A kubeconfig that can access your cluster. You need admin permissions in your Nebius account to create and configure these resources. ### Create a Kubernetes cluster Follow the Nebius docs to [create your first cluster and node group](https://docs.nebius.com/kubernetes/quickstart). Configure a public endpoint for the control plane so you can reach it from your machine and from Anyscale. Copy the cluster ID from the Nebius console or CLI output for later use with `kubectl` and `helm`: Nebius cluster ID `catbg5...` ### Create a CPU node group Create a CPU node group for general workloads such as the Anyscale operator and Ray head nodes. In the [Nebius node group docs](https://docs.nebius.com/kubernetes/node-groups/manage), create a node group with a CPU platform and preset that matches your needs (for example, `cpu-d3` with the `4vcpu-16gb` preset). ### Create a GPU node group (optional) If you need GPUs, add a GPU node group to the same cluster. See [Creating and modifying node groups](https://docs.nebius.com/kubernetes/node-groups/manage) and [Setting up GPUs](https://docs.nebius.com/kubernetes/gpu/set-up). ### Create and configure a Nebius Object Storage bucket Create a Nebius Object Storage bucket to use as default system storage for your Anyscale cloud. Anyscale uses this bucket to store assets such as logs, workspace snapshots, and checkpoints. Capture the following Nebius identifiers for use throughout this guide. You can find your project ID and tenant ID in the [Nebius console](https://console.nebius.com/). Nebius project ID `project-e00...` Nebius tenant ID `tenant-e00...` Complete the following steps: 1. **Set the project and create a service account.** From the Nebius CLI (after `nebius profile create` and signing in), create a service account and add it to the `editors` group so it can create and manage Object Storage buckets: ```bash nebius config set parent-id export NB_SA_ID=$(nebius iam service-account create \ --name object-storage-sa --format json \ | jq -r '.metadata.id') export NB_EDITORS_GROUP_ID=$(nebius iam group get-by-name \ --name editors --parent-id --format json \ | jq -r '.metadata.id') nebius iam group-membership create \ --parent-id $NB_EDITORS_GROUP_ID \ --member-id $NB_SA_ID ``` 2. **Create an access key and capture credentials.** Nebius access keys don't expire by default. For production deployments, Anyscale recommends setting an expiration with the `--expires-at` flag and rotating keys before they expire. ```bash export NB_ACCESS_KEY_ID=$(nebius iam access-key create \ --account-service-account-id $NB_SA_ID \ --description 'Anyscale Object Storage' \ --format json | jq -r '.resource_id') export NB_ACCESS_KEY_AWS_ID=$(nebius iam access-key get-by-id \ --id $NB_ACCESS_KEY_ID \ --format json | jq -r '.status.aws_access_key_id') export NB_SECRET_ACCESS_KEY=$(nebius iam access-key get-secret-once \ --id $NB_ACCESS_KEY_ID --format json \ | jq -r '.secret') echo NB_ACCESS_KEY_AWS_ID; echo NB_SECRET_ACCESS_KEY; ``` Capture the output for your Nebius access key ID and your access key secret in the following fields: Nebius access key ID (AWS-like) `YCAJ...` Nebius access key secret `YCP...` 3. **Configure the AWS CLI for Nebius.** Nebius Object Storage is S3-compatible. In this step, you add a `nebius` profile to your `~/.aws/config` file so you can use the AWS CLI to manage your Nebius Object Storage bucket. Choose a name for your bucket and provide the [Nebius region](https://docs.nebius.com/overview/regions) where your cluster is located, for example `eu-north1`. Nebius bucket name `my-anyscale-bucket` Nebius region `eu-north1` ```text [profile nebius] aws_access_key_id= aws_secret_access_key= region= endpoint_url=https://storage..nebius.cloud:443 s3 = addressing_style = virtual ``` 4. **Create the bucket on Nebius Object Storage.** This command uses the AWS CLI as an S3-compatible client. Because the `nebius` profile points at the Nebius endpoint, the bucket is created on Nebius — not on AWS. ```bash aws s3 mb s3:// --profile nebius ``` 5. **Enable CORS for your bucket.** Anyscale requires CORS on your Object Storage bucket. Create a file named `cors.json` with the following contents: ```json { "CORSRules": [ { "AllowedOrigins": ["https://*.anyscale.com"], "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"], "AllowedHeaders": ["*"], "ExposeHeaders": ["Accept-Ranges", "Content-Range", "Content-Length"], "MaxAgeSeconds": 3600 } ] } ``` Apply the CORS configuration to your bucket: ```bash aws s3api put-bucket-cors \ --bucket \ --cors-configuration file://cors.json \ --profile nebius ``` ### Connect to the cluster Create a kubeconfig so you can run `kubectl` and `helm` against your cluster: ```bash nebius mk8s cluster get-credentials --id --external ``` Verify that you can reach the cluster and that your bucket is accessible: ```bash kubectl get nodes aws s3 ls s3:// --profile nebius ``` ## Step 2: Configure Envoy Gateway for your cluster Anyscale requires an ingress controller for features such as workspaces and services. These steps install Envoy Gateway v1.7.0 on your Nebius cluster. :::note Other gateway and ingress controllers are supported. See [Ingress and gateway controllers](/clouds/kubernetes.md#ingress-support) for all supported options. ::: ### Install Envoy Gateway Run the following commands to install Envoy Gateway v1.7.0: ```bash helm install eg oci://docker.io/envoyproxy/gateway-helm \ --version v1.7.0 \ --namespace envoy-gateway-system \ --create-namespace ``` ```bash kubectl wait --for=condition=available deployment/envoy-gateway \ -n envoy-gateway-system --timeout=120s ``` ### Create an EnvoyProxy resource Create a file named `envoyproxy.yaml` with the following contents: ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: envoy-proxy namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: LoadBalancer ``` Apply the resource: ```bash kubectl apply -f envoyproxy.yaml ``` ### Create a GatewayClass Create a file named `gatewayclass.yaml` with the following contents: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: eg spec: controllerName: gateway.envoyproxy.io/gatewayclass-controller parametersRef: group: gateway.envoyproxy.io kind: EnvoyProxy name: envoy-proxy namespace: envoy-gateway-system ``` Apply the resource: ```bash kubectl apply -f gatewayclass.yaml ``` ## Step 3: Create an Anyscale service account API key Anyscale uses a service account API key to establish a trust relationship between the Anyscale operator and the control plane. Run the following command to log in to your Anyscale organization: ```bash anyscale login ``` Run the following command to confirm your identity and permissions. You must have the `owner` role to register a new cloud. ```bash anyscale auth show ``` Copy your organization ID from the output: Anyscale organization ID `org_xxxxx` Provide a human-readable name for the service account. Anyscale recommends that you use a unique service account and API key for this configuration. Anyscale service account name `nebius-operator-sa` ```bash anyscale service-account create --name ``` Anyscale returns an API key for your service account. Copy the value to the following field: Anyscale API key `aph0_xxxxx` :::note Anyscale service account API keys don't expire. Delete an API key or service account to revoke access. See [Service accounts](/auth/service-accounts.md) and [API keys](/auth/api-keys.md). ::: ## Step 4: Create your Anyscale cloud Register a new Anyscale cloud that points at your Kubernetes cluster and Nebius Object Storage bucket. Anyscale cloud name `nebius-dev` Run the following command: ```bash anyscale cloud register --name \ --provider generic \ --region \ --compute-stack k8s \ --cloud-storage-bucket-name s3:// \ --cloud-storage-bucket-endpoint https://storage..nebius.cloud:443 ``` The command output includes the **cloud deployment ID**. Copy it: Cloud resource ID `cldrsrc_xxxxx` ### Grant the service account access to your cloud The Anyscale operator authenticates as the service account you created in the previous step. Grant the service account collaborator access to your cloud so the operator can manage resources without a 403 error. Create a file named `collaborators.yaml` with the following contents: ```yaml collaborators: - email: "@org-.serviceaccount.com" permission_level: "collaborator" ``` Run the following command to grant access: ```bash anyscale cloud add-collaborators --cloud --users-file collaborators.yaml ``` ## Step 5: Configure and install the Anyscale operator Create a Helm values file for the Anyscale operator and install it into your cluster. ### Create the Gateway The `anyscale-operator` namespace must exist before you create the Gateway. Create it with the following command. If the namespace already exists, this command returns an error; that's expected and you can proceed. ```bash kubectl create namespace anyscale-operator ``` Create a file named `gateway.yaml`. Enter your cloud deployment ID from step 4. Hyphens are applied automatically for the certificate name: Cloud resource ID (for certificates) `cldrsrc_1234` ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: gateway namespace: anyscale-operator spec: gatewayClassName: eg listeners: - name: http port: 80 protocol: HTTP allowedRoutes: namespaces: from: All - name: https port: 443 protocol: HTTPS hostname: '*.i.anyscaleuserdata.com' tls: mode: Terminate certificateRefs: - kind: Secret name: anyscale--certificate allowedRoutes: namespaces: from: All - name: https-session port: 443 protocol: HTTPS hostname: '*.s.anyscaleuserdata.com' tls: mode: Terminate certificateRefs: - kind: Secret name: anyscale-svc--certificate allowedRoutes: namespaces: from: All ``` Apply the Gateway and retrieve its external address: ```bash kubectl apply -f gateway.yaml ``` ```bash kubectl get gateway gateway -n anyscale-operator \ -o jsonpath='{.status.addresses[0].value}' ``` Record the Gateway address in the following field: Gateway address `lb.example.com` ### Create a values file Create a `values.yaml` file. The following example uses the values you provided throughout this guide to create a minimum viable configuration for the Anyscale operator. Anyscale recommends that you start with a minimal spec and then customize after you've verified functional behavior. ```yaml global: cloudProvider: generic cloudDeploymentId: auth: anyscaleCliToken: aws: region: workloads: serviceAccount: name: anyscale-operator credentialMount: aws: enabled: true createSecret: create: true accessKeyId: secretAccessKey: endpointUrl: https://storage..nebius.cloud:443 addressingStyle: virtual signatureVersion: s3v4 networking: gateway: enabled: true name: "gateway" namespace: "anyscale-operator" apiVersion: "gateway.networking.k8s.io/v1" hostname: "" # ip: "" # Use this instead of hostname if the LB provides an IP address ``` :::warning Keep your `values.yaml` and any secrets (tokens, keys) secure. Don't commit them to version control. ::: ### Install the Anyscale operator Run the following commands: ```bash helm repo add anyscale https://anyscale.github.io/helm-charts helm repo update anyscale helm upgrade anyscale-operator anyscale/anyscale-operator \ --namespace anyscale-operator \ -f values.yaml \ --create-namespace \ -i ``` ## Step 6: Verify your Anyscale cloud It can take a few minutes for the operator to become ready. Verify the cloud: ```bash anyscale cloud verify --name ``` Optionally, submit a small job to confirm end-to-end behavior: ```bash anyscale job submit --cloud \ --working-dir https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip \ -- python hello_world.py ``` ## Next steps - [Configure the Helm chart for the Anyscale operator](/clouds/kubernetes/configure-helm.md) for instance types, autoscaling, and networking. - [Helm configuration reference](/clouds/kubernetes/helm-ref.md) for all operator parameters. - [Deploy Anyscale on Kubernetes](/clouds/kubernetes.md) for general Kubernetes deployment requirements and options. --- Previous: [Deploy Anyscale on CoreWeave](/clouds/kubernetes/coreweave.md) | Next: [Deploy on non-managed Kubernetes](/clouds/kubernetes/generic.md)