Skip to main content

Deploy Anyscale on Nebius

View Markdown

Deploy Anyscale on Nebius

This page describes how to deploy a new Anyscale cloud on Managed Service for Kubernetes on Nebius AI Cloud, using Nebius Object Storage as S3-compatible storage for cluster logs, snapshots, and artifacts.

Requirements​

Before you start, ensure that you meet the following requirements:

  • You're an Anyscale organization owner.
  • You have an Anyscale account and a Nebius AI Cloud account.
  • The following CLI tools are installed: Anyscale CLI, Nebius AI Cloud CLI, AWS CLI, kubectl, helm, and jq.

Step 1: Create and configure Nebius resources​

Anyscale requires the following Nebius resources and configurations:

  • A Managed Service for Kubernetes cluster.
  • A CPU node group.
  • A GPU node group (optional, for GPU workloads).
  • An Object Storage bucket (S3-compatible) for Anyscale system storage.
  • A kubeconfig that can access your cluster.

You need admin permissions in your Nebius account to create and configure these resources.

Create a Kubernetes cluster​

Follow the Nebius docs to create your first cluster and node group. Configure a public endpoint for the control plane so you can reach it from your machine and from Anyscale.

Copy the cluster ID from the Nebius console or CLI output for later use with kubectl and helm:

Create a CPU node group​

Create a CPU node group for general workloads such as the Anyscale operator and Ray head nodes. In the Nebius node group docs, create a node group with a CPU platform and preset that matches your needs (for example, cpu-d3 with the 4vcpu-16gb preset).

Create a GPU node group (optional)​

If you need GPUs, add a GPU node group to the same cluster. See Creating and modifying node groups and Setting up GPUs.

Create and configure a Nebius Object Storage bucket​

Create a Nebius Object Storage bucket to use as default system storage for your Anyscale cloud. Anyscale uses this bucket to store assets such as logs, workspace snapshots, and checkpoints.

Capture the following Nebius identifiers for use throughout this guide. You can find your project ID and tenant ID in the Nebius console.

Complete the following steps:

  1. Set the project and create a service account. From the Nebius CLI (after nebius profile create and signing in), create a service account and add it to the editors group so it can create and manage Object Storage buckets:

    nebius config set parent-id <your_project_id>

    export NB_SA_ID=$(nebius iam service-account create \
      --name object-storage-sa --format json \
      | jq -r '.metadata.id')

    export NB_EDITORS_GROUP_ID=$(nebius iam group get-by-name \
      --name editors --parent-id <your_tenant_id> --format json \
      | jq -r '.metadata.id')

    nebius iam group-membership create \
      --parent-id $NB_EDITORS_GROUP_ID \
      --member-id $NB_SA_ID

  2. Create an access key and capture credentials. Nebius access keys don't expire by default. For production deployments, Anyscale recommends setting an expiration with the --expires-at flag and rotating keys before they expire.

    export NB_ACCESS_KEY_ID=$(nebius iam access-key create \
      --account-service-account-id $NB_SA_ID \
      --description 'Anyscale Object Storage' \
      --format json | jq -r '.resource_id')

    export NB_ACCESS_KEY_AWS_ID=$(nebius iam access-key get-by-id \
      --id $NB_ACCESS_KEY_ID \
      --format json | jq -r '.status.aws_access_key_id')

    export NB_SECRET_ACCESS_KEY=$(nebius iam access-key get-secret-once \
      --id $NB_ACCESS_KEY_ID --format json \
      | jq -r '.secret')

    echo NB_ACCESS_KEY_AWS_ID;
    echo NB_SECRET_ACCESS_KEY;

    Capture the output for your Nebius access key ID and your access key secret in the following fields:

  3. Configure the AWS CLI for Nebius. Nebius Object Storage is S3-compatible. In this step, you add a nebius profile to your ~/.aws/config file so you can use the AWS CLI to manage your Nebius Object Storage bucket.

    Choose a name for your bucket and provide the Nebius region where your cluster is located, for example eu-north1.

    [profile nebius]
    aws_access_key_id=<access_key_id>
    aws_secret_access_key=<access_key_secret>
    region=<your_bucket_region>
    endpoint_url=https://storage.<your_bucket_region>.nebius.cloud:443
    s3 =
        addressing_style = virtual

  4. Create the bucket on Nebius Object Storage. This command uses the AWS CLI as an S3-compatible client. Because the nebius profile points at the Nebius endpoint, the bucket is created on Nebius — not on AWS.

    aws s3 mb s3://<your_bucket_name> --profile nebius

  5. Enable CORS for your bucket. Anyscale requires CORS on your Object Storage bucket. Create a file named cors.json with the following contents:

    {
        "CORSRules": [
            {
            "AllowedOrigins": ["https://*.anyscale.com"],
            "AllowedMethods": ["GET", "PUT", "POST", "DELETE", "HEAD"],
            "AllowedHeaders": ["*"],
            "ExposeHeaders": ["Accept-Ranges", "Content-Range", "Content-Length"],
            "MaxAgeSeconds": 3600
            }
        ]
    }

    Apply the CORS configuration to your bucket:

    aws s3api put-bucket-cors \
        --bucket <your_bucket_name> \
        --cors-configuration file://cors.json \
        --profile nebius

Connect to the cluster​

Create a kubeconfig so you can run kubectl and helm against your cluster:

nebius mk8s cluster get-credentials --id <your_cluster_id> --external

Verify that you can reach the cluster and that your bucket is accessible:

kubectl get nodes
aws s3 ls s3://<your_bucket_name> --profile nebius

Step 2: Configure Envoy Gateway for your cluster​

Anyscale requires an ingress controller for features such as workspaces and services. These steps install Envoy Gateway v1.7.0 on your Nebius cluster.

note

Other gateway and ingress controllers are supported. See Ingress and gateway controllers for all supported options.

Install Envoy Gateway​

Run the following commands to install Envoy Gateway v1.7.0:

helm install eg oci://docker.io/envoyproxy/gateway-helm \
  --version v1.7.0 \
  --namespace envoy-gateway-system \
  --create-namespace
kubectl wait --for=condition=available deployment/envoy-gateway \
  -n envoy-gateway-system --timeout=120s

Create an EnvoyProxy resource​

Create a file named envoyproxy.yaml with the following contents:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-proxy
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: LoadBalancer

Apply the resource:

kubectl apply -f envoyproxy.yaml

Create a GatewayClass​

Create a file named gatewayclass.yaml with the following contents:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: envoy-proxy
    namespace: envoy-gateway-system

Apply the resource:

kubectl apply -f gatewayclass.yaml

Step 3: Create an Anyscale service account API key​

Anyscale uses a service account API key to establish a trust relationship between the Anyscale operator and the control plane.

Run the following command to log in to your Anyscale organization:

anyscale login

Run the following command to confirm your identity and permissions. You must have the owner role to register a new cloud.

anyscale auth show

Copy your organization ID from the output:

Provide a human-readable name for the service account. Anyscale recommends that you use a unique service account and API key for this configuration.

anyscale service-account create --name <your_service_account_name>

Anyscale returns an API key for your service account. Copy the value to the following field:

note

Anyscale service account API keys don't expire. Delete an API key or service account to revoke access. See Service accounts and API keys.

Step 4: Create your Anyscale cloud​

Register a new Anyscale cloud that points at your Kubernetes cluster and Nebius Object Storage bucket.

Run the following command:

anyscale cloud register --name <your_cloud_name> \
  --provider generic \
  --region <your_bucket_region> \
  --compute-stack k8s \
  --cloud-storage-bucket-name s3://<your_bucket_name> \
  --cloud-storage-bucket-endpoint https://storage.<your_bucket_region>.nebius.cloud:443

The command output includes the cloud deployment ID. Copy it:

Grant the service account access to your cloud​

The Anyscale operator authenticates as the service account you created in the previous step. Grant the service account collaborator access to your cloud so the operator can manage resources without a 403 error.

Create a file named collaborators.yaml with the following contents:

collaborators:
  - email: "<your_service_account_name>@org-<your_org_id>.serviceaccount.com"
    permission_level: "collaborator"

Run the following command to grant access:

anyscale cloud add-collaborators --cloud <your_cloud_name> --users-file collaborators.yaml

Step 5: Configure and install the Anyscale operator​

Create a Helm values file for the Anyscale operator and install it into your cluster.

Create the Gateway​

The anyscale-operator namespace must exist before you create the Gateway. Create it with the following command. If the namespace already exists, this command returns an error; that's expected and you can proceed.

kubectl create namespace anyscale-operator

Create a file named gateway.yaml.

Enter your cloud deployment ID from step 4. Hyphens are applied automatically for the certificate name:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: gateway
  namespace: anyscale-operator
spec:
  gatewayClassName: eg
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: All
  - name: https
    port: 443
    protocol: HTTPS
    hostname: '*.i.anyscaleuserdata.com'
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: anyscale-<cloud-resource-id>-certificate
    allowedRoutes:
      namespaces:
        from: All
  - name: https-session
    port: 443
    protocol: HTTPS
    hostname: '*.s.anyscaleuserdata.com'
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: anyscale-svc-<cloud-resource-id>-certificate
    allowedRoutes:
      namespaces:
        from: All

Apply the Gateway and retrieve its external address:

kubectl apply -f gateway.yaml
kubectl get gateway gateway -n anyscale-operator \
  -o jsonpath='{.status.addresses[0].value}'

Record the Gateway address in the following field:

Create a values file​

Create a values.yaml file. The following example uses the values you provided throughout this guide to create a minimum viable configuration for the Anyscale operator. Anyscale recommends that you start with a minimal spec and then customize after you've verified functional behavior.

global:
  cloudProvider: generic
  cloudDeploymentId: <your_cloud_resource_id>
  auth:
    anyscaleCliToken: <your_anyscale_api_key>
  aws:
    region: <your_bucket_region>

workloads:
  serviceAccount:
    name: anyscale-operator

credentialMount:
  aws:
    enabled: true
    createSecret:
      create: true
      accessKeyId: <access_key_id>
      secretAccessKey: <access_key_secret>
      endpointUrl: https://storage.<your_bucket_region>.nebius.cloud:443
      addressingStyle: virtual
      signatureVersion: s3v4

networking:
  gateway:
    enabled: true
    name: "gateway"
    namespace: "anyscale-operator"
    apiVersion: "gateway.networking.k8s.io/v1"
    hostname: "<gateway-address>"
    # ip: "<gateway-ip>"  # Use this instead of hostname if the LB provides an IP address
warning

Keep your values.yaml and any secrets (tokens, keys) secure. Don't commit them to version control.

Install the Anyscale operator​

Run the following commands:

helm repo add anyscale https://anyscale.github.io/helm-charts
helm repo update anyscale
helm upgrade anyscale-operator anyscale/anyscale-operator \
  --namespace anyscale-operator \
  -f values.yaml \
  --create-namespace \
  -i

Step 6: Verify your Anyscale cloud​

It can take a few minutes for the operator to become ready. Verify the cloud:

anyscale cloud verify --name <your_cloud_name>

Optionally, submit a small job to confirm end-to-end behavior:

anyscale job submit --cloud <your_cloud_name> \
  --working-dir https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip \
  -- python hello_world.py

Next steps​