Skip to main content

Set up Anyscale on AKS

View Markdown

Set up Anyscale on AKS

This page describes how to set up an Anyscale cloud on Azure Kubernetes Service (AKS) using the anyscale cloud setup command. This flow creates a working baseline so you can run jobs and services. You can customize networking, node pools, and security after setup.

important

The anyscale cloud setup CLI flow for Kubernetes is in beta. Anyscale recommends this flow for self-service onboarding. Clouds deployed with this flow are production-ready and eligible for support. All features available for Anyscale on AKS are available whether you use anyscale cloud setup or anyscale cloud register.

By default, this flow prompts you to install ingress-nginx. You can skip this and configure your own ingress or gateway controller. To fully customize the ingress setup, use anyscale cloud register and configure your Helm chart manually. See Anyscale on Kubernetes.

Contact Anyscale support for configuration and customization help.

Prerequisites​

Before you start, ensure the following:

  • You are an Anyscale organization owner.
  • You have an existing AKS cluster with OIDC issuer and Microsoft Entra Workload ID enabled. The cloud setup command uses these to create a federated identity for the Anyscale operator. To create a cluster, see the Azure workload identity quickstart in the Azure docs or use the Anyscale Terraform module for AKS for production deployments. To enable OIDC and workload identity on an existing cluster, see Enable workload identity on an existing AKS cluster in the Azure docs.
  • You have sufficient privileges in Azure to create resources in the cluster's resource group. The CLI deploys an ARM template that creates a storage account, blob container, user-assigned managed identity, federated identity credential, and role assignment.
  • The Anyscale CLI is installed with Azure support: pip install -U "anyscale[azure]". The [azure] extra is required for this flow and isn't included in the default install.
  • The following CLI tools are installed: az, kubectl, helm.
  • You have run anyscale login and are signed in to Azure with az login. See Authenticate the Anyscale CLI and Sign in with Azure CLI in the Azure docs.

Cluster and cloud details​

Enter the following values. You'll use them when connecting with kubectl and running cloud setup.

Connect to your AKS cluster​

Configure kubectl to use your cluster:

az aks get-credentials --resource-group <resource-group> --name <cluster-name> --overwrite-existing

Verify the connection:

kubectl get nodes

Run cloud setup​

Run the cloud setup command:

anyscale cloud setup \
  --stack k8s \
  --cluster-name <cluster-name> \
  --resource-group <resource-group> \
  --functional-verify

The CLI prompts you for additional values to complete your cloud setup.

  • The --functional-verify runs a post-setup check to confirm the cloud can run a workload.

When prompted for a namespace, press Enter to use the default anyscale-operator or type a different namespace. Anyscale recommends a dedicated namespace per Anyscale cloud. See Anyscale on Kubernetes.

What the CLI does​

The CLI runs the following steps:

  1. Discovery: Fetches your AKS cluster details, including the OIDC issuer URL, and configures your kubeconfig.
  2. ARM deployment: Creates a storage account and blob container with CORS for the Anyscale UI, a user-assigned managed identity for the Anyscale operator, a federated identity credential linking the operator's Kubernetes service account to that identity, and a role assignment so the identity can access the storage account.
  3. Cloud registration: Registers the Anyscale cloud with the control plane using the new storage and identity resources.
  4. Operator install: Generates a Helm values file and installs the Anyscale operator in your chosen namespace. The CLI also prompts you to install ingress-nginx as the ingress controller for dashboards and Anyscale services. You can skip this step and configure your own ingress controller.
  5. Verification: If you used --functional-verify, runs a check to confirm the cloud is usable.
warning

If setup fails, the CLI doesn't automatically remove resources created in earlier steps. For troubleshooting and cleanup steps, contact Anyscale support and provide the CLI output. You need sufficient privileges in the Azure subscription to clean up or retry.

Verify your cloud​

If you didn't use --functional-verify, verify the cloud manually:

anyscale cloud verify --name <cloud-name>

You can also submit a small job to confirm the cloud works:

anyscale job submit --cloud <cloud-name> --working-dir https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip -- python hello_world.py

Next steps​