--- title: "Set up Anyscale on AKS" description: "Set up an Anyscale cloud on an existing AKS cluster using the anyscale cloud setup command." --- # Set up Anyscale on AKS This page describes how to set up an Anyscale cloud on Azure Kubernetes Service (AKS) using the `anyscale cloud setup` command. This flow creates a working baseline so you can run jobs and services. You can customize networking, node pools, and security after setup. :::important The `anyscale cloud setup` CLI flow for Kubernetes is in beta. Anyscale recommends this flow for self-service onboarding. Clouds deployed with this flow are production-ready and eligible for support. All features available for Anyscale on AKS are available whether you use `anyscale cloud setup` or `anyscale cloud register`. By default, this flow prompts you to install `ingress-nginx`. You can skip this and configure your own ingress or gateway controller. To fully customize the ingress setup, use [`anyscale cloud register`](/admin/azure/create-aks-cloud.md) and configure your Helm chart manually. See [Anyscale on Kubernetes](/k8s.md). Contact [Anyscale support](mailto:support@anyscale.com) for configuration and customization help. ::: ## Prerequisites Before you start, ensure the following: - You are an Anyscale organization owner. - You have an existing AKS cluster with OIDC issuer and Microsoft Entra Workload ID enabled. The cloud setup command uses these to create a federated identity for the Anyscale operator. To create a cluster, see the [Azure workload identity quickstart](https://learn.microsoft.com/en-us/azure/aks/workload-identity-deploy-cluster?tabs=new-cluster) in the Azure docs or use the [Anyscale Terraform module for AKS](https://github.com/anyscale/terraform-kubernetes-anyscale-foundation-modules/tree/main/examples/azure/aks-new_cluster) for production deployments. To enable OIDC and workload identity on an existing cluster, see [Enable workload identity on an existing AKS cluster](https://learn.microsoft.com/en-us/azure/aks/workload-identity-deploy-cluster?tabs=existing-cluster) in the Azure docs. - You have sufficient privileges in Azure to create resources in the cluster's resource group. The CLI deploys an ARM template that creates a storage account, blob container, user-assigned managed identity, federated identity credential, and role assignment. - The Anyscale CLI is installed with Azure support: `pip install -U "anyscale[azure]"`. The `[azure]` extra is required for this flow and isn't included in the default install. - The following CLI tools are installed: `az`, `kubectl`, `helm`. - You have run `anyscale login` and are signed in to Azure with `az login`. See [Authenticate the Anyscale CLI](/auth.md#cli-auth) and [Sign in with Azure CLI](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli) in the Azure docs. ## Cluster and cloud details Enter the following values. You'll use them when connecting with `kubectl` and running cloud setup. Azure resource group name `my-aks-rg` AKS cluster name `my-aks-cluster` Anyscale cloud name `my-aks-cloud` ## Connect to your AKS cluster Configure `kubectl` to use your cluster: ```bash az aks get-credentials --resource-group --name --overwrite-existing ``` Verify the connection: ```bash kubectl get nodes ``` ## Run cloud setup Run the cloud setup command: ```bash anyscale cloud setup \ --stack k8s \ --cluster-name \ --resource-group \ --functional-verify ``` The CLI prompts you for additional values to complete your cloud setup. - The `--functional-verify` runs a post-setup check to confirm the cloud can run a workload. When prompted for a namespace, press Enter to use the default `anyscale-operator` or type a different namespace. Anyscale recommends a dedicated namespace per Anyscale cloud. See [Anyscale on Kubernetes](/k8s.md). ## What the CLI does The CLI runs the following steps: 1. **Discovery**: Fetches your AKS cluster details, including the OIDC issuer URL, and configures your kubeconfig. 2. **ARM deployment**: Creates a storage account and blob container with CORS for the Anyscale UI, a user-assigned managed identity for the Anyscale operator, a federated identity credential linking the operator's Kubernetes service account to that identity, and a role assignment so the identity can access the storage account. 3. **Cloud registration**: Registers the Anyscale cloud with the control plane using the new storage and identity resources. 4. **Operator install**: Generates a Helm values file and installs the Anyscale operator in your chosen namespace. The CLI also prompts you to install `ingress-nginx` as the ingress controller for dashboards and Anyscale services. You can skip this step and configure your own ingress controller. 5. **Verification**: If you used `--functional-verify`, runs a check to confirm the cloud is usable. :::warning If setup fails, the CLI doesn't automatically remove resources created in earlier steps. For troubleshooting and cleanup steps, contact [Anyscale support](mailto:support@anyscale.com) and provide the CLI output. You need sufficient privileges in the Azure subscription to clean up or retry. ::: ## Verify your cloud If you didn't use `--functional-verify`, verify the cloud manually: ```bash anyscale cloud verify --name ``` You can also submit a small job to confirm the cloud works: ```bash anyscale job submit --cloud --working-dir https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip -- python hello_world.py ``` ## Next steps - **Customize the operator**: See [Configure the Helm chart for the Anyscale operator](/k8s/configure-helm.md). - **Manage permissions**: See [Configure managed identities for clusters on Anyscale on AKS](/admin/azure/aks-iam.md). - **Set up storage integrations**: See [Access blob storage and ADLS](/admin/azure/storage.md) and [Configure shared storage with Azure blob PVC for AKS](/admin/azure/pvc.md). - **Access private container registries**: See [Configure Azure Container Registry](/admin/azure/container-registry.md). --- Previous: [Overview](/admin/azure/configure-aks.md) | Next: [Deploy Anyscale on AKS (cloud register)](/admin/azure/create-aks-cloud.md)