Skip to main content

Deploy Anyscale on Amazon EKS

View Markdown

Deploy Anyscale on Amazon EKS

Complete the following steps to configure and deploy a new Anyscale cloud on EKS.

1. Install the Anyscale CLI​

pip install -U anyscale
anyscale login # authenticate

2. Authenticate the AWS CLI​

Set up your AWS credentials locally if you haven't done so before. See the AWS configuration guide.

aws configure
note

Before you continue, make sure your AWS credentials have the following permissions:

  • Launch EC2 instances in the AWS region you plan to use.
  • Manage these resources: VPC, subnets, Security Group, IAM, and S3.

See Manage Anyscale clouds on AWS.

3. Use the Anyscale Terraform module to create an EKS cluster​

Anyscale provides a Terraform module to deploy an EKS cluster and supporting AWS resources.

note

To use an existing EKS cluster, follow the existing EKS cluster example or see the Anyscale Operator documentation and the Anyscale Terraform repository.

Enter information about where you want to deploy your EKS cluster:

Clone the Terraform module and navigate to the EKS example:

git clone https://github.com/anyscale/terraform-kubernetes-anyscale-foundation-modules
cd terraform-kubernetes-anyscale-foundation-modules/examples/aws/eks-public/

Run the following command to create and populate a Terraform variable file:

cat <<EOF > terraform.tfvars
eks_cluster_name = "<your_eks_cluster_name>"
aws_region = "<your_aws_region>"
EOF
note

The Terraform example enables GPU node pools (T4) by default. To customize or disable GPU pools, set gpu_instance_types in your terraform.tfvars (for example, use an empty map {} to disable GPU pools).

Run the following commands to apply the Terraform configuration. This may take several minutes.

terraform init
terraform plan
terraform apply
note

Some Terraform errors may be resolved by re-running terraform apply.

Collect and enter the following values from your Terraform output:

4. Install additional EKS components​

In this step, you install the required components for autoscaling, load balancing, and ingress on your EKS cluster. For more information about customizing EKS, see the Anyscale Terraform repository.

Run the following command to connect your terminal to the EKS cluster:

aws eks update-kubeconfig --region <your_aws_region> --name <your_eks_cluster_name>

Install the Kubernetes Autoscaler Helm chart:

helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm upgrade cluster-autoscaler autoscaler/cluster-autoscaler \
  --version 9.46.0 \
  --namespace kube-system \
  --set awsRegion=<your_aws_region> \
  --set 'autoDiscovery.clusterName'=<your_eks_cluster_name> \
  --install

Install the AWS load balancer controller:

helm repo add eks https://aws.github.io/eks-charts
helm upgrade aws-load-balancer-controller eks/aws-load-balancer-controller \
  --version 1.13.2 \
  --namespace kube-system \
  --set clusterName=<your_eks_cluster_name> \
  --install

Install the Nginx ingress controller. The Terraform example repo includes a sample values file sample-values_nginx.yaml. You can use this file or supply your own.

note

The Anyscale operator chart can optionally install the NGINX Ingress Controller as a dependency (ingress-nginx.enabled: true in Helm values). This guide follows the Terraform example and installs NGINX manually so you can use the example's values file.

helm repo add nginx https://kubernetes.github.io/ingress-nginx
helm upgrade ingress-nginx nginx/ingress-nginx \
  --version 4.12.1 \
  --namespace ingress-nginx \
  --values sample-values_nginx.yaml \
  --create-namespace \
  --install

If you intend to use NVIDIA GPUs in your Anyscale workloads, install the NVIDIA device plugin:

helm repo add nvdp https://nvidia.github.io/k8s-device-plugin
helm upgrade nvdp nvdp/nvidia-device-plugin \
  --namespace nvidia-device-plugin \
  --version 0.17.1 \
  --values sample-values_nvdp.yaml \
  --create-namespace \
  --install

5. Register the Anyscale cloud resources​

Run the following command with the values from your Terraform output. Verify all variables are entered correctly.

anyscale cloud register \
  --name <your_cloud_name> \
  --provider aws \
  --region <your_aws_region> \
  --compute-stack k8s \
  --kubernetes-zones us-west-2a,us-west-2b \
  --anyscale-operator-iam-identity arn:aws:iam::<your_aws_account_id>:role/<your_anyscale_operator_iam_role> \
  --s3-bucket-id s3://<your_storage_bucket_name>

Record the cloud resource ID from the output in the following field:

6. Install and deploy the Anyscale operator on your EKS cluster​

In this step, you add the Anyscale operator Helm chart to your EKS cluster, create a values.yaml file that describes your cloud and AWS identity, and install the operator with Helm.

Add the Anyscale operator Helm chart​

Run the following command to add the Anyscale operator Helm chart:

helm repo add anyscale https://anyscale.github.io/helm-charts
helm repo update anyscale

Create a values YAML file​

Create a values.yaml file. The following example uses the values from earlier steps for a minimal AWS configuration.

global:
  cloudDeploymentId: <your_cloud_resource_id>
  cloudProvider: aws
  aws:
    region: <your_aws_region>

workloads:
  serviceAccount:
    name: anyscale-operator

To customize the Helm chart with custom patches or additional pod shapes, see Configure the Helm chart for the Anyscale operator.

Install the Anyscale operator on EKS​

Run the following command to install the Anyscale operator with Helm using your values.yaml file.

helm upgrade anyscale-operator anyscale/anyscale-operator \
  --namespace anyscale-operator \
  -f values.yaml \
  --create-namespace \
  --wait \
  -i

It may take several minutes for your Anyscale cloud to be ready. You can watch the deployment status with the following command:

kubectl get deployments anyscale-operator -n anyscale-operator -w

7. Verify your Anyscale cloud​

Run the following command to verify that your cloud is functional:

anyscale cloud verify --name <your_cloud_name>