Compute configuration options for Kubernetes

View Markdown

Compute configuration options for Kubernetes

This page provides an example of a compute config for Kubernetes, including AKS, EKS, and GKE deployments. Anyscale applies compute configs as a strategic merge patch to the Pod specifications generated by Anyscale before sending them to the Kubernetes API.

For a general overview of compute configs, see Compute configuration on Anyscale.

For configuration settings applicable to all Anyscale clouds, see Advanced settings for compute configs on Anyscale.

Node-specific configurations override any cluster-wide configurations for that node type. For a full reference on configuring these properties, see the official Kubernetes documentation.

note

On Azure, Anyscale clouds use AKS exclusively. All compute config options on this page apply to AKS deployments. For AKS setup, see Anyscale on AKS.

Instance types on Kubernetes

On Kubernetes, instance types are pod shapes that a Kubernetes admin defines in the Helm chart for the Anyscale operator. These aren't cloud provider VM types. They represent the resource requests and limits for pods that Anyscale schedules on your cluster.

Your Kubernetes admin configures accelerator mappings in the Helm chart to determine how GPU types map to instance types. The default mappings vary by cloud provider. See Configure the Helm chart for the Anyscale operator.

tip

Use cluster-level advanced_instance_config for settings that apply to all pods, such as labels, annotations, tolerations, and node selectors. Use worker-group-level advanced_instance_config for settings specific to a worker group. Worker-group settings fully replace the cluster-level settings for that group.

If your workload uses a machine pool on EKS or GKE, Anyscale applies your advanced_instance_config as a strategic merge patch over the machine pool launch template rather than fully replacing it. See Override machine pool defaults from a compute config.

Example Kubernetes compute configuration

The following is an example of compute configuration options for Anyscale on Kubernetes:

Anyscale console

{
  "metadata": {
    // Add a new label.
    "labels": {"new-label": "example-value"},
    // Add a new annotation.
    "annotations": {"new-annotation": "example-value"}
  },
  "spec": {
    // Add a node selector.
    "nodeSelector": {"disktype": "ssd"},
    "tolerations": [{
      "effect": "NoSchedule",
      "key": "dedicated",
      "value": "example-anyscale"
    }],
    "containers": [{
      "name": "ray",
      // Enable privileged container in order to use Ray image_uri runtime env to run nested worker container via podman.
      "securityContext": {
        "privileged": true
      },
      // Add a PersistentVolumeClaim to the Ray container.
      "volumeMounts": [{
        "name": "pvc-volume",
        "mountPath": "/mnt/pvc-data"
      }]
    },{
      // Add a sidecar for exporting logs/metrics.
      "name": "monitoring-sidecar",
      "image": "timberio/vector:latest",
      "ports": [{
        "containerPort": 9000
      }],
      "volumeMounts": [{
        "name": "vector-volume",
        "mountPath": "/mnt/vector-data"
      }]
    }],
    "volumes": [{
      "name": "pvc-volume",
      "persistentVolumeClaim": {
        "claimName": "my-pvc"
      }
    },{
      "name": "vector-volume",
      "emptyDir": {}
    }]
  }
}

Anyscale CLI

compute_config:
  cloud: <your-cloud-name>
  head_node:
    instance_type: 4CPU-16GB
  worker_nodes:
    - instance_type: 4CPU-16GB
      min_nodes: 1
      max_nodes: 10
  advanced_instance_config:
    metadata:
      # Add a new label.
      labels:
        new-label: example-value
      # Add a new annotation.
      annotations:
        new-annotation: example-value
    spec:
      # Add a node selector.
      nodeSelector:
        disktype: ssd
      # Add tolerations.
      tolerations:
        - effect: NoSchedule
          key: dedicated
          value: example-anyscale
      containers:
        - name: ray
          # Enable privileged container in order to use Ray image_uri runtime env to run nested worker container via podman.
          securityContext:
            privileged: true
          # Add a PersistentVolumeClaim to the Ray container.
          volumeMounts:
            - name: pvc-volume
              mountPath: /mnt/pvc-data
        # Add a sidecar for exporting logs/metrics.
        - name: monitoring-sidecar
          image: timberio/vector:latest
          ports:
            - containerPort: 9000
          volumeMounts:
            - name: vector-volume
              mountPath: /mnt/vector-data
      volumes:
        - name: pvc-volume
          persistentVolumeClaim:
            claimName: my-pvc
        - name: vector-volume
          emptyDir: {}