Compute configuration options for Kubernetes

View Markdown

Compute configuration options for Kubernetes

This page provides an example of a compute config for Kubernetes, including AKS, EKS, and GKE deployments. Anyscale applies compute configs as a strategic merge patch to the Pod specifications generated by Anyscale before sending them to the Kubernetes API.

For a general overview of compute configs, see Compute configuration on Anyscale.

For configuration settings applicable to all Anyscale clouds, see Advanced settings for compute configs on Anyscale.

Node-specific configurations override any cluster-wide configurations for that node type. For a full reference on configuring these properties, see the official Kubernetes documentation.

note

Anyscale on Azure runs exclusively on AKS. All compute config options on this page apply to Azure deployments. For Azure cloud setup, see Deploy Anyscale on Azure Kubernetes Service (AKS).

Instance types on Kubernetes

On Kubernetes, instance types are pod shapes that a Kubernetes admin defines in the Helm chart for the Anyscale operator. These aren't cloud provider VM types. They represent the resource requests and limits for pods that Anyscale schedules on your cluster.

Your Kubernetes admin configures accelerator mappings in the Helm chart to determine how GPU types map to instance types. The default mappings vary by cloud provider. See Configure the Helm chart for the Anyscale operator.

Example Kubernetes compute configuration

The following is an example of compute configuration options for Anyscale on Kubernetes:

Anyscale console

{
  "metadata": {
    // Add a new label.
    "labels": {"new-label": "example-value"},
    // Add a new annotation.
    "annotations": {"new-annotation": "example-value"}
  },
  "spec": {
    // Add a node selector.
    "nodeSelector": {"disktype": "ssd"},
    "tolerations": [{
      "effect": "NoSchedule",
      "key": "dedicated",
      "value": "example-anyscale"
    }],
    "containers": [{
      "name": "ray",
      // Enable privileged container in order to use Ray image_uri runtime env to run nested worker container via podman.
      "securityContext": {
        "privileged": true
      },
      // Add a PersistentVolumeClaim to the Ray container.
      "volumeMounts": [{
        "name": "pvc-volume",
        "mountPath": "/mnt/pvc-data"
      }]
    },{
      // Add a sidecar for exporting logs/metrics.
      "name": "monitoring-sidecar",
      "image": "timberio/vector:latest",
      "ports": [{
        "containerPort": 9000
      }],
      "volumeMounts": [{
        "name": "vector-volume",
        "mountPath": "/mnt/vector-data"
      }]
    }],
    "volumes": [{
      "name": "pvc-volume",
      "persistentVolumeClaim": {
        "claimName": "my-pvc"
      }
    },{
      "name": "vector-volume",
      "emptyDir": {}
    }]
  }
}

Anyscale CLI

compute_config:
  cloud: <your-cloud-name>
  head_node:
    instance_type: 4CPU-16GB
  worker_nodes:
    - instance_type: 4CPU-16GB
      min_nodes: 1
      max_nodes: 10
  advanced_instance_config:
    metadata:
      # Add a new label.
      labels:
        new-label: example-value
      # Add a new annotation.
      annotations:
        new-annotation: example-value
    spec:
      # Add a node selector.
      nodeSelector:
        disktype: ssd
      # Add tolerations.
      tolerations:
        - effect: NoSchedule
          key: dedicated
          value: example-anyscale
      containers:
        - name: ray
          # Enable privileged container in order to use Ray image_uri runtime env to run nested worker container via podman.
          securityContext:
            privileged: true
          # Add a PersistentVolumeClaim to the Ray container.
          volumeMounts:
            - name: pvc-volume
              mountPath: /mnt/pvc-data
        # Add a sidecar for exporting logs/metrics.
        - name: monitoring-sidecar
          image: timberio/vector:latest
          ports:
            - containerPort: 9000
          volumeMounts:
            - name: vector-volume
              mountPath: /mnt/vector-data
      volumes:
        - name: pvc-volume
          persistentVolumeClaim:
            claimName: my-pvc
        - name: vector-volume
          emptyDir: {}