--- title: "Set up Envoy Gateway" description: "Set up Envoy Gateway for your Anyscale Kubernetes deployment." --- # Set up Envoy Gateway This page describes how to set up Envoy Gateway for an Anyscale Kubernetes deployment. If you have an existing deployment using `ingress-nginx`, this guide also covers migrating traffic to Envoy Gateway. :::note `ingress-nginx` is deprecated. Anyscale recommends migrating to Gateway API for all Kubernetes deployments. ::: ## Prerequisites Before starting, confirm you have the following: - Kubernetes 1.24 or later. - Your Anyscale cloud ID and cloud resource ID. Find both in the Anyscale console under **Cloud details**. - The latest version of the Anyscale CLI. Run `pip install --upgrade anyscale` to update. Enter your IDs below to auto-populate the commands and configuration on this page. Cloud Resource ID `cldrsrc_abc123xyz` Cloud ID `cld_abc123xyz` :::note If you don't have active Anyscale services running, you don't need to run the migration commands in [Step 6](#migrate-traffic). Complete Steps 1 through 5 to set up Gateway infrastructure, then upgrade the operator. Gateway takes over automatically. ::: ## Step 1: Install Envoy Gateway Run the following command to install Envoy Gateway v1.7.0: ```bash helm install eg oci://docker.io/envoyproxy/gateway-helm \ --version v1.7.0 \ --namespace envoy-gateway-system \ --create-namespace ``` Run the following command to verify Envoy Gateway is running before proceeding: ```bash kubectl wait --for=condition=available deployment/envoy-gateway \ -n envoy-gateway-system --timeout=120s ``` ## Step 2: Create an EnvoyProxy resource Create an `EnvoyProxy` resource to configure the Envoy Gateway load balancer. Anyscale recommends a dedicated `EnvoyProxy` per Anyscale cloud resource for isolation. Select your cloud provider and save the following as `envoyproxy.yaml`: ::::tabs :::tab[AWS] ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: envoy-proxy namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: LoadBalancer annotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing ``` ::: :::tab[Google Cloud] ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: envoy-proxy namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: LoadBalancer annotations: cloud.google.com/load-balancer-type: "External" ``` ::: :::tab[Azure] ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: envoy-proxy namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: LoadBalancer annotations: service.beta.kubernetes.io/azure-load-balancer-internal: "false" ``` ::: :::tab[CoreWeave] ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: envoy-proxy namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: LoadBalancer annotations: service.beta.kubernetes.io/coreweave-load-balancer-type: "public" ``` ::: :::tab[Nebius] ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: envoy-proxy namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: LoadBalancer ``` ::: :::: Run the following command to apply the EnvoyProxy resource: ```bash kubectl apply -f envoyproxy.yaml ``` ## Step 3: Create a GatewayClass Create a `GatewayClass` that references the `EnvoyProxy` from Step 2. Anyscale recommends a dedicated `GatewayClass` per Anyscale cloud resource for isolation. Save the following as `gatewayclass.yaml`: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: eg spec: controllerName: gateway.envoyproxy.io/gatewayclass-controller parametersRef: group: gateway.envoyproxy.io kind: EnvoyProxy name: envoy-proxy namespace: envoy-gateway-system ``` Run the following command to apply the GatewayClass: ```bash kubectl apply -f gatewayclass.yaml ``` ## Step 4: Create the Gateway The Gateway resource requires TLS certificate secrets that Anyscale provisions automatically. Secret names include the Anyscale cloud resource ID. The following table describes when Anyscale provisions these certificates: | Secret name | Covers | Provisioned | | --- | --- | --- | | `anyscale--certificate` | Head node access | After operator installation | | `anyscale-svc--certificate` | Service access | After your first Anyscale service runs | :::note Kubernetes secret names require lowercase characters and hyphens. If you entered your cloud resource ID in the field above, the YAML on this page converts the format automatically. Otherwise, convert underscores to hyphens and lowercase the result manually. For example, `cldrsrc_abc123` becomes `cldrsrc-abc123`. Then the secrets would be named `anyscale-cldrsrc-abc123-certificate` and `anyscale-svc-cldrsrc-abc123-certificate`. ::: Create the Gateway in the Anyscale operator namespace with three listeners. Save the following as `gateway.yaml`: ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: gateway namespace: anyscale-operator spec: gatewayClassName: eg listeners: - name: http port: 80 protocol: HTTP allowedRoutes: namespaces: from: Same - name: https port: 443 protocol: HTTPS hostname: '*.i.anyscaleuserdata.com' tls: mode: Terminate certificateRefs: - kind: Secret name: anyscale--certificate allowedRoutes: namespaces: from: Same - name: https-session port: 443 protocol: HTTPS hostname: '*.s.anyscaleuserdata.com' tls: mode: Terminate certificateRefs: - kind: Secret name: anyscale-svc--certificate allowedRoutes: namespaces: from: Same ``` :::note The `anyscale-svc--certificate` secret isn't provisioned until your first Anyscale service runs. The Gateway applies successfully regardless. Kubernetes processes the listener configuration and picks up the secret automatically once it's provisioned. ::: :::note The `http` listener on port 80 is required for the canary migration process in Step 6. During migration, the Ingress controller makes an internal HTTP call to the Envoy service. If you're doing a fresh installation with no active services, it has no effect on external traffic. All client-facing traffic uses the HTTPS listeners. ::: Run the following command to create the Gateway: ```bash kubectl apply -f gateway.yaml ``` Run the following command to get the Gateway's load balancer address. Use this value in the next step. ```bash kubectl get gateway gateway -n anyscale-operator \ -o jsonpath='{.status.addresses[0].value}' ``` ## Step 5: Configure the Anyscale operator Update your `values.yaml` to enable Gateway mode. This configuration is additive. It doesn't replace your existing ingress settings, and both can coexist during migration. Run the following command to get your current Anyscale operator values: ```bash helm get values anyscale-operator -n anyscale-operator ``` Add the following to your `values.yaml`: ```yaml networking: gateway: enabled: true name: "gateway" namespace: "anyscale-operator" apiVersion: "gateway.networking.k8s.io/v1" hostname: "" # From Step 4 ip: "" ``` Run the following command to upgrade the Anyscale operator: ```bash helm upgrade anyscale-operator anyscale/anyscale-operator \ -n anyscale-operator \ -f values.yaml ``` Run the following command to verify you've successfully created `HTTPRoute` resources: ```bash kubectl get httproutes -n anyscale-operator ``` :::note To revert Gateway mode at any point after this step, see [Roll back the migration](#rollback). ::: ## Step 6: Migrate traffic :::important You must complete this step if you have active Anyscale services. If you don't have any Anyscale services actively deployed, you can skip this step. ::: For deployments with active services, use the following migration commands to gradually shift traffic from Ingress to Gateway. The following instructions use Ingress and HTTPRoute to serve traffic in parallel during the transition, ensuring traffic continues to flow without interruption. :::note During migration, traffic shifting applies only to Anyscale services. Jobs and workspaces continue running on Nginx until you set the weight to 100, at which point all workloads switch to Gateway. ::: ### Step 6a: Start the migration Run the following command to start the migration. During this initial phase, both Ingress and `HTTPRoute` serve traffic, but all traffic flows through the Ingress load balancer. ```bash anyscale cloud start-gateway-migration --cloud-id ``` ### Step 6b: Verify dual-stack operation Run the following command to confirm that both Ingress and `HTTPRoute` specs are present in your namespace: ```bash anyscale cloud gateway-migration-status --cloud-id ``` ### Step 6c: Shift traffic to Gateway Gradually shift traffic from Ingress to Gateway by increasing the canary weight. Monitor your services between each increment before continuing. Run the following command to shift 10% of traffic to Gateway: ```bash anyscale cloud set-gateway-canary-weight --cloud-id --weight 10 ``` After monitoring your services, increase the weight: ```bash anyscale cloud set-gateway-canary-weight --cloud-id --weight 50 ``` After verifying stability, complete the migration: ```bash anyscale cloud set-gateway-canary-weight --cloud-id --weight 100 ``` :::note Wait at least 60 seconds between weight increments before increasing further. To roll back at any point, set the weight back to 0. Traffic returns to Ingress immediately. Contact Anyscale support if requests aren't reaching the Envoy controller or if setting the weight to 0 doesn't restore traffic through Ingress. Check the Envoy controller logs to diagnose routing issues. ::: At `weight=100`, migration is complete and Anyscale no longer generates Ingress resources. ### Step 6d: Verify the migration Run the following command to confirm that all traffic flows through Gateway: ```bash anyscale cloud gateway-migration-status --cloud-id ``` ### Roll back the migration If you encounter issues during the migration, you have the following options depending on how far you've progressed: - **Pause and revert to Ingress**: Set the canary weight to 0. All traffic returns to flowing through the Ingress load balancer. ```bash anyscale cloud set-gateway-canary-weight --cloud-id --weight 0 ``` - **Revert Gateway mode entirely**: Set the canary weight to 0 first, then set `networking.gateway.enabled` to `false` in `values.yaml` and upgrade the operator. The operator returns to generating Ingress resources. ```bash helm upgrade anyscale-operator anyscale/anyscale-operator \ -n anyscale-operator \ -f values.yaml ``` ## Troubleshoot The following table lists common troubleshooting commands:
DescriptionCommand
Check Gateway status
kubectl get gateway gateway -n anyscale-operator
kubectl describe gateway gateway -n anyscale-operator
Check Envoy Gateway controller logs
kubectl logs -n envoy-gateway-system \
  -l control-plane=envoy-gateway --tail=50
Check operator logs
kubectl logs -n anyscale-operator \
  -l app.kubernetes.io/name=anyscale-operator --tail=50
Verify HTTPRoutes
kubectl get httproutes -n anyscale-operator
Verify TLS certificates
kubectl get secrets -n anyscale-operator | grep certificate
--- Previous: [Configure Helm chart](/k8s/configure-helm.md) | Next: [Helm configuration reference](/k8s/helm-ref.md)