Get started with Anyscale Services V1

caution

This page is for the deprecated Services V1. Please use Services V2 instead.

note

Use of Anyscale Services requires Ray 1.12+.

Anyscale Services are the recommended way to move your Ray Serve Applications to production.

Key features of Anyscale Services:

1. Automatic failure recovery
2. Automated email alerting
3. Persistent host name
4. [coming soon] High Availability and 0 downtime upgrade

This walk-through will help you run your Ray Serve Application as a production service on Anyscale.

Setup your environment

Install the latest ray and Anyscale

pip install ray
pip install -U anyscale

Clone the Anyscale Examples Repo.

git clone https://github.com/anyscale/docs_examples.git anyscale_doc_examples
cd anyscale_doc_examples

Setup an bucket in your cloud provider account. Your laptop needs to write to the bucket, and Anyscale needs to read files in the bucket. Configure permissions according to instructions for AWS or GCP.

info

We need a bucket because we will be packaging a local directory as a zip containing files for our Service, and hosting that zip in the specified bucket. If you haven't configured a bucket, you can follow along with the first part of this tutorial using example zip file stored on GitHub.

If you already have a hosted zip file with your code, that will work too. Any zip file that meets the format is accepted. Importantly, the zip file must contain only a single top-level directory. See Remote URIs for more details about Ray Runtime Environments, and see here for more detail about Anyscale-specific runtime_env configuration.

Deploy your first Anyscale Service

When you submit a Service YAML, Anyscale runs your Service on a managed cluster and monitors it. In the case of failure, Anyscale will alert you and restart your Service.

note

Unlike Jobs, Service names must be unique within a project. This enables referencing services by name. If you call deploy on a YAML with a service name that exists, that service will be updated.

Let's run the example.

CLI

anyscale service deploy service.yaml

(anyscale +3.3s) Query the status of the service with `anyscale service list --service-id service_sc2JFXVupTCLy6CKqcYq5rmN`.
(anyscale +3.3s) View the service in the UI at https://console.anyscale-staging.com/services/service_sc2JFXVupTCLy6CKqcYq5rmN.

service.yaml

name: service-demo
entrypoint: python serve_hello.py
runtime_env:
  working_dir: https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip
healthcheck_url: "/healthcheck"

hello_world.py

from fastapi import FastAPI
from ray import serve
import os

# Use this var to test service inplace update. When the env var is updated, users see new return value.
msg = os.getenv("SERVE_RESPONSE_MESSAGE", "Hello world!")

serve.start(detached=True)

app = FastAPI()

@serve.deployment(route_prefix="/")
@serve.ingress(app)
class HelloWorld:
    @app.get("/")
    def hello(self):
        return msg
    
    @app.get("/healthcheck")
    def healthcheck(self):
        return

HelloWorld.deploy()

Query the Service

You can find the address of the Service in the Anyscale UI by clicking the "Query" button in the top right hand corner.

You should find a command that looks something like the below

Query With Curl

curl -H "Authorization: Bearer SECRET_TOKEN" https://serve-session-3aruf6xjitbft3k8tnnzmdfm.i.anyscaleuserdata-staging.com

"Hello world!"

Upgrade the Service

Let's upgrade the service to show a different message when we query it.

In this case, we just need to edit the entrypoint in our YAML to set the SERVE_RESPONSE_MESSAGE environment variable.

To see the change, redeploy the service.

CLI

anyscale service deploy service.yaml


(anyscale +3.3s) Query the status of the service with `anyscale service list --service-id service_sc2JFXVupTCLy6CKqcYq5rmN`.
(anyscale +3.3s) View the service in the UI at https://console.anyscale-staging.com/services/service_sc2JFXVupTCLy6CKqcYq5rmN.

service.yaml

name: service-demo
- entrypoint: python serve_hello.py
+ entrypoint: SERVE_RESPONSE_MESSAGE="hello from new service" python serve_hello.py
runtime_env:
  working_dir: https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip
healthcheck_url: "/healthcheck"

hello_world.py

from fastapi import FastAPI
from ray import serve
import os

# Use this var to test service inplace update. When the env var is updated, users see new return value.
msg = os.getenv("SERVE_RESPONSE_MESSAGE", "Hello world!")

serve.start(detached=True)

app = FastAPI()

@serve.deployment(route_prefix="/")
@serve.ingress(app)
class HelloWorld:
    @app.get("/")
    def hello(self):
        return msg
    
    @app.get("/healthcheck")
    def healthcheck(self):
        return

HelloWorld.deploy()

Now when you query the service, you should see the new message.

Query With Curl

curl -H "Authorization: Bearer SECRET_TOKEN" https://serve-session-3aruf6xjitbft3k8tnnzmdfm.i.anyscaleuserdata-staging.com

"hello from new service"

Change to use you own bucket

You can upload a local directory with the runtime_nev.working_dir field in the Service YAML. To do so, you must also specify a remote bucket and ensure that both your laptop and cluster have network and IAM permissions to access it. For more information, view docs for accessing an Amazon S3 or Google Cloud Storage bucket.

CLI

anyscale service deploy service.yaml


(anyscale +3.3s) Query the status of the service with `anyscale service list --service-id service_sc2JFXVupTCLy6CKqcYq5rmN`.
(anyscale +3.3s) View the service in the UI at https://console.anyscale-staging.com/services/service_sc2JFXVupTCLy6CKqcYq5rmN.

job.yaml

name: service-demo
entrypoint: SERVE_RESPONSE_MESSAGE="hello from new service" python serve_hello.py
+ compute_config: config_with_bucket_permission
runtime_env:
-  working_dir: https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip
+  working_dir: .
+  upload_path: "s3://your-bucket/path"
healthcheck_url: "/healthcheck"

hello_world.py

from fastapi import FastAPI
from ray import serve
import os

# Use this var to test service inplace update. When the env var is updated, users see new return value.
msg = os.getenv("SERVE_RESPONSE_MESSAGE", "Hello world!")

serve.start(detached=True)

app = FastAPI()

@serve.deployment(route_prefix="/")
@serve.ingress(app)
class HelloWorld:
    @app.get("/")
    def hello(self):
-        return msg
+        return msg + ", and a code change"
    
    @app.get("/healthcheck")
    def healthcheck(self):
        return

HelloWorld.deploy()

Now when you query the service, you should see the new message.

Query With Curl

curl -H "Authorization: Bearer SECRET_TOKEN" https://serve-session-3aruf6xjitbft3k8tnnzmdfm.i.anyscaleuserdata-staging.com

"hello from new service, and a code change"

Terminate

When we are done, let's go ahead and terminate the service using the CLI.

anyscale service terminate -n service-demo

Conclusion

In this tutorial, we have created, updated, and queried an Anyscale Service. To see more information about Services, see deploy, query and manage Anyscale services.