Developing Ray Serve Applications

You can deploy your Ray Serve application to Anyscale for testing and development using Ray Client.

First, install the necessary dependencies for Ray Serve locally:

pip install "ray[serve]"

Then, create an Anyscale project for the application and set the ANYSCALE_PROJECT_NAME environment variable.

$ mkdir hello_world
$ cd hello_world
$ anyscale project create
Name [hello_world]: hello_world
Created project with name hello_world at https://console.anyscale.com/projects/prj_6SCoYQrJU4BYzpcpj0mKxk.
Please specify the project id as prj_6SCoYQrJU4BYzpcpj0mKxk when calling Anyscale CLI or SDK commands or Ray Client.

$ export ANYSCALE_PROJECT_NAME="hello_world"

Write and run a simple "hello world" Ray Serve application.

deploy.py

import ray
from ray import serve

ray.init("anyscale://my_cluster", autosuspend=-1)

serve.start(detached=True)

@serve.deployment
def hello(request):
   name = request.query_params["name"]
   return f"Hello {name}!"

hello.deploy()

After the script runs, you should see a link to the cluster like this:

> python deploy.py
...
Url to head node of cluster is: https://session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com

Your function is now deployed at https://serve-session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com. Use the code snippet below to test your Serve deployment, replacing the URL with your own URL printed from running deploy.py. Get your user_service_token from the SDK read model for your cluster.

import requests
from anyscale import AnyscaleSDK

sdk = AnyscaleSDK()
cluster = sdk.search_clusters(clusters_query={"name": {"equals": "my_cluster"}}).results[0]

resp = requests.get(
   "https://serve-session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com"
   "/hello?name=Anyscale", headers={"Authorization" : f"Bearer {cluster.user_service_token}"})
print(resp.text)
# output: Hello Anyscale!

To update your application, simply re-run the deployment script on the cluster again. Your local directory will automatically be synced to the cluster each time you run the script.