Skip to main content

Developing Ray Serve Applications

You can deploy your Ray Serve application to Anyscale for testing and development using Ray Client.

First, install the necessary dependencies for Ray Serve locally:

pip install "ray[serve]"

Then, create an Anyscale project for the application and set the ANYSCALE_PROJECT_NAME environment variable.

$ mkdir hello_world
$ cd hello_world
$ anyscale project create
Name [hello_world]: hello_world
Created project with name hello_world at https://console.anyscale.com/projects/prj_6SCoYQrJU4BYzpcpj0mKxk.
Please specify the project id as prj_6SCoYQrJU4BYzpcpj0mKxk when calling Anyscale CLI or SDK commands or Ray Client.

$ export ANYSCALE_PROJECT_NAME="hello_world"

Write and run a simple "hello world" Ray Serve application.

deploy.py
import ray
from ray import serve

ray.init("anyscale://my_cluster", autosuspend=-1)

serve.start(detached=True)

@serve.deployment
def hello(request):
name = request.query_params["name"]
return f"Hello {name}!"

hello.deploy()

After the script runs, you should see a link to the cluster like this:

> python deploy.py
...
Url to head node of cluster is: https://session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com

Your function is now deployed at https://serve-session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com. Use the code snippet below to test your Serve deployment, replacing the URL with your own URL printed from running deploy.py. Get your user_service_token from the SDK read model for your cluster.

import requests
from anyscale import AnyscaleSDK

sdk = AnyscaleSDK()
cluster = sdk.search_clusters(clusters_query={"name": {"equals": "my_cluster"}}).results[0]

resp = requests.get(
"https://serve-session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com"
"/hello?name=Anyscale", headers={"Authorization" : f"Bearer {cluster.user_service_token}"})
print(resp.text)
# output: Hello Anyscale!

To update your application, simply re-run the deployment script on the cluster again. Your local directory will automatically be synced to the cluster each time you run the script.