Developing Ray Serve Applications
You can deploy your Ray Serve application to Anyscale for testing and development using Ray Client.
First, install the necessary dependencies for Ray Serve locally:
pip install "ray[serve]"
Then, create an Anyscale project for the application and set the ANYSCALE_PROJECT_NAME
environment variable.
$ mkdir hello_world
$ cd hello_world
$ anyscale project create
Name [hello_world]: hello_world
Created project with name hello_world at https://console.anyscale.com/projects/prj_6SCoYQrJU4BYzpcpj0mKxk.
Please specify the project id as prj_6SCoYQrJU4BYzpcpj0mKxk when calling Anyscale CLI or SDK commands or Ray Client.
$ export ANYSCALE_PROJECT_NAME="hello_world"
Write and run a simple "hello world" Ray Serve application.
import ray
from ray import serve
ray.init("anyscale://my_cluster", autosuspend=-1)
serve.start(detached=True)
@serve.deployment
def hello(request):
name = request.query_params["name"]
return f"Hello {name}!"
hello.deploy()
After the script runs, you should see a link to the cluster like this:
> python deploy.py
...
Url to head node of cluster is: https://session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com
Your function is now deployed at https://serve-session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com
. Use the code snippet below to test your Serve deployment, replacing the URL with your own URL printed from running deploy.py
. Get your user_service_token
from the SDK read model for your cluster.
import requests
from anyscale import AnyscaleSDK
sdk = AnyscaleSDK()
cluster = sdk.search_clusters(clusters_query={"name": {"equals": "my_cluster"}}).results[0]
resp = requests.get(
"https://serve-session-9mhabpzb13x3up3qad4co.i.anyscaleuserdata.com"
"/hello?name=Anyscale", headers={"Authorization" : f"Bearer {cluster.user_service_token}"})
print(resp.text)
# output: Hello Anyscale!
To update your application, simply re-run the deployment script on the cluster again. Your local directory will automatically be synced to the cluster each time you run the script.