Get started with Anyscale Services V1
This page is for the deprecated Services V1. Please use Services V2 instead.
Use of Anyscale Services requires Ray 1.12+.
Anyscale Services are the recommended way to move your Ray Serve Applications to production.
Key features of Anyscale Services:
- Automatic failure recovery
- Automated email alerting
- Persistent host name
- [coming soon] High Availability and 0 downtime upgrade
This walk-through will help you run your Ray Serve Application as a production service on Anyscale.
Setup your environment
Install the latest ray and Anyscale
pip install ray
pip install -U anyscale
Clone the Anyscale Examples Repo.
git clone https://github.com/anyscale/docs_examples.git anyscale_doc_examples
cd anyscale_doc_examples
Setup an bucket in your cloud provider account. Your laptop needs to write to the bucket, and Anyscale needs to read files in the bucket. Configure permissions according to instructions for AWS or GCP.
We need a bucket because we will be packaging a local directory as a zip containing files for our Service, and hosting that zip in the specified bucket. If you haven't configured a bucket, you can follow along with the first part of this tutorial using example zip file stored on GitHub.
If you already have a hosted zip file with your code, that will work too. Any zip file that meets the format is accepted. Importantly, the zip file must contain only a single top-level directory. See Remote URIs for more details about Ray Runtime Environments, and see
here for more detail about Anyscale-specific runtime_env
configuration.
Deploy your first Anyscale Service
When you submit a Service YAML, Anyscale runs your Service on a managed cluster and monitors it. In the case of failure, Anyscale will alert you and restart your Service.
Unlike Jobs, Service names must be unique within a project. This enables referencing services by name. If you call deploy on a YAML with a service name that exists, that service will be updated.
Let's run the example.
- CLI
- service.yaml
- hello_world.py
anyscale service deploy service.yaml
(anyscale +3.3s) Query the status of the service with `anyscale service list --service-id service_sc2JFXVupTCLy6CKqcYq5rmN`.
(anyscale +3.3s) View the service in the UI at https://console.anyscale-staging.com/services/service_sc2JFXVupTCLy6CKqcYq5rmN.
name: service-demo
entrypoint: python serve_hello.py
runtime_env:
working_dir: https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip
healthcheck_url: "/healthcheck"
from fastapi import FastAPI
from ray import serve
import os
# Use this var to test service inplace update. When the env var is updated, users see new return value.
msg = os.getenv("SERVE_RESPONSE_MESSAGE", "Hello world!")
serve.start(detached=True)
app = FastAPI()
@serve.deployment(route_prefix="/")
@serve.ingress(app)
class HelloWorld:
@app.get("/")
def hello(self):
return msg
@app.get("/healthcheck")
def healthcheck(self):
return
HelloWorld.deploy()
Query the Service
You can find the address of the Service in the Anyscale UI by clicking the "Query" button in the top right hand corner.
You should find a command that looks something like the below
curl -H "Authorization: Bearer SECRET_TOKEN" https://serve-session-3aruf6xjitbft3k8tnnzmdfm.i.anyscaleuserdata-staging.com
"Hello world!"
Upgrade the Service
Let's upgrade the service to show a different message when we query it.
In this case, we just need to edit the entrypoint
in our YAML to set the SERVE_RESPONSE_MESSAGE
environment variable.
To see the change, redeploy the service.
- CLI
- service.yaml
- hello_world.py
anyscale service deploy service.yaml
(anyscale +3.3s) Query the status of the service with `anyscale service list --service-id service_sc2JFXVupTCLy6CKqcYq5rmN`.
(anyscale +3.3s) View the service in the UI at https://console.anyscale-staging.com/services/service_sc2JFXVupTCLy6CKqcYq5rmN.
name: service-demo
- entrypoint: python serve_hello.py
+ entrypoint: SERVE_RESPONSE_MESSAGE="hello from new service" python serve_hello.py
runtime_env:
working_dir: https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip
healthcheck_url: "/healthcheck"
from fastapi import FastAPI
from ray import serve
import os
# Use this var to test service inplace update. When the env var is updated, users see new return value.
msg = os.getenv("SERVE_RESPONSE_MESSAGE", "Hello world!")
serve.start(detached=True)
app = FastAPI()
@serve.deployment(route_prefix="/")
@serve.ingress(app)
class HelloWorld:
@app.get("/")
def hello(self):
return msg
@app.get("/healthcheck")
def healthcheck(self):
return
HelloWorld.deploy()
Now when you query the service, you should see the new message.
curl -H "Authorization: Bearer SECRET_TOKEN" https://serve-session-3aruf6xjitbft3k8tnnzmdfm.i.anyscaleuserdata-staging.com
"hello from new service"
Change to use you own bucket
You can upload a local directory with the runtime_nev.working_dir
field in the Service YAML. To do so, you must also specify a remote bucket and ensure that both your laptop and cluster have network and IAM permissions to access it. For more information, view docs for accessing an Amazon S3 or Google Cloud Storage bucket.
- CLI
- job.yaml
- hello_world.py
anyscale service deploy service.yaml
(anyscale +3.3s) Query the status of the service with `anyscale service list --service-id service_sc2JFXVupTCLy6CKqcYq5rmN`.
(anyscale +3.3s) View the service in the UI at https://console.anyscale-staging.com/services/service_sc2JFXVupTCLy6CKqcYq5rmN.
name: service-demo
entrypoint: SERVE_RESPONSE_MESSAGE="hello from new service" python serve_hello.py
+ compute_config: config_with_bucket_permission
runtime_env:
- working_dir: https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip
+ working_dir: .
+ upload_path: "s3://your-bucket/path"
healthcheck_url: "/healthcheck"
from fastapi import FastAPI
from ray import serve
import os
# Use this var to test service inplace update. When the env var is updated, users see new return value.
msg = os.getenv("SERVE_RESPONSE_MESSAGE", "Hello world!")
serve.start(detached=True)
app = FastAPI()
@serve.deployment(route_prefix="/")
@serve.ingress(app)
class HelloWorld:
@app.get("/")
def hello(self):
- return msg
+ return msg + ", and a code change"
@app.get("/healthcheck")
def healthcheck(self):
return
HelloWorld.deploy()
Now when you query the service, you should see the new message.
curl -H "Authorization: Bearer SECRET_TOKEN" https://serve-session-3aruf6xjitbft3k8tnnzmdfm.i.anyscaleuserdata-staging.com
"hello from new service, and a code change"
Terminate
When we are done, let's go ahead and terminate the service using the CLI.
anyscale service terminate -n service-demo
Conclusion
In this tutorial, we have created, updated, and queried an Anyscale Service. To see more information about Services, see deploy, query and manage Anyscale services.