Quick method for adding an S3 bucket
Context
Setting up an S3 bucket is necessary for doing many things on Ray on Anyscale, including:
- General data storage that lasts beyond cluster lifespan
- Doing model checkpointing with Ray Tune or RLlib
Use this page if you:
- Bring your own compute (AWS).
- Only need one bucket for your Ray application on Anyscale.
Setting up
Prerequisites
- Your AWS Account ID. Should be in upper right-hand corner of your AWS console when you click on your username dropdown.
- An AWS user with EC2/IAM/S3 access.
Now let's start the steps. Note that directions are slightly different depending on your cloud type.
Steps: Bring your own cloud (AWS)
- Log into your AWS account
- Visit this link to be guided through creating an S3 bucket that will be read/writeable by your Anyscale compute instances. You will input:
- Region you want your S3 bucket in
- Anyscale cloud ID (in lowercase)
- Easy way to lowercase your cloud ID:
python -c "import sys; print(sys.argv[1].lower())" cld_XXXXXXXXXXXXXXXXXXXXXXXX
- Easy way to lowercase your cloud ID:
- A "ClusterRole" name
- Format:
arn:aws:iam::<your_aws_account_id>:role/ray-autoscaler-v1
- Format:
- In your compute config: add this "ClusterRole" information so your cloud can give your Ray worker nodes access.
- JSON blob to be added to the
Advanced Configuration
section of your compute config:{ "IamInstanceProfile": { "Arn": "<ClusterRole ARN string with 'role' replaced by 'instance-role'>" } }
. Learn more here about advanced configuration of compute configs.- To clarify, in the compute config, you want your ClusterRole ARN string to be slightly different than the one you input into the form:
arn:aws:iam::<your_aws_account_id>:instance-profile/ray-autoscaler-v1
- In sed parlance:
s/role/instance-profile/g
- To clarify, in the compute config, you want your ClusterRole ARN string to be slightly different than the one you input into the form:
- JSON blob to be added to the
For example ...

What if I want a more advanced setup?
If you'd like to assign compute roles that have more complex setups, or bring your own IAM role for compute instances, that's totally fine too.