Skip to main content

Accessing logs

Anyscale offers multiple ways to access logs for your Ray clusters.

Log viewer​

The log viewer allows you to view and download log files generated from your Ray apps. To access the log viewer UI, select the Logs tab in the workspace, job or service detail page.

Services logs without log query

The limit of this basic log viewer is that you can only view logs from individual log files. When you download logs, all the log files on the cluster will be downloaded.

Log ingestion and query​

Log ingestion and query is an optional feature that allows you to search, filter, query logs more easily. Any cluster created after enabling this feature sends logs to the Anyscale Control Plane, where they're retained for 30 days. You may incur extra data transfer costs from your cloud provider by enabling this feature.

It is enabled by default for Anyscale Hosted clouds. For self-hosted clouds, you must opt-in with the following command:

# Change the `cloud_id` to your cloud ID
anyscale cloud config update --cloud-id CLOUD_ID --enable-log-ingestion

See the CLI reference for more information.

To query logs, select the Logs tab in the workspace,job or service detail page.

Service logs

By default, the logs shown are from the last hour with no filters. You can change the time range by selecting an end time and window to look back.

To filter the logs, use the search bar to find specific keywords, request IDs, and regular expression patterns.

Log query example

note

To disable log ingestion and query, run anyscale cloud config update --cloud-id CLOUD_ID --disable-log-ingestion. Subsequently, created Ray clusters won't send logs to the Anyscale Control Plane, however, Anyscale retains previously sent logs for 30 days.

Ray Dashboard​

Ray Dashboard is a consolidated tool for monitoring and debugging your applications available when the Ray cluster is running. To access the logs, select the Ray Dashboard tab and then Logs tab.

For information on the log files, see Configuring logging.

Logs CLI​

To download logs, even after the cluster has terminated, run the following CLI command:

# Change `cluster_id` to your cluster ID
anyscale logs cluster --id CLUSTER_ID --download --download-dir /tmp

See the CLI reference for more information and options.

Find logs​

Anyscale stores logs in {organization_id}/{cloud_id}/logs and /logs folders. The /logs folder is a legacy location. Anyscale sends most logs to the {organization_id}/{cloud_id}/logs folder. Anyscale stores all job logs, web terminal command logs, and Ray logs. For performance reasons, Anyscale stores logs in various formats for different use cases. For example, when streaming logs, Anyscale may produce many small files to allow for fresher data to be downloaded by the user.