Skip to main content
Version: Canary 🐤

Metrics

Anyscale and Ray provides built-in metrics for monitoring your Ray clusters and the workloads you run on them. This page describes the metrics available and how to access them.

Available metrics​

Ray metrics​

Ray comes with a set of built-in metrics. You can view the full list of metrics in the System Metrics Ray documentation

Anyscale metrics​

Anyscale adds additional metrics for your Ray application depending on the workload. For services, service-level metrics provide information on service health, request latency, and request throughput. For more information, see Monitoring a service.

For all metrics exported by Ray and Anyscale, Anyscale adds additional labels to each metric to help you filter and aggregate them. The labels include:

  • cluster_id: The ID of the Ray cluster where the metric is emitted.
  • service_id: The ID of the service where the metric is emitted if the cluster is part of an Anyscale Service.
  • service_version_name: The name of the service version where the metric is emitted if the cluster is part of a Anyscale Service.
  • prodjob_id: The ID of the production job where the metric is emitted if the cluster is part of an Anyscale Job.
  • prodjob_name: The name of the production job where the metric is emitted if the cluster is part of an Anyscale Job.
  • node_id: The ID of the node where the metric is emitted.
  • node_ip: The IP address of the node where the metric is emitted.
  • org_name: The name of the organization where the metric is emitted.
  • cloud_id: The ID of the cloud where the metric is emitted.
  • is_anyscale_hosted: A boolean indicating whether the metric is emitted from an Anyscale-hosted cloud.

Custom application metrics​

As a developer, you can add custom metrics to your applications. This is useful if you want to track the behavior of your application over time. You can use the ray.metrics module to add custom metrics to your application. For more information, see the Adding Application-Level Metrics Ray documentation

Accessing metrics​

Anyscale UI​

In the workspaces, jobs, and services pages there are tabs to view metrics.

Metrics in the Anyscale UI

Grafana​

You can access the metrics in Grafana by clicking on the "View in Grafana" button in the metrics page. Using Grafana allows you to create custom dashboards and explore the metrics using advanced queries, including filtering and aggregating by labels. See the Grafana documentation for more information.

Metrics in Grafana

Ray Dashboard​

You can access all Ray system metrics in the Ray Dashboard by clicking on the "Ray Dashboard" tab. See the Ray Dashboard documentation for more information.

Metrics in the Ray Dashboard