Skip to main content

Integrations

Data Scientists and Engineers have an array of third-party tools and libraries which help in their daily work. Some of the ones we have seen so far have been

Each of these tools have a different pattern for integrating with Anyscale and Ray. These details should assist in developing your integrations.

Fundamentally there are two integration types:

  • Code-level Integrations in which you'll integrate with a particular tool by modifying your code
  • Service-level Integrations in which you'll integrate with a particular tool by setting some configuration that will automatically log information.

Code-level integrations​

Most of the code that data scientists and ML engineers use come from third-party libraries and are imported and leveraged from within the Python application. Many integrations with third-party tools are no different. With an API token in hand, all it takes for most integrations is to

  • set your token in a runtime environment variable
  • include the third party integration as a dependency
  • use logging statements or other integrations

Service-level Integrations​

To enable sevice-level integrations, you may need to install corresponding agents or programs on every node of your clusters. The step may include

  • install required libraries or packages in your cluster environments
  • use post-build command to install or initialize the integrations with your API tokens.