Anyscale Private Endpoints allows you to seamlessly serve the most popular open source large language models (LLMs) in your cloud environment without compromising on performance. As a result, you can concentrate on customizing your LLM-powered applications while adhering to the highest security standards.
Securely add a conversational AI layer to your applications using open source models. With this tool, you can build responsive chatbots, automate customer support, analyze content, and code with AI assistance, while keeping your data within your own cloud environment for enhanced privacy and compliance.
Anyscale Private Endpoints supports the most popular open source LLMs.
Code Llama - Instruct 34B
Email the support team at firstname.lastname@example.org to request the addition of another model to Anyscale Private Endpoints.
To get a general introduction to Anyscale Private Endpoints, follow these foundational guides:
- Get started: Introductory steps for Anyscale Private Endpoints setup.
- OpenAI migration guide: Transition models and data from OpenAI to Anyscale.
- Model configuration: Customize and optimize deployed open source models.
- Observability and alerting: Tools for monitoring deployed models and setting up alerts for key metrics.
For hands-on tutorials that use specific tools and frameworks, check out this collection of starter content:
- Prompting: First steps for designing prompts and prompt templates.
- Create a chatbot: Create a simple, streaming chatbot.
- LangChain integration: Setup LangChain with Anyscale for advanced chat agents.
- Arize integration: Implement Arize for model monitoring in Anyscale.
- Weights & Biases integration: Track experiments and tune models with W&B.
- LlamaIndex integration: Enhance your LLMs by building an indexing and querying engine with LlamaIndex.
- Anyscale Endpoint Cookbook: A GitHub repo with even more examples applications built with Anyscale Private Endpoints for common use cases.
By training on your unique datasets, you can tailor a language model to understand and generate text that aligns with your organization's tone, terminology, and objectives. This API simplifies the process of adapting open source models to recognize the nuances of your business's communications, support queries, and technical documents.
The following models are available for fine-tuning:
Llama 2 - 7 B
Llama 2 - 13 B
- Get started: How to run a basic fine-tuning job.
For additional assistance or feature requests for Anyscale Private Endpoints, contact the support team at email@example.com.