Fine-tuning API
These docs are for the new Anyscale design. If you started using Anyscale before April 2024, use Version 1.0.0 of the docs. If you're transitioning to Anyscale Preview, see the guide for how to migrate.
Fine-tuning allows you to take a pre-trained model and adapt it to your specific use case. You provide a dataset, and then train the model on that dataset. The fine-tuned model generates text that follows the same form as your training dataset.
Prerequisitesβ
- Upload your files with the Files API - Upload file
- Use either (1) the OpenAI SDK or (2) cURL as a REST client
The fine-tuning API is fully compatible with OpenAI's fine-tuning API, aside from listing events. You only need to setup your API base URL and API key to point to Anyscale Endpoints and you can use OpenAI's SDK to get started.
Use the OpenAI SDKβ
import openai
client = openai.OpenAI(
base_url = "https://api.endpoints.anyscale.com/v1",
api_key = "esecret_yourAuthTokenHere" # from https://app.endpoints.anyscale.com/credentials
)
Use cURLβ
export ANYSCALE_BASE_URL="https://api.endpoints.anyscale.com/v1"
export ANYSCALE_API_KEY="esecret_yourAuthTokenHere"
APIsβ
Create fine-tuning jobβ
POST https://api.endpoints.anyscale.com/v1/fine_tuning/jobs
Submits a fine-tuning job.
Request bodyβ
-
model
string required- Model being fine-tuned. See Supported models.
-
training_file
string required- ID of the
.jsonl
file corresponding to the training dataset uploaded via Files API. See Upload file.
- ID of the
-
validation_file
string (See Data Format - Validation Data)- ID of the
.jsonl
file corresponding to the validation dataset uploaded via Files API. See Upload file.
- ID of the
-
hyperparameters
objectn_epochs
integer (See Data Format - Number of epochs)- Number of epochs to train for. If not specified, it's auto-calculated based on the training dataset size.
context_length
integer (See Data Format - Context Length)- Context window size during training. If not specified, it's auto-calculated based on sequence length of the examples in the dataset.
-
suffix
string Β Defaults to user's first name- Appended to
model
along with a hash to generatefine_tuned_model
(For example:mistralai/Mixtral-8x7B-Instruct-v0.1:my_suffix:aBc1234
)
- Appended to
Returnsβ
A fine-tuning.job object
Use the OpenAI SDKβ
client.fine_tuning.jobs.create(
model="mistralai/Mixtral-8x7B-Instruct-v0.1",
training_file="file_123",
)
Use cURLβ
curl "$ANYSCALE_BASE_URL/fine_tuning/jobs" \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $ANYSCALE_API_KEY" \
-d '{
"model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
"training_file": "file_123"
}'
Get fine-tuning jobβ
GET https://api.endpoints.anyscale.com/v1/fine_tuning/jobs/{fine_tuning_job_id}
Retrieves a fine-tuning job by ID.
Path parametersβ
fine_tuning_job_id
- ID of the fine-tuning job
Returnsβ
A fine-tuning.job object
Use the OpenAI SDKβ
client.fine_tuning.jobs.retrieve("eftjob_123")
Use cURLβ
curl "$ANYSCALE_BASE_URL/fine_tuning/jobs/eftjob_123" \
-H "Authorization: Bearer $ANYSCALE_API_KEY"
List fine-tuning jobsβ
GET https://api.endpoints.anyscale.com/v1/fine_tuning/jobs
Lists all fine-tuning jobs that you have access to, from newest to oldest.
Query parameters optionalβ
after
string- ID of the fine-tuning job from the last page of results
limit
integer Β Defaults to 20- Maximum number of fine-tuning jobs to return
Returnsβ
A list of fine-tuning.job objects
{
"data": [fine-tuning.job, ...]
}
Use the OpenAI SDKβ
client.fine_tuning.jobs.list()
Use cURLβ
curl "$ANYSCALE_BASE_URL/fine_tuning/jobs" \
-H "Authorization: Bearer $ANYSCALE_API_KEY"
Cancel fine-tuning jobβ
POST https://api.endpoints.anyscale.com/v1/fine_tuning/jobs/{fine_tuning_job_id}/cancel
Cancels a fine-tuning job by ID.
Path parametersβ
fine_tuning_job_id
- ID of the fine-tuning job
Returnsβ
The fine-tuning.job object that terminated.
Use the OpenAI SDKβ
client.fine_tuning.jobs.cancel("eftjob_123")
Use cURLβ
curl "$ANYSCALE_BASE_URL/fine_tuning/jobs/eftjob_123/cancel" \
-H "Authorization: Bearer $ANYSCALE_API_KEY" \
-X 'POST'
Additional informationβ
After your job has entered succeeded
status, Anyscale sends you an email notifying you that your fine-tuned model is ready for use. See Query a model for more details on how to query your fine-tuned model.
See Supported models for a list of open source models that can be fine-tuned.
Job schedulingβ
Fine-tuning jobs process via a queue. Your job queue with pending
status and scheduled to run.
Modelsβ
fine-tuning.job
β
Attributesβ
id
string required- ID of the fine-tuning job
created_at
string required- UTC creation data and time of the fine-tuning job.
creator_id
string required- ID of the user who created the fine-tuning job
status
string required- Status of the fine-tuning job
pending
|running
|succeeded
|failed
|cancelled
model
string required- Model for fine-tuning. See Supported models.
fine_tuned_model
string required- Name of the fine-tuned model for use on
succeeded
status
For example,mistralai/Mixtral-8x7B-Instruct-v0.1:my_suffix:aBc1234
.
- Name of the fine-tuned model for use on
training_file
string required- ID of the .jsonl file used for training
validation_file
string (See Data Format - Validation Data)- ID of the .jsonl file used for validation
hyperparameters
objectn_epochs
integer (See Data Format - Number Of Epochs)- Number of epochs to train for. If not specified, it's auto-calculated based on the training dataset size.
context_length
integer (See Data Format - Context Length)- Number of tokens to use as context for each training example
result_files
array[string]- Contains ID of the
.jsonl
file that contains fine-tuning results. See Result file.
- Contains ID of the
trained_tokens
integer (see Data Format - Token Counting)- Number of tokens trained on
finished_at
string- UTC date and time of when the fine-tuning job entered
succeeded
,failed
, orcancelled
from a status ofrunning
- UTC date and time of when the fine-tuning job entered
error
string- Error message if the job
failed
- Error message if the job
hyperparameters
(if not specified during Create fine-tuning job) and trained_tokens
are available only on succeeded
status. To view these fields while the job is running
, access the result file.
Exampleβ
{
"id": "eftjob_123",
"created_at": "2023-01-01T00:00:00.123456+00:00",
"creator_id": "euser_123",
"status": "succeeded",
"model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
"fine_tuned_model": "mistralai/Mixtral-8x7B-Instruct-v0.1:my_suffix:123aBcD",
"training_file": "file_123",
"validation_file": "file_456",
"hyperparameters": {
"n_epochs": 1,
"context_length": 512
},
"result_files": ["file_789"],
"trained_tokens": 123456,
"finished_at": "2023-01-01T00:30:00.123456+00:00",
"error": null
}
Result fileβ
The result file contains the fine-tuning results in .jsonl
format.
Read the file with the Files API - Get file contents:
Use the OpenAI SDKβ
result_file_id = client.fine_tuning.jobs.retrieve("eftjob_123").result_files[0]
client.files.retrieve_content(result_file_id)
Exampleβ
{"epoch": 0, "iteration": 1, "train_loss": 2.942214250564575, "trained_tokens": 36192, "valid_loss": null, "perplexity": null, "hyperparameters": {"n_epochs": 5}, "time_since_job_start": 173.76054668426514}
...
{"epoch": 0, "iteration": 13, "train_loss": 0.5098391771316528, "trained_tokens": 8325090, "valid_loss": 0.44729286432266235, "perplexity": 1.5640722931908475, "hyperparameters": {"n_epochs": 1, "context_length": 512}, "time_since_job_start": 241.6144163608551}
If you provide a validation dataset, the job reports perplexity and validation loss at the end of each training epoch. For lines that don't correspond to the end of each epoch, the value for these metrics are null
.