TutorialProvider guide

Batch API

Use Batch for evaluations and other work that can wait. It costs 50% less than synchronous calls and has a 24-hour processing window. Match results to requests by custom_id.

From
Provider guide: published by OpenAI on its own site, developers.openai.com.
Checked
, against the source

Before you start

  • An OpenAI API key, and work that does not need an immediate answer, such as evaluations, classifying a large dataset or embedding a content repository.
  • A model that supports Batch. The model reference says which do.

Steps

  1. Write a .jsonl file with one request per line: a unique custom_id, the method, the endpoint URL and the body. One file targets a single model, with at most 50,000 requests and 200 MB.
  2. Upload the file through the Files API with the purpose batch.
  3. Create the batch with the file's ID, the endpoint and the 24h completion window.
  4. Check its status as it moves from validating to in_progress, finalizing and completed, or to failed, expired or cancelled.
  5. Download the output file named by output_file_id. Failed requests are written to the file named by error_file_id.

What you get

One output line for each successful request. Lines may come back in a different order, so match each result to its request by custom_id. Batch costs 50% less than the synchronous APIs and completes within 24 hours.

Dossiers

Next read

Deep dive · related guide

Prompt caching