Sample many rollouts

In this tutorial, you sample many rollouts of two prompts. A rollout is one continuation that the model samples. You sample them in one request, then in many requests at once. At the end, you send the same requests as a Job, which runs in the background and keeps the results.

You need an API key with tokens in ACS_API_KEY. Your first steered Completion gets you one. You also need curl, jq, Python, and the openai SDK.

Sample two prompts in one requestCopy Anchor Link

Send a list of prompts, and set n to the number of choices for each prompt. Choice i of prompt p has the index p * n + i.

curl "$ACS_API_BASE/completions" \
  -H "Authorization: Bearer $ACS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-8b", "prompt": ["The sky is", "The grass is"], "max_tokens": 2, "n": 2}' \
  | jq -c '.choices[] | [.index, .text]'

The curl command prints the index and the text of each of the 4 choices:

[0," falling,"]
[1," always blue"]
[2," greener"]
[3," . ."]

The Python program prints the number of each choice's prompt, the number of the choice for that prompt, and its text. Then it prints the prompt tokens and the generated tokens of the whole request:

0 0 ' a-going'
0 1 ' the limit'
1 0 ' always gre'
1 1 ' good,'
8 8

usage adds up the tokens of all the prompts and all the choices. prompt_tokens counts each prompt once, whatever n is.

At temperature 1, the model samples each choice on its own, so the texts usually differ.

The Completion comes back when its last choice ends. One Hold covers all its choices, as The Hold explains.

Send many requests at onceCopy Anchor Link

Send each request on its own connection. Each one comes back when its own Completion ends. These examples send the same request 8 times at once.

for i in 1 2 3 4 5 6 7 8; do
  curl -s "$ACS_API_BASE/completions" \
    -H "Authorization: Bearer $ACS_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"model": "llama-8b", "prompt": "The sky is", "max_tokens": 2}' \
    | jq -c '.choices[0].text' &
done
wait

The curl commands print the text of each Completion, as jq reads it:

" the limit"
" no place"
" no longer"
" blue sometimes"
" the color"
" the limit"
" a wonder"
" the limit"

The Python program sends 8 requests of its own, and prints their texts:

' a distinct'
' growing over'
' falling!'
' clear and'
' melty'
' bright,'
' brightly lit'
' falling...'

Your API key's traffic limits set how many requests run at once. Each running request holds its tokens until it ends, so the requests share what you have left. Handle the end of your Budget shows what to do when they no longer fit.

Run the same work as a JobCopy Anchor Link

A Job runs one Completion request in the API and keeps its result. Your script can stop, and the Job keeps running.

1. Submit the two prompts of the first step as one request with n: 2:

   Copy Text

   curl "$ACS_API_BASE/completion-jobs" \
     -H "Authorization: Bearer $ACS_API_KEY" \
     -H "Content-Type: application/json" \
     -H "Idempotency-Key: rollouts-001" \
     -d '{"model": "llama-8b", "prompt": ["The sky is", "The grass is"], "max_tokens": 2, "n": 2}'

   The API answers 202 with the queued Job:

   Copy Text

   {
     "cancel_requested": false,
     "created_at": "2026-11-02T18:35:08.037+00:00",
     "expires_at": null,
     "finished_at": null,
     "id": "01a19d8b-338e-7a52-9329-6190679221fa",
     "model": "llama-8b",
     "request_id": "req_job_01a19d8b338e7a5293296190679221fa",
     "result_expired": false,
     "result_url": "/v1/completion-jobs/01a19d8b-338e-7a52-9329-6190679221fa/result",
     "started_at": null,
     "status": "queued",
     "status_url": "/v1/completion-jobs/01a19d8b-338e-7a52-9329-6190679221fa",
     "usage_pending": false
   }

2. Put the id in a shell variable for the next commands, for example JOB_ID=01a19d8b-338e-7a52-9329-6190679221fa.

3. Check on the Job:

   Copy Text

   curl "$ACS_API_BASE/completion-jobs/$JOB_ID" \
     -H "Authorization: Bearer $ACS_API_KEY"

   When the Job ends, its status is succeeded:

   Copy Text

   {
     "id": "01a19d8b-338e-7a52-9329-6190679221fa",
     "request_id": "req_job_01a19d8b338e7a5293296190679221fa",
     "model": "llama-8b",
     "status": "succeeded",
     "cancel_requested": false,
     "created_at": "2026-11-02T18:35:08.037+00:00",
     "started_at": "2026-11-02T18:35:08.065+00:00",
     "finished_at": "2026-11-02T18:35:08.132+00:00",
     "expires_at": "2026-11-03T18:35:08.132+00:00",
     "result_expired": false,
     "status_url": "/v1/completion-jobs/01a19d8b-338e-7a52-9329-6190679221fa",
     "result_url": "/v1/completion-jobs/01a19d8b-338e-7a52-9329-6190679221fa/result",
     "usage_pending": false
   }

4. Fetch the result, and print the text of each choice with jq:

   Copy Text

   curl "$ACS_API_BASE/completion-jobs/$JOB_ID/result" \
     -H "Authorization: Bearer $ACS_API_KEY" | jq -c '[.choices[].text]'

The result is one Completion with the n choices of each prompt, in the order of prompt. If the request failed, the result is the error object with its own status, so check for an error key before you read choices.

The Job takes its Hold when it runs, not when you submit it, as Holds of background work explains.

Next stepsCopy Anchor Link

Run a Job shows how to check on, cancel, and fetch a Job, and read Activations in one. Run a Job from a shell script submits a Job, waits until it ends, and saves its result. Handle the end of your Budget keeps a script running near the end of your tokens.