Cookbook

A recipe does one task from start to end, with a program or a script to copy. A guide covers one part of the API. Every recipe reads the two shell variables that the Quickstart sets up, and says what else it needs. A recipe with a whole program or notebook links to it as a file to download.

• Run a Job from a shell script submits a Job with curl, waits until it ends, and saves its result to a file.
• Download a harvest from a shell script submits a harvest with curl, waits until it ends, and downloads its files into a folder.
• Handle the end of your Budget detects a lowered max_tokens and handles the 429 errors in Python.
• Score a text adds up the log probabilities of a text's tokens to get its log-likelihood.
• Evaluate with Inspect runs an Inspect eval on a Base model through POST /v1/completions.
• Steer with a contrast vector builds a Steering vector from two sets of prompts, and steers a Base model with it.
• Run the workshop notebook goes from a first Completion to a steered Base model in a Jupyter notebook, with exercises.

Where an output shows text that meta-llama/Llama-3.1-8B wrote, it is one sample, and your output differs. Why repeats differ explains how to get the same tokens again, and why vLLM's batching can still change one.