Cancelling a generation

Cancelling a generation interrupts a model call in progress, for example when the user leaves the page or clicks stop. It frees resources on the server side and limits costs, because generation is billed by the number of tokens produced.

Example

const ctrl = new AbortController();
fetch(url, { signal: ctrl.signal });
ctrl.abort();

Related concepts

The whole glossary