Choosing a model

Choosing a model means balancing quality, size, required memory, speed, supported languages, context length, and license. You start from the need, test candidates on an evaluation set, and keep the smallest model that reaches the required quality.

Example

Test two sizes of the same model on 50 real cases and keep the smallest one that is good enough.

Related concepts

The whole glossary