Byte Pair Encoding builds a subword vocabulary by starting from characters (or bytes) and merging, at each step, the most frequent pair of symbols in the corpus. Rare words are split into known pieces, which keeps the vocabulary at a reasonable size with no out-of-vocabulary words. It is widely used in LLMs.