language model

A Language Model encodes statistical information about one or more languages. Intuitively, this information tells us how likely a word is to appear in a given context.

There are two main types of language models:

• Masked language models
• Autoregressive language models