transformer

The transformer architecture is a deep neural network architecture introduced in the 2017 paper Attention Is All You Need.

Originally, the transformer was developed for machine tranlsation, specifically to translate English texts to German and French. Its architecture consists of two submodules:

• the encoder (that performs the encoding)
• the decoder (that performs the decoding)

Both of them consist of many layers connected by the self-attention mechanism.