The self-attention mechanism allows the model to weigh the importance of different words or tokens in a sequence relative to each other, allowing to capture contextual relationships and long-range dependencies within the input data.
This weighting mechanism enhance the ability of the model to generate coherent and relevant output.