Architecture on davethehuman.com

Attention Is All You Need

Mon, 17 Aug 2026 00:00:00 +0000

The paper that introduced the self-attention mechanism.

BERT

Mon, 17 Aug 2026 00:00:00 +0000

BERT (Bidirectional Encoder Representations from Transformer) is a variant of the transformer architecture built upon the original encoder submodule. It specialises in masked word prediction and it is very strong in classification tasks, like sentiment prediction and document categorization.

GPT

Mon, 17 Aug 2026 00:00:00 +0000

GPT (Generative Pretrained Trasnsformer) is a variant of the transformer architecture built upon the original decoder submodule and is designed for generative tasks. It was introduced by the paper “Improving Language Understanding by Generative Pre-Training” and specialises in generating texts and its application includes machine translation, text summarization, fiction writing, writing computer code, and more.

This type of models were primarily designed and trained for text completion tasks but they showed very versatile capabilities, both with zero-shot and few-shot learning tasks.

Improving Language Understanding by Generative Pre-Training

Mon, 17 Aug 2026 00:00:00 +0000

The paper that introduced GPT.

transformer

Mon, 17 Aug 2026 00:00:00 +0000

The transformer architecture is a deep neural network architecture introduced in the 2017 paper Attention Is All You Need.

Originally, the transformer was developed for machine tranlsation, specifically to translate English texts to German and French. Its architecture consists of two submodules:

the encoder (that performs the encoding and produces an embedding representation)
the decoder (that performs the decoding)

Both of them consist of many layers connected by the self-attention mechanism. Famous variants of the transformer architecture are BERT and GPT.

agentic loop

Tue, 05 May 2026 00:00:00 +0000

The agentic loop consists of the core components of an AI agent:

Brain (or reasoning engine)
Planning: the agent uses techniques like COT or ReAct to decide what to do next based on previous outcomes
Memory:
- Short-term: the immediate context window (current conversation)
- Long-term: vector databases or logs that allow the agent to “remember” past experiences across different sessions
Tools (capabilities): APIs, web search, code execution environments, or database access. Tools allow the agents to affect the real world (e.g. booking a flight, sending an email, etc.)

AI agent

Tue, 05 May 2026 00:00:00 +0000

An AI agent is an autonomous software system that uses a reasoning engine (typically a Large Language Model) to perceive its environment, reason about goals, and execute multi-step actions using external tools to achieve a specific objective. It basically uses an LLM as its brain.

Unlike standard LLMs, which are reactive (answering prompts), agents are proactive (pursuing goals). Some example differences between them:

capabilities: the standard LLM can generate text and code, and the AI agent can get to action execution via APIs/Tools
logic: the LLM works with one-shot generation while the AI agent can do iterative planning and correction
memory: a standard LLM resets at every session while the AI agent’s memory persist via external storage

The agentic loop identifies the core components of AI agents.