Chain of thought is a style of intermediate-step generation that the LLM uses to make all the reasoning stages explicit and easier to follow.
With COT, the LLM does not just recall a fact but it rather gets to the conclusion through intermediate passages, resembling a person articulating their thoughts out loud.
Example:
Prompt: “Alice has 3 apples and Bob has 5 apples. Alice gives 1 apple to Bob. How many apples do they have together?”
Step 1: Identify initial counts (Alice = 3; Bob = 5)
Step 2: Update counts after transfer (Alice=2; Bob = 6)
Step 3: Calculate total: 2 + 6 = 8
Answer: 8 apples
Even though this can look like human thought, there’s a key difference: it is not deterministic reasoning but rather probabilistic reasoning, without guarantees of logical consistency.