A neural network is a computational model inspired by the biological brain structure, consisting of interconnected layers of neurons that map input features to target outputs.
A standard feedforward neural network (FNN) is organized into three main types of layers:
- Input Layer: Receives the raw input vector \(x\).
- Hidden Layer(s): Extracts latent representations using weighted sums and activation function.
- Output Layer: Produces the final prediction \(\hat{y}\).
The core mechanism of the learning process of a neural network operates in a two-step feedback loop:
- forward propagation that computes predictions and measures error via a loss function
- backpropagation that calculates gradients of the loss with respect to all weights using gradient descent to minimize the loss