In deep learning, a neuron is the foundational building block of a neural network. It computes a scalar output from a vector of inputs.
Formula#
Given an input vector \(\mathbf{x} = [x_1, x_2, \dots, x_n]\), a weight vector \(\mathbf{w} = [w_1, w_2, \dots, w_n]\), and a scalar bias \(b\):
$$ z = \mathbf{w}^T \mathbf{x} + b = \sum_{i=1}^{n} w_i x_i + b $$The final output \(a\) is obtained by applying an activation function \(g(\cdot)\):
$$ a = g(z) $$Key Components#
- weights (\(\mathbf{w}\)): Determine the importance of each input feature.
- biases (\(b\)): Allows shifting the activation threshold.
- linear combination (\(z\)): Weighted sum of inputs.
- activation (\(a\)): Introduces non-linearity into the network.