Linear algebra is the data format
Data becomes tensors: a scalar is rank 0, a vector rank 1, a matrix rank 2, and a batch of images rank 4. Learning is mostly repeated matrix multiplication followed by a non-linearity, which is why GPUs — massively parallel matrix engines — dominate the field.
Calculus is the learning signal
Training minimises a loss function. The gradient of the loss with respect to each parameter says which direction increases the error, so we step the opposite way. The chain rule, applied backwards through the network, is what backpropagation implements.
Probability is the language of uncertainty
Classifiers output distributions, not answers. Cross-entropy measures the distance between the predicted distribution and the truth. Bayes' rule underpins priors, calibration, and reasoning about when a confident model is actually wrong.
import numpy as np
def softmax(z):
e = np.exp(z - z.max())
return e / e.sum()
def cross_entropy(probs, label):
return -np.log(probs[label] + 1e-12)
logits = np.array([2.0, 0.5, -1.0])
p = softmax(logits)
print(p, cross_entropy(p, 0))