The maths you actually need

Arjun had spent five months building on top of other people’s models. The support bot answered allergy questions. The RAG system searched the 200-page operations handbook. The agents booked delivery slots. Every one of them called an API, and every one of them worked.

So when Priya wrote “Part 2: Deep Learning” on the whiteboard on Monday morning, he frowned. “Why go backwards? The models already exist.”

“Because the next three things Meera wants don’t exist as an API,” Priya said. She wrote them underneath: forecast tomorrow’s croissant sales for each store, sort the pastry photos on the online shop, read the supplier invoices. “We train those ourselves. And before any of them, a small one: warn the store when an online order is likely to arrive late.”

That last one had a number attached. About 600 online orders a week, roughly 120 of them late, and an average refund of ₹350 each. Meera had done the sum herself: ₹42,000 a week handed back, plus the angry emails. “If we knew an order was going to be late before the rider left,” she said, “we could call the customer and offer a discount instead of a refund.”

Then she looked at the board. “Is this the part with the maths? I failed calculus in college.”

“You need four ideas,” Priya said. “Not a degree. Arjun needs them well enough that when training goes wrong at 11pm, he knows where to look.”

Arjun’s rule of thumb

Arjun did not want to wait for maths. Over lunch he wrote a “late score” himself. Distance matters, number of items matters (bigger orders take longer to pack), order value matters (big orders are fiddly cakes). Add them up, and flag anything above a threshold.

def late_score(distance_km, items, order_value_rupees):
    return distance_km + items + order_value_rupees

orders = [
    {"id": "A", "distance_km": 12, "items": 2,  "order_value_rupees": 1800},
    {"id": "B", "distance_km": 14, "items": 1,  "order_value_rupees": 250},
    {"id": "C", "distance_km": 2,  "items": 9,  "order_value_rupees": 1200},
]
for o in orders:
    score = late_score(o["distance_km"], o["items"], o["order_value_rupees"])
    print(o["id"], score, "LATE?" if score > 1000 else "fine")

A 1814 LATE?
B 265 fine
C 1211 LATE?

Order C was a cake going two kilometres down the road. Flagged. Order B was a single croissant going 14 kilometres across the city in the rain. Not flagged. He looked at the output for a while.

“The rupees are drowning everything else,” he admitted. “A ₹1,200 order scores 1,200 before distance even gets a vote.”

“Right,” said Priya. “And that’s three separate problems. The features are on different scales. Every feature counts equally, which nobody believes. And your answer is ‘yes’ or ‘no’, when what Meera needs is ‘how likely’. Each of those problems has a bit of maths that fixes it. Then there’s a fourth problem: you picked the formula by hand. We want the computer to pick it. That’s the calculus.”

Making the numbers fair: statistics

“Imagine a cooking competition,” Priya said, “with three judges. One scores out of 10, one out of 100, one out of 10,000. You add up the scores. Who decides the winner?”

“The 10,000 judge.”

“Every time. The others might as well go home. So before adding anything, we put every judge on the same scale.”

Two statistics do it. The mean (the average) tells you where a feature’s values sit. The standard deviation (roughly, how far a typical value is from the mean) tells you how spread out they are:

$$\mu =\frac{1}{n}\sum _i{x}_i\sigma =\sqrt{\frac{1}{n}\sum _i(x_i-\mu {)}^2}$$

Arjun pulled a month of Crumb’s orders and computed them:

Feature	Mean $\mu$	Standard deviation $\sigma$
Distance	8 km	4 km
Items	6	3
Order value	₹1,000	₹400

Now standardise each value (rescale it so the feature has mean 0 and standard deviation 1) with a z-score:

$$z=\frac{x-\mu }{\sigma }$$

A z-score answers “how many standard deviations from normal is this?”. Order A becomes:

• Distance: $(12-8)/4=1.0$, a bit further than usual.
• Items: $(2-6)/3=-1.33$, noticeably small.
• Value: $(1800-1000)/400=2.0$, quite big.

Now all three speak at the same volume. Rain is already a 0 or 1, so it stays as it is: order A is going out in the rain, rain = 1.

Letting each feature have its own say: linear algebra

Next problem: Arjun’s formula counted every feature equally. Priya drew order A as a single row of numbers.

$$\mathbf{x}=[1.0,-1.33,2.0,1]\text{(distance, items, value, rain)}$$

An ordered list of numbers like this is a vector. Position 0 always means distance, position 3 always means rain.

“Now suppose we believe distance matters a lot, items a little, value a little, and rain a lot. Write those beliefs as another vector.”

$$\mathbf{w}=[0.8,0.3,0.1,1.2]$$

These are weights (one multiplier per feature, saying how loudly that feature’s vote counts). To combine them, multiply matching positions and add. That is the dot product:

$$\mathbf{w}\cdot \mathbf{x}=\sum _i{w}_i{x}_i=0.8(1.0)+0.3(-1.33)+0.1(2.0)+1.2(1)=0.8-0.4+0.2+1.2=1.8$$

Add one more number, a bias (a fixed nudge up or down, here $b=-0.5$, meaning “most orders are on time, so start from a slightly negative opinion”), and you get the score:

$$z=\mathbf{w}\cdot \mathbf{x}+b=1.8-0.5=1.3$$

That is one artificial neuron. A weighted vote plus a starting opinion. It really is that small.

Many orders and many neurons at once: matrices

Crumb gets hundreds of orders. Stack them as rows and you get a matrix (a grid of numbers, rows by columns). Four orders with four features each is a $(4\times 4)$ matrix. A network also has many neurons, each with its own weight vector. Stack those as columns and multiplying the two matrices computes every neuron’s score for every order in one operation.

The one rule worth memorising is about shapes:

$$(a\times b)\cdot (b\times c)=(a\times c)$$

The inner numbers must match, and they “cancel”. 600 orders with 4 features, $(600\times 4)$, times the weights of 16 neurons, $(4\times 16)$, gives $(600\times 16)$: sixteen scores for each order. When PyTorch shouts mat1 and mat2 shapes cannot be multiplied (600x4 and 5x16), it is telling you the inner numbers (4 and 5) disagree. You will see that error a lot. Now you can read it.

Where the analogy of “weights as beliefs” breaks: in a real model nobody chooses the weights. They start random and get learned, and after training most individual weights mean nothing a human could name. Priya’s hand-picked numbers are only for today.

Turning a score into a chance: probability

A score of 1.3 is not useful to the store manager. “Is 1.3 bad?” What she wants is “79% chance this is late, call the customer”.

A probability is a number between 0 and 1 saying how likely something is. For a yes/no question you squash any score into that range with the sigmoid function:

$$\sigma (z)=\frac{1}{1+e^{-z}}$$

Big positive scores become close to 1, big negative scores close to 0, and 0 becomes exactly 0.5. For order A:

$$\sigma (1.3)=\frac{1}{1+e^{-1.3}}=\frac{1}{1+0.273}\approx 0.786$$

The neuron says: about a 79% chance order A will be late.

When there are more than two possible answers, like a photo that could be a croissant, a pain au chocolat or a danish, you want a full probability distribution (a chance for every option, each between 0 and 1, all adding up to exactly 1). The tool for that is softmax: raise $e$ to each score, then divide by the total.

$$\text{softmax}(z_i)=\frac{e^{z_i}}{{\sum }_j{e}^{z_j}}$$

Scores of $[2.0,1.0,0.1]$ become $e^{2.0}\approx 7.39$, $e^{1.0}\approx 2.72$, $e^{0.1}\approx 1.11$. The total is 11.21, so the probabilities are about $[0.66,0.24,0.10]$. The biggest score wins, but nobody gets zero. Sigmoid is really softmax for the two-option case.

Grading a guess

Here is the bit that makes learning possible. Once you know what actually happened, you can grade the guess with one number. Take the probability the model gave to what really happened, and compute $-\log$ of it. This is cross-entropy loss (a score of how wrong a probability was; lower is better):

• If order A really was late, the model gave “late” 0.786. Loss: $-\ln (0.786)\approx 0.24$. A small penalty.
• If order A was actually on time, the model gave “on time” only 0.214. Loss: $-\ln (0.214)\approx 1.54$. A big penalty.

Confident and right costs almost nothing. Confident and wrong costs a lot. That is exactly the pressure you want on a model.

Which knob to turn: calculus

Order A, it turned out, arrived on time. The rider took a shortcut. So the neuron’s 79% was a bad call, and the loss is 1.54. Which of the five numbers (four weights and the bias) should change, and which way?

“Before the neuron, let’s do something easier,” Priya said. “Croissants. Bake them too short and they’re doughy. Too long and they’re burnt. Taste peaks at 18 minutes.” She wrote:

$$\text{taste}(t)=100-(t-18{)}^2$$

“You’re at 15 minutes. Should you bake longer or shorter?”

“Longer, obviously. But the computer doesn’t know the curve’s shape.”

“It doesn’t need to. It only needs the slope where it’s standing.”

A derivative is the slope of a function at a point: if I nudge the input a tiny bit, how much does the output change, and in which direction? For the taste curve, the derivative is $-2(t-18)$.

• At 15 minutes: $-2(15-18)=+6$. Positive slope, so more time means more taste. Bake longer.
• At 20 minutes: $-2(20-18)=-4$. Negative, so bake shorter.
• At 18 minutes: 0. You are at the top.

The slope alone tells you which way to move. You never need to see the whole curve.

A neuron has five knobs, not one. Each knob gets its own slope (“if I nudge only this weight, how does the loss change?”), called a partial derivative. Put all five in a list and you have the gradient (a vector of slopes, one per parameter, pointing in the direction that increases the loss fastest). To reduce the loss, step the opposite way.

Blame passing down a chain

To find each slope, you need one more idea. Oven temperature affects crust colour. Crust colour affects the customer’s star rating. If one extra degree darkens the crust by 0.5 units, and one unit of colour adds 0.2 stars, how many stars does one degree add?

$$\frac{d\text{rating}}{d\text{temp}}=\frac{d\text{rating}}{d\text{colour}}\times \frac{d\text{colour}}{d\text{temp}}=0.2\times 0.5=0.1$$

That is the chain rule (when A affects B and B affects C, the effect of A on C is the slopes multiplied together). The neuron is a chain too: weights affect the score $z$, $z$ affects the probability $p$, $p$ affects the loss. Multiply the slopes along that chain and, for sigmoid with cross-entropy, it simplifies beautifully:

$$\frac{\partial L}{\partial w_i}=(p-y)x_i\frac{\partial L}{\partial b}=p-y$$

where $y$ is what actually happened (1 for late, 0 for on time). Read it in words: how wrong the probability was, times how much this feature was “on” for this order.

Following order A all the way through

Now trace the whole thing with real numbers. Order A arrived on time, so $y=0$, and $p-y=0.786-0=0.786$.

Parameter	Feature value $x_i$	Gradient $(p-y)x_i$	Meaning
Distance weight	1.0	+0.786	Lower it
Items weight	−1.33	−1.048	Raise it
Value weight	2.0	+1.572	Lower it, most of all
Rain weight	1	+0.786	Lower it
Bias	(always 1)	+0.786	Lower it

Take a small step against the gradient, with a step size of 0.1: new value = old value − 0.1 × gradient.

Parameter	Before	After
Distance weight	0.800	0.721
Items weight	0.300	0.405
Value weight	0.100	−0.057
Rain weight	1.200	1.121
Bias	−0.500	−0.579

Run order A through again: the score drops from 1.30 to 0.61, the late probability from 0.786 to 0.648, and the loss from 1.54 to 1.04. One nudge, and the neuron is already less wrong about this order. Do that for thousands of orders, thousands of times, and you have training. That is lesson 4 of this module.

The same trace in code

Install NumPy (uv add numpy, or pip install numpy) and run this. Each block is one idea from above.

import numpy as np

# Order A: distance 12 km, 2 items, ₹1,800, raining
order = np.array([12.0, 2.0, 1800.0, 1.0])

# 1. Statistics: standardise the first three features
mean = np.array([8.0, 6.0, 1000.0])
std = np.array([4.0, 3.0, 400.0])
x = order.copy()
x[:3] = (order[:3] - mean) / std
print(x.round(2))                      # [ 1.   -1.33  2.    1.  ]

# 2. Linear algebra: weighted vote plus bias
w = np.array([0.8, 0.3, 0.1, 1.2])
b = -0.5
z = x @ w + b                          # @ is the dot product
print(round(z, 3))                     # 1.3

# 3. Probability: squash to a chance, then grade it
p = 1 / (1 + np.exp(-z))
y = 0                                  # it actually arrived on time
loss = -np.log(1 - p) if y == 0 else -np.log(p)
print(round(p, 3), round(loss, 3))     # 0.786 1.541

# 4. Calculus: gradient, then one small step downhill
grad_w = (p - y) * x
grad_b = p - y
w = w - 0.1 * grad_w
b = b - 0.1 * grad_b
p_new = 1 / (1 + np.exp(-(x @ w + b)))
print(grad_w.round(3))                 # [ 0.786 -1.048  1.572  0.786]
print(round(p_new, 3))                 # 0.648

Every number matches the tables. This twenty-line script contains, in miniature, every idea a neural network uses. Bigger networks have more neurons, more layers and better ways to compute the gradients, but nothing fundamentally new.

Where this maths bites you

Knowing these four ideas is what lets you diagnose problems. Here are the ones Arjun hit in the following weeks:

• One neuron draws straight lines. A single weighted vote can say “more distance, more late”. It cannot say “late between 6pm and 9pm, fine otherwise”, because that is a bump, not a slope. Stacking neurons in layers, with a bend between them, fixes it. That is lesson 3.
• Standardising with the wrong numbers. If you compute the mean and standard deviation on all your data, including the orders you later test on, you have leaked information from the test into training. Compute them on training data only, save them, and reuse the same numbers when the model runs in production. Forgetting to save them is a classic bug: the live model receives raw rupees and outputs nonsense.
• Overflowing exponentials. $e^{1000}$ is too big for a computer, so a naive softmax on large scores returns nan. Libraries subtract the largest score first, which changes nothing mathematically. Use the library’s softmax, never your own, in real code.
• Treating a probability as a fact. “79% late” is the model’s belief, shaped entirely by its training data. Order A was on time. A good model is right about 79% of the time when it says 79%, which is a property called calibration, and it has to be checked, not assumed.
• Forgetting the derivative is local. The slope says which way is downhill from where you stand, not where the bottom is. That is why training takes many small steps, and why step size matters so much. Next lesson.

Try it yourself

1. Order B: 14 km, 1 item, ₹250, raining. Standardise it with the same means and standard deviations, and compute its score and late probability using the original weights [0.8, 0.3, 0.1, 1.2] and bias −0.5. Then do the same for order C: 2 km, 9 items, ₹1,200, not raining.
2. Did the neuron get Arjun’s two bad calls right this time?

Show the answer

Order B: $x=[1.5,-1.67,-1.88,1]$. Score $=0.8(1.5)+0.3(-1.67)+0.1(-1.88)+1.2(1)-0.5=1.2-0.5-0.19+1.2-0.5=1.21$. Probability $\approx 0.77$. Flagged as likely late.

Order C: $x=[-1.5,1.0,0.5,0]$. Score $=-1.2+0.3+0.05+0-0.5=-1.35$. Probability $\approx 0.21$. Fine.

Yes: the long rainy croissant run is flagged and the short cake delivery is not. Same data, fair scales and sensible weights.

Common mistakes

• Treating maths as a gate you must pass first. You do not need to derive anything to train good models. Learn these four ideas, let code make them concrete, and come back to the notation when it stops being scary.
• Ignoring shapes. Most deep learning bugs are shape bugs. Print .shape often, and write the expected shape in a comment next to each line that changes it.
• Reading a probability as certainty. A model can say “97% croissant” about a photo of a cat, because cats were never one of its options.
• Standardising with test or production data. Compute $\mu$ and $\sigma$ from training data only, save them with the model, and apply the same numbers everywhere.
• Believing the gradient points at the answer. It points downhill from here, nothing more.

Recap

• Statistics: mean and standard deviation let you standardise features so none drowns out the others.
• Linear algebra: a neuron is a dot product of weights and features plus a bias; matrices do many at once, and the inner shapes must match.
• Probability: sigmoid and softmax turn scores into chances; cross-entropy punishes confident wrong answers hard.
• Calculus: a derivative is a slope; the gradient is every parameter’s slope; the chain rule multiplies slopes along a chain.
• One training step is: predict, grade with the loss, compute the gradient, step each parameter a little the other way.

At the end of the day Meera asked Arjun to explain it, “like I’m a customer, not an engineer”.

He thought about it. “We describe each order with a few numbers, and make them fair so the rupees don’t shout. The model gives each number a weight, adds them up, and turns the total into a chance of being late. When it’s wrong, we check which way each weight should move to make it less wrong, and move them a tiny bit. Repeat a few thousand times.”

“That’s it?”

“That’s the whole engine,” said Priya. “The interesting question is how big each tiny bit should be. Tomorrow we go for a walk in the fog.”

Go deeper

• NumPy documentation for the array operations used above.
• 3Blue1Brown has beautiful visual series on linear algebra and calculus. Watch them for intuition, not exams.
• PyTorch tutorials, where these same ideas reappear as tensors in two lessons’ time.