Forecasting with RNNs

Every morning at 5am, the Koregaon Park store’s baker decides how many croissants to make. Each one costs ₹38 in butter, flour and labour and sells for ₹120. Whatever is left at 8pm goes to the staff or to a shelter. Whatever is not baked is a customer who leaves with nothing, or worse, with a coffee and a bad mood.

The store manager’s method is honest and old: “same as the same day last week, plus a bit if it feels busy.” Meera pulled October’s numbers. On average, 24 croissants a day went unsold, about ₹28,000 of waste over the month. And on nine days they ran out before 6pm.

“You built a model that reads temperature logs minute by minute,” she said to Arjun, dropping eighteen months of daily sales on his desk. “This is a sequence too. How many do we bake tomorrow?”

The data, and the manager’s number to beat

Crumb’s real sales are confidential, so the course uses a generator that behaves like them. It produces about three years of daily croissant sales for one store, from January 2024 to October 2026, with everything a real store has: weekends much busier than weekdays, a store growing about 10% a year, cooler months selling more, monsoon rain cutting sales by about a fifth, festival days (Diwali, Christmas, Holi) selling about 70% more, and day-to-day noise.

# sales.py
import math
from datetime import date, timedelta
import torch

FESTIVALS = {date(2024, 11, 1), date(2024, 12, 25), date(2025, 10, 21), date(2025, 12, 25),
             date(2026, 3, 4), date(2026, 11, 8)}           # Diwali, Christmas, Holi

def make_sales(start=date(2024, 1, 1), days=1035, seed=7):
    """Fake daily croissant sales for one store.
    Returns a list of dates and a (days, 4) tensor: sales, day_of_week, festival, rain."""
    g = torch.Generator().manual_seed(seed)
    rows, dates = [], []
    for t in range(days):
        d = start + timedelta(days=t)
        dow = d.weekday()                                    # 0 = Monday
        weekly = [0.85, 0.82, 0.88, 0.92, 1.05, 1.35, 1.45][dow]
        monsoon = d.month in (6, 7, 8, 9)
        rain = float(torch.rand(1, generator=g).item() < (0.65 if monsoon else 0.08))
        trend = 1 + 0.10 * t / 365                           # growing ~10% a year
        yearly = 1 + 0.08 * math.cos(2 * math.pi * (d.timetuple().tm_yday - 15) / 365)
        festival = float(d in FESTIVALS or (d - timedelta(days=1)) in FESTIVALS)
        mean = 140 * weekly * trend * yearly * (0.78 if rain else 1.0) * (1.7 if festival else 1.0)
        sales = max(0.0, mean * (1 + 0.08 * torch.randn(1, generator=g).item()))
        rows.append([round(sales), dow, festival, rain])
        dates.append(d)
    return dates, torch.tensor(rows, dtype=torch.float32)

Before any model, Priya asked for two numbers. “What would a lazy forecaster score? If your model can’t beat lazy, we’re done.”

A baseline is the simplest reasonable forecast, the number any model must beat to be worth running. Time series have two classics:

• Naive: tomorrow will be like today. Predict yesterday’s sales.
• Seasonal naive: tomorrow will be like the same day last week. That is the store manager’s method, minus the “bit if it feels busy”.

The score is mean absolute error, or MAE (the average size of the miss, in croissants, ignoring whether it was over or under). On the 304 days from January to October 2026:

Forecast	MAE (croissants per day)
Naive (yesterday)	38.9
Seasonal naive (same day last week)	23.6

Naive is terrible, because it predicts Monday’s quiet numbers for Saturday’s rush. Seasonal naive is much better, because the weekly rhythm dominates. That 23.6 is the manager’s number. Every model in this lesson has to beat it.

Arjun’s first forecaster

Arjun reused last lesson’s idea. Take every run of 28 consecutive days as the input, and the 29th day as the target. That gives about a thousand training examples from three years of data. Each of these is a sliding window (a fixed-length slice of history that moves forward one day at a time, each paired with the value that came next).

He standardised the sales, shuffled all thousand windows, put 80% into training and 20% into testing, and trained an LSTM on the sales numbers alone.

Random-split test MAE: 16.0 croissants

Sixteen. A third better than the manager. He had the Slack message to Meera typed out when Priya asked a question.

“Which days are in your test set?”

“A random 20% of them.”

“From when?”

He checked. Test days were scattered across all three years. There was a test day in March 2024, surrounded by training windows from February and April 2024. There was a training window ending on 14 March 2026 whose last input day, 13 March, was the target of a test window. The model had seen the answers to some of its exam questions as part of other questions.

“And your standardisation?”

“Mean and standard deviation of all the sales.”

“Including 2026, which is busier than 2024 because the store is growing. So the training data was quietly told how high sales would get in the future.”

This is data leakage (information from the test period, or from the future, sneaking into training). In ordinary datasets it is a subtle bug. In time series it is the default, because shuffling is the default.

You can only use what you’d know at 5am

“Think of a weather forecaster,” Priya said. “To test them honestly, you give them the data up to last Tuesday and ask about Wednesday. You never let them peek at Thursday. And you’d never test them on a random day from 2019 after they’d studied 2018 and 2020.”

The rule for time series is strict and simple: train on the past, validate on the more recent past, test on the most recent period. Everything the model or your preprocessing learns must come from the training period alone. Arjun’s new split:

Split	Period	Target days
Training	Jan 2024 to Jun 2025	519
Validation	Jul 2025 to Dec 2025	184
Test	Jan 2026 to Oct 2026	304

The mean and standard deviation now come from the training period only (150.5 and 44.0 croissants). Same model, same data, honest split:

Time-split test MAE: 19.3 croissants

Still better than the manager’s 23.6, but a lot less impressive than 16.0. About three croissants a day of the original “improvement” was the model peeking. That is a big chunk of a 7-croissant win.

What the model needs to know about tomorrow

The sales-only model knew the shape of the last 28 days. It did not know that tomorrow is a Saturday, that it is Diwali, or that the forecast says rain. The store manager knows all three at 5am.

Priya split features into two kinds:

• History features, one row per past day in the window: that day’s sales (standardised), its day of week, whether it was a festival, whether it rained.
• Known-in-advance features for the target day: its day of week, whether it is a festival, and the rain forecast. These are things genuinely known before the target day starts.

Day of week needs a trick. If you feed it as a number from 0 to 6, the model thinks Sunday (6) and Monday (0) are as far apart as possible, when they are neighbours. So each day becomes a point on a circle, using two numbers: $\sin (2\pi \cdot d/7)$ and $\cos (2\pi \cdot d/7)$. Sunday and Monday end up close together, as they should.

The model is the LSTM from last lesson reading the 28-day window, with one change: after it finishes reading, its final hidden state is joined with tomorrow’s known features, and a small network turns that into a forecast.

flowchart LR
    A["28 days of history (sales, weekday, festival, rain)"] --> B["LSTM reads day by day"]
    B --> C["Final hidden state: 32 numbers"]
    D["Tomorrow: weekday, festival, rain forecast"] --> E["Join"]
    C --> E
    E --> F["Small MLP"]
    F --> G["Tomorrow's sales"]

One forecast, traced from start to finish

Take a single forecast: Thursday 9 July 2026, deep in the monsoon.

The window. The 28 days from 11 June to 8 July. Here are the last seven, as raw values and as the model sees them (sales standardised as $(x-150.5)/44.0$, weekday as sine and cosine, festival, rain):

Day	Sales	Rain	Model input
Thu 2 Jul	164	no	[0.31, 0.43, -0.90, 0, 0]
Fri 3 Jul	197	no	[1.06, -0.43, -0.90, 0, 0]
Sat 4 Jul	171	yes	[0.46, -0.97, -0.22, 0, 1]
Sun 5 Jul	155	yes	[0.10, -0.78, 0.62, 0, 1]
Mon 6 Jul	153	no	[0.06, 0.00, 1.00, 0, 0]
Tue 7 Jul	93	yes	[-1.31, 0.78, 0.62, 0, 1]
Wed 8 Jul	146	no	[-0.10, 0.97, -0.22, 0, 0]

Notice how rain drags sales down: a rainy Saturday sold less than a dry Friday, and rainy Tuesday was the worst day of the week.

The known features for Thursday. Weekday 3 becomes $[\sin ,\cos ]=[0.434,-0.901]$, no festival, and the forecast says rain: [0.434, -0.901, 0, 1].

The LSTM. It reads the 28 rows in order, updating its 32-number hidden state each day, exactly as the proofing model read minutes. The final state summarises “the recent level, the weekly pattern, and how rain has been biting”.

The head. The 32 numbers and the 4 known features go through the small network, which outputs $-0.733$ in standardised units. Converting back: $-0.733\times 44.0+150.5\approx 118$ croissants.

What happened.

Forecast	Prediction	Actual	Miss
Seasonal naive (last Thursday, which was dry)	164	121	43
LSTM with known features	118	121	3

The manager’s method copied a dry Thursday onto a wet one. The model knew it would rain. Change only the rain flag to “dry” and the same model predicts 152, so it has learned that rain costs this store about 34 croissants on a July Thursday.

The real code

The window builder turns the series into tensors. day_features and known_features produce the two kinds of features; build stacks them for a list of target days.

# windows.py
import math
from datetime import date
import torch
from sales import make_sales

WINDOW = 28
dates, data = make_sales()
sales = data[:, 0]

def day_features(i: int, mean: float, std: float) -> list[float]:
    dow = data[i, 1].item()
    return [(sales[i].item() - mean) / std,
            math.sin(2 * math.pi * dow / 7), math.cos(2 * math.pi * dow / 7),
            data[i, 2].item(), data[i, 3].item()]

def known_features(i: int) -> list[float]:
    dow = data[i, 1].item()
    return [math.sin(2 * math.pi * dow / 7), math.cos(2 * math.pi * dow / 7),
            data[i, 2].item(), data[i, 3].item()]

def build(targets: list[int], mean: float, std: float):
    X = torch.tensor([[day_features(j, mean, std) for j in range(i - WINDOW, i)] for i in targets])
    K = torch.tensor([known_features(i) for i in targets])
    y = torch.tensor([[(sales[i].item() - mean) / std] for i in targets])
    return X, K, y            # (n, 28, 5), (n, 4), (n, 1)

# Split by TIME, by the date of the day being predicted
targets = range(WINDOW, len(dates))
train_idx = [i for i in targets if dates[i] <= date(2025, 6, 30)]
val_idx = [i for i in targets if date(2025, 7, 1) <= dates[i] <= date(2025, 12, 31)]
test_idx = [i for i in targets if dates[i] >= date(2026, 1, 1)]

mean, std = sales[train_idx].mean().item(), sales[train_idx].std().item()   # training period only
X_train, K_train, y_train = build(train_idx, mean, std)
X_val, K_val, y_val = build(val_idx, mean, std)
X_test, K_test, y_test = build(test_idx, mean, std)

The model takes two inputs, which is a preview of next lesson:

# model.py
import torch
from torch import nn

class Forecaster(nn.Module):
    def __init__(self, n_features: int = 5, n_known: int = 4, hidden: int = 32):
        super().__init__()
        self.lstm = nn.LSTM(n_features, hidden, batch_first=True)
        self.head = nn.Sequential(nn.Linear(hidden + n_known, 32), nn.ReLU(), nn.Linear(32, 1))

    def forward(self, window, known):          # window: (batch, 28, 5), known: (batch, 4)
        _, (h, _) = self.lstm(window)          # h: (layers, batch, hidden)
        summary = h[-1]                        # last layer's final state: (batch, hidden)
        return self.head(torch.cat([summary, known], dim=1))   # (batch, 1)

Training uses nn.L1Loss, which is mean absolute error, so the model optimises exactly the number you report. It keeps the weights from the epoch with the best validation MAE:

import torch
from torch import nn
from windows import X_train, K_train, y_train, X_val, K_val, y_val, X_test, K_test, y_test, mean, std
from model import Forecaster

torch.manual_seed(0)
model = Forecaster()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
loss_fn = nn.L1Loss()
best_val, best_state = float("inf"), None

for epoch in range(150):
    model.train()
    perm = torch.randperm(len(X_train))
    for k in range(0, len(X_train), 32):
        idx = perm[k:k + 32]
        loss = loss_fn(model(X_train[idx], K_train[idx]), y_train[idx])
        optimizer.zero_grad()
        loss.backward()
        nn.utils.clip_grad_norm_(model.parameters(), 1.0)
        optimizer.step()
    model.eval()
    with torch.no_grad():
        val = loss_fn(model(X_val, K_val), y_val).item()
    if val < best_val:
        best_val = val
        best_state = {k: t.clone() for k, t in model.state_dict().items()}

model.load_state_dict(best_state)
with torch.no_grad():
    test_mae = (model(X_test, K_test) - y_test).abs().mean().item() * std   # back to croissants
print(f"test MAE {test_mae:.1f} croissants")

Shuffling inside the training set is fine: each window still only contains data from before its own target, and every target is in the training period. What must never be shuffled is the boundary between training, validation and test.

It takes about half a minute on a laptop CPU. The results on the 2026 test period:

Forecast	Test MAE	Rainy days (96)	Dry days (208)
Seasonal naive (manager)	23.6	25.1	23.0
LSTM, sales only	19.3
LSTM with known features	12.6	9.1	14.2

Almost half the manager’s error. The biggest win is on rainy days, where copying last week is at its worst.

Where forecasting breaks

Arjun put the model in shadow mode for October. Three things went wrong, and they are the three things that go wrong with almost every forecaster.

It was consistently low. Priya had him compute the average signed error (prediction minus actual) by month. In January to March 2026 the model under-forecast by about 10 croissants a day; in October, by about 10 again. The store had grown since the training period ended in mid-2025: average sales were 150.5 in training and 167.7 in the test period. The model had learned “normal” from a smaller store. This is drift, and forecasts are especially exposed to it because the future is, by definition, outside the training range.

The fix is to stop asking the model for absolute numbers. Divide every value in a window by that window’s own average level, and ask the model to predict tomorrow relative to the recent level (“8% above the last four weeks”). Then multiply back. The model learns shapes, and the recent average carries the growth. In Arjun’s test, this cut the average bias from about −4 to about +1 croissant a day and the MAE from 12.6 to 12.2.

Festivals were a guess. Diwali happens once a year. With a year and a half of training data, the model had seen exactly one Diwali. No model learns a pattern from one example. For rare events, use rules and people: a multiplier the store manager can set (“Diwali: ×1.7”), applied on top of the model’s forecast, and a big obvious override button. Pretending the model knows is worse than admitting it does not.

Forecasting further ahead got worse fast. Meera wanted a week-ahead forecast for ordering butter. The simple approach is recursive: predict tomorrow, feed that prediction back in as if it were real, predict the day after, and so on. Errors compound, because day 7 is built on six guesses. The better approach is to train the model to output all seven days at once (a final layer with 7 outputs), so it never feeds on its own mistakes.

Two more honest points from Priya:

• The best model is often not an RNN. For daily sales with good calendar features, a gradient-boosted tree model (such as LightGBM) or even linear regression on the same features often matches an LSTM, trains in seconds, and is easier to explain. Try them. Use the LSTM when the sequence shape genuinely matters, or when you have many related series (all six stores, hundreds of products) to learn from together.
• The average is not the right number to bake. A leftover croissant costs ₹38; a missed sale costs ₹82 of margin plus goodwill. When missing is twice as costly as wasting, you should bake above the average forecast. Models can be trained to predict a higher percentile directly (with a quantile loss), or you can add a safety margin based on the model’s typical error. That is a business decision. Ask Meera, not the loss function.

Try it yourself

1. Compute the seasonal naive MAE separately for rainy and dry days in the test period, and check it against the table.
2. Remove the rain flag from the known features (set it to 0 for every target day) and retrain. How much does the test MAE rise, and on which days?
3. Implement the relative-to-recent-level version: in build, divide each window’s sales by the window’s mean, predict tomorrow divided by that same mean, and multiply back when evaluating. Compare MAE and average signed error by month.

Show the answer

1. About 25.1 on rainy days and 23.0 on dry days. Seasonal naive is worst exactly when the weather changes from last week.
2. In Arjun’s run the test MAE rose from 12.6 to 19.4. Rainy days got much worse (9.1 to 22.7, nearly as bad as the manager’s method), and dry days got somewhat worse too (14.2 to 17.9), because without the flag the model cannot tell whether a dip in the window was weather or a genuine slowdown.
3. You should see the monthly bias shrink towards zero, especially in early 2026 and October, and a small MAE improvement (in Arjun’s run, 12.6 to 12.2). The bigger the drift in the data, the bigger the gain.

Common mistakes

• Shuffling before splitting. It leaks the future into training. Split by date first; shuffle only within the training set.
• Standardising with all the data. It tells the model how big the future gets. Compute statistics on the training period.
• Skipping baselines. “MAE 12.6” means nothing alone. “12.6 versus the manager’s 23.6” is a decision.
• Using features you will not have at prediction time. Actual weather, tomorrow’s footfall, end-of-day totals. If it is not known at 5am, it cannot be an input.
• Trusting the model on rare events. One Diwali is not a pattern. Use rules and human overrides.

Recap

• Time series forecasting turns history into sliding windows, each paired with the value that came next.
• Split strictly by time; compute normalisation statistics on the training period only; otherwise you leak the future and overstate accuracy.
• Always compare against naive and seasonal naive baselines, measured with an interpretable metric such as MAE.
• Feed the model what is genuinely known in advance (weekday as sine and cosine, festivals, the weather forecast), not just past values.
• Expect drift, rare events and compounding multi-step errors; predicting relative to recent levels, rules for festivals and direct multi-step outputs are the standard fixes.

At the end of October, Arjun showed Meera the shadow-mode results. “The model’s average miss is about 12 croissants a day; the old way misses about 24. It’s best on rainy days, because it reads the forecast and last week’s method doesn’t. It can’t predict Diwali, so the manager sets a festival multiplier. And because a missing croissant costs us twice what a leftover one does, we bake a little above the forecast on purpose.”

“Twelve a day, six stores, every day,” Meera said. “That’s real money.”

Then she slid a second sheet across. “While you’re at it, can the same model tell us which pastries will sell out, not just how many croissants? And use the photos of the display case the staff take every morning?”

Arjun opened his mouth to say “that’s three different models.” Priya got there first. “Or one model with two inputs and two outputs. Tomorrow.”

Go deeper

• PyTorch tutorials for nn.LSTM and building custom datasets for sequences.
• scikit-learn user guide for TimeSeriesSplit, a ready-made way to do several honest time-based splits (“walk-forward validation”), and for quick regression baselines.