AutoModels, AutoTokenizers and AutoClasses

Meera had two new requests, and she wrote them on the whiteboard herself.

1. “Show me how SURE the email sorter is. If it’s unsure, send the email to a person.”
2. “Rate our Google reviews 1 to 5 stars, so I can see which store is slipping.”

Arjun nodded confidently. The pipeline function had served him well. But a pipeline gave him one label and one score, and he wanted every score for every label, plus the model’s inner numbers so he could later spot duplicate complaints. Time to go one level lower.

He had seen AutoModel in a tutorial. It sounded like the general-purpose version of everything. He wrote:

from transformers import AutoTokenizer, AutoModel

ckpt = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(ckpt)
model = AutoModel.from_pretrained(ckpt)

out = model(**tok("The croissants were stale.", return_tensors="pt"))
print(out.logits)

The terminal printed a load report listing four weights as UNEXPECTED (classifier.weight, classifier.bias and two more), then crashed:

AttributeError: 'BaseModelOutput' object has no attribute 'logits'

Inside the vending machine

Priya drew a vending machine on the whiteboard. Coin slot on one side, can dropping out the other.

“This is pipeline. Coin in, drink out. Lovely, until it eats your coin at two in the morning. Then you need to know what’s inside.” She drew three boxes inside the machine: tokenizer, model, post-processing.

flowchart LR
  A["Text"] --> B["Tokenizer: text to token IDs"]
  B --> C["Model: IDs to raw scores"]
  C --> D["Post-process: scores to labels"]
  D --> E["Answer"]

“And your crash is about the middle box,” she said. “Let me draw it bigger.”

She drew a long oven line, like the one in Crumb’s central kitchen. Dough goes in one end. Mixing, proving, baking. At the very end, one last station: decorating.

“Same oven line every day. What decides whether a birthday cake or a loaf of bread comes out?”

“The last station,” Arjun said. “Decorating, or slicing, or boxing.”

“Right. A model is the same. Most of it is a shared body that turns tokens into rich internal numbers. At the end sits a small head, a final layer that turns those numbers into the output one particular task needs: two sentiment scores, five star ratings, the start and end of an answer. The checkpoint you loaded has a sentiment head. But you asked for AutoModel…”

“…which is just the body,” Arjun finished. “So it threw the head away. That’s what UNEXPECTED meant: weights in the file that the class I asked for had no place for.”

“And no head means no logits.”

Following one sentence through the machine

Priya made him do it properly, one box at a time, with a single review: “The croissants were stale.”

Box 1: the tokenizer

A tokenizer splits text into tokens (small pieces, often parts of words) and maps each one to a number in the model’s vocabulary. A model never sees letters, only these numbers.

from transformers import AutoTokenizer

CKPT = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(CKPT)

enc = tokenizer("The croissants were stale.")
print(enc["input_ids"])
print(tokenizer.convert_ids_to_tokens(enc["input_ids"]))

[101, 1996, 13675, 10054, 22341, 2015, 2020, 26729, 1012, 102]
['[CLS]', 'the', 'cr', '##ois', '##sant', '##s', 'were', 'stale', '.', '[SEP]']

Ten tokens for five words. Read them carefully:

• [CLS] (ID 101) and [SEP] (ID 102) are special tokens. This model was trained with [CLS] at the start of every input and [SEP] at the end, so the tokenizer adds them for you. Its head reads its decision from the [CLS] position.
• The text was lower-cased: The became the. This is an uncased model.
• “croissants” is rare in the English text this vocabulary was built from, so it was split into four pieces: cr, ##ois, ##sant, ##s. The ## means “glued to the previous piece”.

For comparison, “Best brownies in Pune!” became [CLS] best brown ##ies in pune ! [SEP]. Pune got a single token of its own.

Box 2: the model

To run more than one sentence at a time, the tokenizer must make all rows the same length. It does that with padding: filler tokens added to shorter rows. A second list, the attention mask, marks which tokens are real (1) and which are padding (0), so the model ignores the filler.

import torch
from transformers import AutoModelForSequenceClassification

batch = tokenizer(
    ["The croissants were stale.", "Best brownies in Pune!"],
    padding=True,          # pad the shorter row
    truncation=True,       # cut anything longer than the model's limit
    return_tensors="pt",   # return PyTorch tensors (grids of numbers)
)
print(batch["attention_mask"])

model = AutoModelForSequenceClassification.from_pretrained(CKPT)
with torch.no_grad():                 # we are not training: skip the learning bookkeeping
    outputs = model(**batch)          # ** passes input_ids and attention_mask by name
print(outputs.logits)

tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
        [1, 1, 1, 1, 1, 1, 1, 1, 0, 0]])
tensor([[ 4.7167, -3.8317],
        [-4.0440,  4.4721]])

The brownies sentence had 8 tokens, so it got two padding tokens to reach 10. A tensor is PyTorch’s name for a grid of numbers.

The model returns logits: raw scores, one per label, not yet turned into probabilities. For the croissant review, label 0 scored 4.72 and label 1 scored −3.83. Bigger is more likely. But these are not percentages.

Box 3: post-processing

To turn logits into probabilities, we use softmax: raise e (about 2.718) to the power of each score, then divide each result by their total, so everything sums to 1. Do it by hand once:

• e^4.7167 ≈ 111.8 and e^−3.8317 ≈ 0.0217.
• Total ≈ 111.82.
• Probability of label 0 ≈ 111.8 ÷ 111.82 ≈ 0.99981.

Then we need the name of label 0. That lives in the model’s configuration:

probs = outputs.logits.softmax(dim=-1)
for row in probs:
    best = int(row.argmax())
    print(model.config.id2label[best], round(float(row[best]), 5))

NEGATIVE 0.99981
POSITIVE 0.9998

You have rebuilt the pipeline by hand, and you can now see every number it used to hide.

How “Auto” knows what to load

There are hundreds of architectures in transformers: BERT, DistilBERT, Gemma, Llama, Whisper, ViT and many more. Each has its own Python class, such as DistilBertForSequenceClassification. You do not want to memorise them.

A checkpoint is a saved, trained model: a Hub repo name or a local folder holding weights, a config.json and tokenizer files. The Auto classes work like a receptionist. They open the checkpoint’s config.json and read it. Here are the key lines from this one:

{
  "architectures": ["DistilBertForSequenceClassification"],
  "model_type": "distilbert",
  "dim": 768,
  "n_layers": 6,
  "vocab_size": 30522,
  "max_position_embeddings": 512,
  "id2label": { "0": "NEGATIVE", "1": "POSITIVE" }
}

model_type tells AutoTokenizer and AutoModel... which family to build. id2label names the outputs. max_position_embeddings is the longest input the model accepts, in tokens: 512. Change the checkpoint string, and the same three lines of code load a completely different architecture.

The Auto family has more members than models:

You will use the last three in Module 12.

Heads: same body, different jobs

The suffix on AutoModelFor... chooses the head:

So what did Arjun’s original AutoModel call give him? Not nothing:

from transformers import AutoModel

body = AutoModel.from_pretrained(CKPT)
with torch.no_grad():
    hidden = body(**batch).last_hidden_state
print(hidden.shape)

torch.Size([2, 10, 768])

There is a mirror-image trap. If you load a checkpoint that has no trained head into a class that needs one (say, plain distilbert/distilbert-base-uncased into AutoModelForSequenceClassification), transformers warns that some weights are missing or newly initialized. That warning means the head is random. Its answers are noise until you fine-tune it (Module 33). Never ignore it.

The real code: the confidence router

With every score in hand, Meera’s first request takes a dozen lines:

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

CKPT = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(CKPT)
model = AutoModelForSequenceClassification.from_pretrained(CKPT)

def score(texts: list[str], threshold: float = 0.90):
    batch = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
    with torch.no_grad():
        probs = model(**batch).logits.softmax(dim=-1)
    for text, row in zip(texts, probs):
        best = int(row.argmax())
        confidence = float(row[best])
        route = "auto" if confidence >= threshold else "human"
        yield model.config.id2label[best], round(confidence, 3), route, text

for result in score(["The croissants were stale.", "Ordered the eggless brownies."]):
    print(result)

('NEGATIVE', 1.0, 'auto', 'The croissants were stale.')
('NEGATIVE', 0.997, 'auto', 'Ordered the eggless brownies.')

Look at the second line. “Ordered the eggless brownies.” is neutral, yet the model says NEGATIVE at 99.7% and the router trusts it.

For Meera’s second request, swap one string. The checkpoint nlptown/bert-base-multilingual-uncased-sentiment has a head with five labels (1 star to 5 stars), and because Auto reads the config, every other line of the code stays the same. That is the whole point of Auto.

The same three steps for a chat model

Generation follows the same shape, with generate() running the predict-one-token loop you traced in the last lesson:

from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_ID = "google/gemma-3-1b-it"   # gated: accept the licence on the Hub first
tok = AutoTokenizer.from_pretrained(MODEL_ID)
lm = AutoModelForCausalLM.from_pretrained(MODEL_ID, device_map="auto")

messages = [{"role": "user", "content": "Name three Diwali sweets in one line."}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to(lm.device)  # template has <bos> already

output_ids = lm.generate(**inputs, max_new_tokens=60)
new_tokens = output_ids[0][inputs["input_ids"].shape[1]:]   # slice off the prompt
print(tok.decode(new_tokens, skip_special_tokens=True))

Box 1 (tok), box 2 (lm.generate), box 3 (tok.decode). Note the slice: generate() returns the prompt plus the new tokens, so you cut the prompt off before decoding. .to(lm.device) moves the inputs to wherever the model lives (GPU, Apple Silicon or processor); inputs and model must be on the same device.

Library APIs change quickly. If a name has moved, check the docs linked below.

Where it breaks: the 2 a.m. crash

Two weeks later the overnight sorting job died at 2:14 a.m. The culprit was one email: a 900-word complaint from a corporate client about a Diwali order of 200 hampers. Arjun’s early version had no truncation=True, so the tokenizer produced 1,100 tokens, more than the model’s 512-token limit, and the model crashed.

He added truncation=True. The crash stopped. The next morning, Priya pointed at that same email in the dashboard: tagged “late delivery”, routed “auto”. She scrolled to the bottom. The last paragraph said: “Please refund the full amount of ₹1,84,000.” Truncation had cut the email at token 512. The model never saw the most important sentence.

“Truncation turns a loud failure into a silent one,” Priya said. “Silent is worse.”

The fix: for long texts, split them into overlapping windows, score each window, and keep the strongest signal; or score the first and last 250 tokens separately. And log every email that was truncated, so a person can spot-check them.

Other things that bite in production:

• Mixed-up tokenizer and model. Load both from the same checkpoint name. A tokenizer from a different family produces IDs that mean different words to the model: you get an index error if you are lucky and confident nonsense if you are not.
• Wrong head. UNEXPECTED weights mean you dropped a head; missing or newly initialized weights mean you got a random one.
• Logits treated as probabilities. Apply softmax first, and remember softmax confidence is not accuracy.
• Forgetting torch.no_grad(). Without it, PyTorch stores extra data for training, using much more memory. Models loaded with from_pretrained start in evaluation mode already, but no_grad is still on you.
• Decoding the prompt. With generate(), slice off the input tokens.

Try it yourself

Rate three reviews with nlptown/bert-base-multilingual-uncased-sentiment, by hand, with no pipeline:

reviews = [
    "Absolutely loved the plum cake.",
    "It was okay, a bit dry.",
    "Terrible. Arrived late and broken.",
]

First print model.config.id2label. Then tokenize, run the model, softmax, and print the label and confidence for each review. Finally, try a review in Hindi or Marathi.

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

ckpt = "nlptown/bert-base-multilingual-uncased-sentiment"
tok = AutoTokenizer.from_pretrained(ckpt)
model = AutoModelForSequenceClassification.from_pretrained(ckpt)
print(model.config.id2label)   # {0: '1 star', 1: '2 stars', 2: '3 stars', 3: '4 stars', 4: '5 stars'}

batch = tok(reviews, padding=True, truncation=True, return_tensors="pt")
with torch.no_grad():
    probs = model(**batch).logits.softmax(dim=-1)
for text, row in zip(reviews, probs):
    best = int(row.argmax())
    print(model.config.id2label[best], round(float(row[best]), 2), "|", text)

Common mistakes

• Using AutoModel and expecting labels. Pick the AutoModelFor... class whose head matches the task.
• Ignoring load warnings. They tell you when a head was dropped or randomly created.
• Truncating silently. Long inputs lose their ending. Detect and handle long texts on purpose.
• Trusting softmax as certainty. Measure accuracy against your threshold on labelled data.
• Loading the tokenizer from somewhere else. Same checkpoint name for both, every time.

Recap

• A pipeline is three boxes: tokenizer (text to IDs), model (IDs to logits), post-processing (logits to labels).
• Tokenizers add special tokens, split rare words, pad batches and truncate long inputs.
• Auto classes read config.json and build the right architecture from a checkpoint name.
• The For... suffix chooses the head; the head decides what comes out. AutoModel alone gives hidden states, not labels.
• Softmax turns logits into probabilities, id2label names them, and neither tells you how often the model is actually right.

At the end of the week Arjun showed Meera the new dashboard: a confidence column, a “needs a human” queue, and star ratings for every store. “Every model is three steps,” he explained. “Chop the text into numbered pieces, run the numbers through the model, turn its scores into labels. The Auto tools read a small settings file and pick the right parts for us. Now when something breaks at night, I know which of the three boxes to open.”

Meera nodded, then dropped a thick binder on his desk with a thud: the 200-page store operations handbook. “New staff phone me every week with questions that are answered in here. Fridge temperatures. Who approves big refunds. Can your models read this?”

Go deeper

• Hugging Face Transformers documentation: https://huggingface.co/docs
• Hugging Face LLM course (using Transformers, tokenizers): https://huggingface.co/learn/llm-course
• PyTorch tutorials, if tensors feel mysterious: https://pytorch.org/tutorials/