The Hub, datasets, inference providers and CLI

Arjun was proud of his email sorter. Every message in Crumb’s support inbox went to a paid AI API with a long prompt: thirty example emails, then the new one, then “Reply with one label: late delivery, allergy question or refund request.” It was accurate. It was also expensive.

Meera put the month’s invoice on his desk. Forty thousand emails. Each request carried about 2,000 tokens of examples plus a 350-token email, so roughly 95 million input tokens a month. At the price he was paying, that came to about $395, or ₹35,000.

“Thirty-five thousand rupees,” she said, “to sort emails into three piles. And every customer’s name, address and complaint went to a server in another country. Is there a cheaper way? A more private way?”

Priya opened a new browser tab. “Before you write more code, look here. Somebody has very likely trained a model for this already and published it for free.”

The tab said huggingface.co.

Arjun goes shopping without a list

Arjun typed “email classification” into the Hub’s search box. Hundreds of results. He picked one with a promising name, clicked through to its files, and downloaded the whole repository.

Forty minutes later his laptop fan was screaming and his disk had lost 14 GB. The model loaded, then crashed with an out-of-memory error. When he finally scrolled down the model’s page, he found one more line: license: cc-by-nc-4.0. NC means non-commercial. Crumb is a business.

Priya looked at the wreckage. “You bought a delivery truck to carry one sandwich, and the truck isn’t allowed on commercial roads. Let’s slow down. When you borrow a cookbook for a professional kitchen, what do you check before you take it home?”

Arjun thought about it. “That it’s about the cuisine I actually cook. That I’m allowed to use the recipes in a shop. That it’s not some ancient edition. And that it fits on the shelf.”

“Good. The Hub has all four answers on every model’s page. You just skipped them.”

The library in plain words

Think of the Pune public library crossed with GitHub. Anyone can browse millions of items, borrow any of them, and take them home. Authors publish new editions, and the library keeps every old edition too.

On the Hub, the items are:

• Models. A model is a program that has learned a skill from examples, such as sorting emails or transcribing speech. The learned part is a huge list of numbers called parameters (or weights).
• Datasets. A dataset is a collection of examples, used to train models or to test them.
• Spaces. Small demo apps, usually with a web page, so you can try a model in the browser before downloading anything.

Each lives in a repository (repo): a folder with a full version history, exactly like a GitHub repo. Every repo has a name in the form owner/name, such as google/gemma-3-1b-it or facebook/bart-large-mnli. That string is what you pass to code.

Where the analogy breaks: a library book can be read by anyone for any purpose. A model comes with a licence, a set of legal rules about how you may use it. And some “books” weigh hundreds of gigabytes.

Reading the label before you borrow

Every model page starts with a model card: a README that works like the ingredients label on a box of Crumb’s cookies. Here is the checklist Priya made Arjun write on a sticky note:

Look for	Where on the page	Why it matters
Task	Tag under the name, e.g. “Zero-Shot Classification”	Tells you what it does and which code to use
Licence	Right-hand panel	Whether Crumb may use it in a paid product
Size	Parameter count; “Files and versions” tab	Whether it fits on your machine
Languages	Tags and card text	An English-only model will stumble on Hinglish
Training data and limits	Card text	Where it is likely to be wrong or biased
Downloads, likes, last update	Top of the page	A rough sign that people use and maintain it
Inference Providers	Right-hand panel	Whether you can call it without downloading

Downloads tell you a model is popular, not that it is good for your data. Always test it on twenty or thirty of your own real examples before you trust it.

Following one email through the Hub

Priya picked one real email from the inbox, and they followed it all the way through.

“My order #1042 hasn’t arrived and it’s been 3 hours. This was for my son’s birthday.”

Step 1: find a candidate. On the Hub, Arjun clicked Models, filtered by the task Zero-Shot Classification, and sorted by Most downloads. Near the top: facebook/bart-large-mnli.

Zero-shot classification means sorting text into labels the model was never specifically trained on. You hand it the labels at the moment you use it. This model was trained on a different task, natural language inference: deciding whether one sentence implies another. To classify the email, it asks itself three questions: does this email imply “This text is about late delivery”? “…about an allergy question”? “…about a refund request”? The label with the strongest “yes” wins. That trick is why it can sort into Crumb’s labels without ever having seen a Crumb email.

Step 2: read the label. The card and the right-hand panel said:

• Licence: MIT. Commercial use allowed.
• Size: 407 million parameters. The weights file, model.safetensors, is 1.63 GB.
• Language: English.
• Downloads: millions a month. Widely used, well understood.

Step 3: download only what you need. The “Files and versions” tab told a surprising story. The repo held the same model four times: model.safetensors (1.63 GB), pytorch_model.bin (1.63 GB), flax_model.msgpack (1.63 GB) and rust_model.ot (2.04 GB). Downloading everything would cost 6.9 GB. Downloading what the Python transformers library needs costs about 1.63 GB.

Show the answer

Each file is the same weights saved in a different format for a different framework: PyTorch’s older .bin format, TensorFlow’s .h5, Flax’s .msgpack, Rust’s .ot, sometimes ONNX for other runtimes. You need one. Prefer .safetensors: it loads fast, and unlike .bin (which uses Python’s pickle format) it cannot run hidden code when it is opened. That is a real security difference, not a detail.

Step 4: run it. On Arjun’s laptop, the email came back with these scores:

Label	Score
late delivery	0.986
refund request	0.010
allergy question	0.004

Right answer, very confident. Each email took about 0.28 seconds on the laptop’s processor. Forty thousand emails is about 11,200 seconds, a little over three hours of computing a month, which can run overnight. Cost per email: effectively zero. Data leaving the building: none.

Show the answer

1,630,000,000 ÷ 407,000,000 ≈ 4 bytes per parameter. Four bytes is a 32-bit floating-point number, the standard “full precision” format. Many newer models are stored in 16-bit (2 bytes per parameter), which halves the file. You will use this rough sum, parameters × bytes per parameter, every time you ask “will this fit on my machine?”.

Opening the hood: repos, revisions and the cache

Three mechanisms explain almost every surprise you will meet on the Hub.

A repo is a Git repository with big files. Every change the author makes is a commit with a unique ID (a long hash such as d7645e127eaf..., often shown shortened). Large files are stored in a special way so Git does not choke on them, but the history is real. That means you can ask for an exact version. If the author pushes new weights tomorrow, code that asks for “the latest” gets new behaviour. Code that pins a revision does not.

Downloads go into a cache. The first time you load a model, its files are saved to a folder on your machine, by default ~/.cache/huggingface/hub. Inside, each repo gets a folder like models--facebook--bart-large-mnli, containing:

• blobs/: the actual files, named by their content hash.
• snapshots/<commit>/: a view of the repo at one commit, pointing at the blobs.
• refs/main: which commit “main” meant when you last downloaded.

The second load comes straight from disk. Two versions of a model that share a file store it only once. You can move the whole cache by setting the HF_HOME environment variable, for example to a bigger external drive.

Some actions need an identity. Private repos, gated models (which make you accept a licence first, next lesson) and paid inference all need an access token: a long secret string, beginning hf_, that works like a library card. Create one under Settings → Access Tokens on huggingface.co. A “Read” token is enough for downloading. A “fine-grained” token lets you limit exactly which repos and actions it can touch, which is safer in production.

Building it, one command at a time

Install and log in. If you use plain pip, pip install huggingface_hub installs the library and its command-line tool (CLI, a program you drive by typing commands). With uv:

uv add huggingface_hub transformers torch datasets python-dotenv
uv run hf auth login     # paste your token once; it is saved on this machine
uv run hf auth whoami    # prints your username if login worked

Older tutorials use huggingface-cli login and huggingface-cli download. That was the previous name of the same tool; current versions of huggingface_hub ship only hf. If a tutorial’s command fails, try the hf version.

For scripts and servers, put the token in your project’s .env file as HF_TOKEN=hf_.... Hugging Face libraries read HF_TOKEN from the environment automatically.

Download only what you need. The --include patterns pick files by name:

uv run hf download facebook/bart-large-mnli --include "*.json" "*.safetensors" "*.txt"

You should see five files land in the cache: config.json (1 KB), model.safetensors (1.63 GB), and three small tokenizer files. Not 6.9 GB.

Run it, pinned to a version.

from transformers import pipeline

LABELS = ["late delivery", "allergy question", "refund request"]

classify = pipeline(
    "zero-shot-classification",
    model="facebook/bart-large-mnli",
    revision="d7645e127eaf1aefc7862fd59a17a5aa8558b8ce",  # the exact commit you tested
)

email = "My order #1042 hasn't arrived and it's been 3 hours. This was for my son's birthday."
result = classify(email, candidate_labels=LABELS)
print(result["labels"][0], round(result["scores"][0], 3))
# late delivery 0.986

The pipeline function is the simplest way to run a Hub model: name a task, name a model, pass inputs. Two lessons from now you will open it up and see the three steps it hides.

Test it on your own data, with datasets. The datasets library loads datasets from the Hub, and also from your own files. Arjun exported 120 emails that staff had already labelled into crumb_emails.csv (columns text and label):

from datasets import load_dataset

emails = load_dataset("csv", data_files="crumb_emails.csv", split="train")
print(emails)   # Dataset({features: ['text', 'label'], num_rows: 120})

correct = 0
for row in emails:
    guess = classify(row["text"], candidate_labels=LABELS)["labels"][0]
    correct += guess == row["label"]
print(f"Accuracy: {correct / len(emails):.2f}")

Arjun’s run printed Accuracy: 0.88. Not perfect, but now he knew the number, instead of guessing.

From the Hub, the same function loads public datasets: load_dataset("stanfordnlp/imdb", split="train") gives 25,000 labelled movie reviews. A split is a named slice, usually train (for learning) and test (for checking). Under the hood, datasets are stored in a column-based format and memory-mapped: read from disk as needed, so a 10 GB dataset does not need 10 GB of memory. For truly huge ones, pass streaming=True and rows arrive one at a time. Datasets have licences too.

When your laptop is not enough: Inference Providers

Running a model yourself is like cooking at home. You own the stove, nobody sees your recipe, and each extra plate costs almost nothing. But you need the stove. A 400-million-parameter model runs fine on a laptop. A 70-billion-parameter chat model needs a GPU (a graphics chip that does the maths of AI far faster than a normal processor) with tens of gigabytes of memory.

Inference Providers are the cloud-kitchen option. Inference means running a trained model to get an answer. The Hub forwards your request to a partner company that already has the model loaded on its GPUs, returns the answer, and bills your Hugging Face account. One token, many providers, no servers to manage. Free accounts get a small monthly credit; beyond that you pay per use.

import os
from dotenv import load_dotenv
from huggingface_hub import InferenceClient

load_dotenv()                 # puts HF_TOKEN from .env into the environment
client = InferenceClient()    # reads HF_TOKEN; picks an available provider

# Model availability changes; check the model page's "Inference Providers" panel
MODEL = os.getenv("HF_CHAT_MODEL", "openai/gpt-oss-20b")

reply = client.chat_completion(
    model=MODEL,
    messages=[{"role": "user", "content": "Write a two-line apology for a late birthday cake."}],
    max_tokens=200,
)
print(reply.choices[0].message.content)

The Hub’s router also speaks the same request format as the OpenAI API, so tools built for OpenAI often work by changing the base URL. The Inference Providers docs show how.

Library APIs change quickly. If a name has moved, check the docs linked below.

Here is the honest comparison, which is what Meera really asked about:

	Run it yourself	Inference Provider
Cost per request	Close to zero once set up	Pay per use
Setup	Install, download, maybe a GPU	A token and five lines of code
Privacy	Data never leaves your machine	Data goes to the provider
Big models	Slow or impossible without a GPU	Fast
Who keeps it running	You	Them, but models can be withdrawn

Show the answer

Which provider actually receives the data, and what their terms say about storing or training on it; which country it is processed in; the monthly cost at 40,000 emails; and whether the model you depend on will stay available on that provider. For a three-label sorting job that a 400M-parameter model already handles on a laptop, the honest answer is “no need”. Providers earn their place when the model is too big to run yourself.

Where it breaks

The sorter ran happily for three weeks. Then one night the job failed with No space left on device. Arjun had tried six models while experimenting, and every one of them was still sitting in the cache, 14 GB in total, on a laptop with a small disk. The fix took two commands:

uv run hf cache ls                                   # what is cached, and how big
uv run hf cache rm model/some-owner/some-old-model   # remove one you no longer use

That is the mundane failure. Here are the others experienced engineers watch for:

• Popular but wrong for you. bart-large-mnli is English-only. An email like “Bhaiya cake abhi tak nahi aaya” (Hinglish for “the cake still hasn’t come”) confused it. Crumb gets a few of those every day. Test on your traffic, including its messiest corners.
• Silent upgrades. If the author pushes new weights and you did not pin a revision, your accuracy can change overnight with no code change. Pin what you tested.
• Remote code. Some repos ship their own Python code, and loading them needs trust_remote_code=True, which runs that code on your machine. Only do it for publishers you trust, and pin the revision.
• Licences. Read them before you download, not after you ship. Save a copy with your project.
• Leaked tokens. A write token in a public GitHub repo lets strangers push to your models. Keep tokens in .env, keep .env in .gitignore, and delete any token you leak straight away.
• Provider churn. A model offered by providers today may not be offered next month, and free credit runs out fast. Have a fallback.

Try it yourself

1. On huggingface.co, open distilbert/distilbert-base-uncased-finetuned-sst-2-english. From the card and the files tab, find its licence, its parameter count and the size of model.safetensors. Then work out the bytes per parameter.
2. Download only the files transformers needs with hf download ... --include.
3. Run it on these three reviews:

from transformers import pipeline

clf = pipeline("sentiment-analysis",
               model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
print(clf([
    "The Christmas plum cake was perfect, my mother cried.",
    "Delivery was two hours late and the cream had melted.",
    "Ordered the eggless brownies.",
]))

Show the answer

The licence is Apache 2.0 (commercial use allowed). It has about 67 million parameters, and model.safetensors is about 268 MB: again 4 bytes per parameter. The first review comes back POSITIVE (about 0.9997) and the second NEGATIVE (about 0.993). The third is neutral, yet the model says NEGATIVE with about 0.997 confidence. It only knows two labels, so it is forced to pick one, and its confidence score does not warn you. A model can only answer in the labels it was trained on, and high confidence is not the same as being right.

Common mistakes

• Downloading before reading the card. You pull gigabytes, then discover a non-commercial licence or an English-only model. Read first.
• Downloading every file. Use --include and take one weights format, preferably .safetensors.
• Trusting downloads as quality. Measure accuracy on your own labelled examples, even fifty of them.
• Not pinning a revision. Production code should name the exact commit you tested.
• Forgetting the cache. It grows silently. Check it with hf cache ls.

Recap

• The Hub hosts models, datasets and Spaces, each in a versioned Git repo named owner/name.
• The model card tells you task, licence, size, languages and limits. Read it before downloading.
• hf auth login, hf download --include ... and hf cache ls cover most day-to-day CLI work.
• load_dataset(...) loads Hub datasets or your own CSV files, so you can measure a model on real data.
• You can run a model yourself (near-free per call, private, needs hardware) or call it through an Inference Provider (no setup, pay per use, data leaves your machine).

That evening Arjun explained it to Meera. “The Hub is a library of models other people trained. I borrowed one with an MIT licence, which means we can use it in the business. It runs on the office laptop for about three hours a night, costs us nothing per email, and no customer data leaves the building. It gets about 88 out of 100 right, so anything it is unsure about still goes to a person.”

Meera nodded slowly. “Good. Now, sorting is nice. But what I really want is something that can write the replies. Can a free model do that?”

“For that,” Priya said, “you need a model that has been taught to follow instructions. Tomorrow: Gemma.”

Go deeper

• Hugging Face documentation (Hub, CLI, Inference Providers, Datasets): https://huggingface.co/docs
• Hugging Face LLM course: https://huggingface.co/learn/llm-course