LlamaIndex vs LangChain

By the end of the month, Arjun had the handbook bot in three versions. His hand-written Python, four files and about 310 lines. A LangChain version from the chunking lesson. And the LlamaIndex version from last week. All three scored within two points of each other on the 40-question golden set. He wanted to delete two of them and could not decide which two.

So he did what many developers do. He opened a Reddit thread titled “LangChain vs LlamaIndex in 2026?”, read 140 comments, counted GitHub stars, and posted a poll in the Crumb tech WhatsApp group. Three people voted. One of them was the delivery partner integration contractor, who had never seen the bot.

Then, on Thursday, the decision made itself urgent. Rohan, who helps Arjun with the online shop, ran a routine package upgrade on the LangChain version. The app would not start. One import line copied from a year-old tutorial pointed to a module that no longer existed in the new release. It took Arjun ninety minutes to find where it had moved, during the afternoon rush.

Meera, passing by during the outage, asked the obvious question. “Which one of these is cheapest?”

“None of them,” Priya said. “They’re all free libraries. What you pay for is the model calls, and those are the same in all three. The real cost is different. It’s how long it takes Arjun to fix it at 11 p.m. during Diwali week. Today it was ninety minutes.”

Why the poll was the wrong tool

“What did the poll tell you?” Priya asked.

“That two people like LangChain and one likes LlamaIndex.”

“Did any of them know what Crumb’s bot does?”

“No.”

“Then it told you about them, not about us.” She wrote three questions on the whiteboard. What does our app mostly do? Who will maintain it? What happens when it breaks? “Popularity is a fine tie-breaker. It is a terrible first question. The Reddit thread is mostly people describing the one project they built.”

There is a misconception underneath Arjun’s approach worth naming: the belief that one of these frameworks is better, the way one phone is faster than another. They are not competing on speed or answer quality. In the previous lesson you saw why: the framework wires the pipeline, but the answer quality comes from chunking, retrieval, prompts and evaluation, all of which you control in any of them. Arjun’s three versions scored the same because they made the same choices.

Three kitchens

Priya drew three kitchens on the board.

A meal kit (LlamaIndex). Everything for one style of dish arrives pre-measured with a recipe card. You get a good result fast, and the kit is especially good at its speciality: here, reading many kinds of documents, splitting them, indexing them and retrieving from them.

A huge supermarket (LangChain). Ingredients for any cuisine, in standard packaging, so they combine easily. Far more choice, more aisles to learn, and the store gets rearranged every few months. Its newest section, LangGraph, is where you build agents (programs where the model decides what to do next, calls tools and loops until done), which you will meet in Modules 13 and 14.

Cooking from raw ingredients (no framework). Rice, onions, spices, your own recipe. Slower for a wedding feast; for a weeknight dal it is faster than reading the meal-kit card, and you know exactly what went in.

“So which kitchen makes the best dal?” Priya asked.

Arjun thought about it. “Depends who’s cooking. And how much dal.”

“And whether you’re also making biryani for two hundred next week.”

Where the analogy breaks: in software you can shop in all three at once. Plenty of real systems use a LlamaIndex reader inside a LangGraph agent, or LangChain’s PDF loader feeding hand-written retrieval code. Choosing one does not lock you out of the others. That also means the real question is rarely “which framework?”, but “which framework for which part?”.

Following one change request through all three versions

Priya suggested a better test than a poll: take a real change Meera had asked for, and walk it through each version. The change: “Add the HR policies, but store staff must never see the salary bands section, and HR wants to add supplier contracts from a Google Drive folder next month.”

That breaks into three jobs: load a new file type, restrict access by role, and add a new data source. Here is what each version needed, from Arjun’s notes.

Job 1: load the 30-page HR policy PDF.

Version	What changed
Plain Python	Reused the PDF loading function; about 5 new lines to tag chunks with department
LangChain	PyPDFLoader for the new file; same 5 lines of tagging
LlamaIndex	Dropped the PDF in the folder; SimpleDirectoryReader picked it up; same tagging loop

Roughly a tie. Loading a PDF is a solved problem everywhere.

Job 2: hide the salary bands from store staff.

All three handled it with a metadata filter at search time: a label like "access": "managers" on the salary chunks, and a filter built from the user’s role. In plain Python with Chroma it was a where={"access": {"$in": allowed}} argument; in LangChain, a filter inside search_kwargs; in LlamaIndex, a MetadataFilters object. Three to six lines each. Another tie, and an important one: the safety rule lives in your code in every version, not in the framework.

Job 3: pull supplier contracts from Google Drive.

Version	What it would take
Plain Python	Write Google Drive API code: authentication, listing files, downloading, converting. Arjun estimated 60 to 100 lines and a day
LangChain	Install a community Google Drive loader; configure credentials; about 15 lines
LlamaIndex	Install a Google Drive reader from LlamaHub; configure credentials; about 15 lines

Here the frameworks clearly won. Connectors are their biggest practical gift: someone else already fought with that API.

Then the incident. A week later, a store manager reported that the bot cited page 41 for a rule that was on page 14. Arjun timed how long it took to find the cause in each version.

• Plain Python: about 10 minutes. He printed the chunk metadata, saw that the page number was being read from the wrong field, fixed one line.
• LangChain and LlamaIndex: about 40 minutes each, mostly spent finding where the page metadata was set and which page-numbering convention each loader used (zero-based in one, the printed page label in the other).

“So frameworks are faster to build with and slower to debug,” Arjun said.

“For this team, on this app. That’s the honest answer. Notice it’s not ‘framework good’ or ‘framework bad’. It’s a trade you can now see.”

Opening the hood: what a framework actually costs you

Every framework is a set of abstractions: names and interfaces that hide details so you can think at a higher level. query_engine.query(...) hides a retriever, a postprocessor chain, a prompt template and an LLM call. That is the whole value. It is also the whole cost, in three forms.

1. Abstraction leakage. Hidden details stop being hidden the moment something goes wrong. The page-number bug lived inside a loader’s convention that Arjun had never needed to know about, until he did. Engineers call this a leaky abstraction: you get to ignore the details until they bite, and then you need to know them anyway, plus the framework’s layers on top.

2. Dependencies. Arjun checked how many Python packages each setup pulled in when resolved with uv:

Setup	Packages installed
anthropic + rank-bm25 (plain Python, no vector database)	18
llama-index	69
chromadb + anthropic (plain Python with Chroma)	88
langchain, langchain-community, langchain-text-splitters, langchain-chroma	107

These numbers change with every release, so treat them as a snapshot, not a ranking. Notice too that the vector database, not the framework, is the heaviest single piece. Every package is something that can change, break or need a security update, and a lock file (which uv writes for you as uv.lock) is what keeps today’s working set from silently becoming tomorrow’s broken one.

3. Churn. Both frameworks move fast. LangChain has reorganised itself more than once; its 1.0 release in late 2025 refocused the main package on agents built on LangGraph and moved many older patterns into a separate “classic” package, which is why Rohan’s tutorial import broke. LlamaIndex splits into a core package plus many small integration packages, whose names and versions also move. Plain Python changes only when your SDKs do.

None of this makes frameworks a bad idea. It makes them a trade, and you should know the price before you pay it.

The honest comparison

	LlamaIndex	LangChain (with LangGraph)	No framework
Built for	Connecting LLMs to your data: reading, indexing, retrieving	General LLM apps: integrations, chains, tools; LangGraph for agents	Exactly what you write
Shines at	Many document types, node relationships, retrieval add-ons, query engines that return sources	The largest integration catalogue; stateful agents and workflows	Clarity, control, fast debugging, few dependencies
Time to first demo	Minutes	Minutes to an hour	An hour or two
Seeing the final prompt	Possible; hidden by default	Possible; hidden by default	Always: you wrote it
Upgrade pain	Moderate	Historically high	Low
Agents	Has its own agent and workflow tools	LangGraph, a leading choice (Module 14)	You write the loop (Module 13 shows how)

Some honest notes behind the table:

• Both frameworks are good and actively maintained. Neither is a wrong choice for RAG. Many online arguments mostly reflect what each author built.
• LlamaIndex goes deeper on retrieval. Node relationships, many index types and retrieval postprocessors are its core, not an add-on. Its company also sells hosted services, such as a paid document parser; the open-source library works without them.
• LangChain’s breadth is real. If you need to talk to an unusual database, a niche model provider and three SaaS tools in one app, it probably has a connector for each.
• Plain Python is underrated. A RAG answer path is: retrieve, rerank, format, call the model, check. You wrote one in about 40 lines in the pipeline lesson. Anthropic’s own guidance on building agents makes the same point: start with the simplest thing that works, and add framework layers only when they earn their place.

The same retrieval, three ways

The whole difference, in miniature. Each snippet takes a question and returns the top 4 chunk texts from an index you already built.

# LlamaIndex
retriever = index.as_retriever(similarity_top_k=4)
chunks = [r.node.get_content() for r in retriever.retrieve(question)]

# LangChain
retriever = vectordb.as_retriever(search_kwargs={"k": 4})
chunks = [d.page_content for d in retriever.invoke(question)]

# No framework (Chroma directly)
chunks = collection.query(query_texts=[question], n_results=4)["documents"][0]

For basic retrieval, the code is nearly identical: two lines each. The differences only appear at the edges: how much the framework does that you did not ask for, how easily you can change one piece, and what changes when you upgrade.

A decision guide

flowchart TD
  A["What are you building?"] --> B{"Mostly Q and A over documents?"}
  B -- Yes --> C{"Many formats, sources or advanced retrieval?"}
  C -- Yes --> D["LlamaIndex for ingestion and retrieval"]
  C -- No --> E["Plain Python, borrow a loader if needed"]
  B -- No --> F{"Agents, tools, multi-step workflows?"}
  F -- Yes --> G["LangGraph (LangChain family)"]
  F -- No --> E

And four questions to answer, in writing, before you commit:

1. What does the app mostly do? Retrieval over many sources leans LlamaIndex. Agents with tools and branching lean LangGraph. A simple question-and-answer path leans plain code.
2. Who will maintain it? A team that already knows one framework should usually keep it. Familiarity beats small technical differences, every time.
3. How much control and auditability do you need? Answers about allergens, money or HR policy favour code where every step is visible and logged.
4. How often can you afford to upgrade? Pin versions with a lock file, and upgrade on purpose, on a quiet afternoon, with the golden set as your safety net.

What Crumb chose, and why

Priya and Arjun settled on a mix, and wrote the reasons into the README so the next person would not need a Reddit thread:

• Loading and splitting: LlamaIndex. The handbook, HR policies and soon the Google Drive supplier contracts come in mixed formats. LlamaIndex readers handled them with the least code, and its connectors will save a day when Drive arrives.
• Hybrid search, RRF, reranking, abstention, citations: plain Python. About 80 lines. The team understands every one, the abstention rules are exactly what Meera signed off, and the page-number incident showed this is where fast debugging matters most.
• The future WhatsApp ordering assistant: LangGraph. That is an agent with tools (checking stock, creating orders) and a human approval step for large refunds, which is what LangGraph is built for. A later module.
• Pinned versions, a lock file, and a monthly “upgrade afternoon” where someone upgrades on a branch and runs the golden set before merging.

Not one winner. Each tool where it is strongest, and the business rules in code the team owns.

Where it breaks

• Mixing frameworks doubles the churn. Using two frameworks means tracking two sets of breaking changes. Keep each one at the edges, behind your own functions.
• Tutorials age fast. A tutorial from a year ago may use imports or patterns that no longer exist. Check the date, and prefer the official docs.
• Framework features can quietly move to paid services. Read what is open source and what needs an account before you build on it.
• Plain Python can sprawl. “No framework” works while the code stays small. If you find yourself writing your own connector for every SaaS tool, or your own agent state machine, that is the moment a framework starts paying for itself.

Try it yourself

For each scenario, pick LlamaIndex, LangChain/LangGraph, or no framework, and write one sentence of reasoning.

1. A two-person startup wants a chatbot over 5,000 PDFs, Notion pages and Google Docs, shipped this month.
2. A single internal FAQ of 30 questions that changes twice a year.
3. A support agent that looks up orders, issues refunds through an API, and asks a human before any refund over ₹2,000.
4. A hospital’s policy assistant where auditors must be able to see every prompt and every retrieved passage.

Show the answer

1. LlamaIndex. Many sources and formats, and speed to ship matters. Its connectors and defaults get there fastest; then set the defaults explicitly and tune with RAGAS.
2. No framework, and possibly not even RAG. Thirty questions fit comfortably in one prompt. Paste them in, perhaps with prompt caching (Module 21).
3. LangGraph. This is an agent with tools, branching and a human approval step. Retrieval, if any, is a small part.
4. Plain code for the core, frameworks only at the edges. Auditability favours code where the prompt and retrieved passages are explicit and logged. Borrowing a loader is fine.

Reasonable people could argue 1 or 4 differently. What matters is that your reason is about the app and the team, not the logo.

Common mistakes

• Choosing by stars, polls or hype. Choose by what the app does and who maintains it.
• Using a stack you cannot see inside. Log the final prompt and retrieved chunks from day one.
• Not pinning versions. A routine reinstall can pull a release that renamed half your imports. Commit your lock file.
• Putting business rules inside framework configuration. Keep access control, abstention and citation rules in your own functions, where they survive a framework switch.
• Believing a framework improves answer quality. Quality comes from chunking, retrieval, prompts and evaluation, all of which you control in any of the three.

Recap

Meera came back after the outage was fixed and asked for the decision “in one minute, no brand names”. Arjun gave it to her.

• None of the three costs money by itself. The bill is the AI calls, which are the same in all three. The real cost is time spent building and, especially, fixing.
• The frameworks are great at the edges: reading lots of file types and connecting to things like Google Drive.
• Our own code is better at the core, where the rules live (who can see salary bands, when to say “I don’t know”), because we can see and fix every line quickly.
• So we use a framework for reading documents, our own code for answering, and LangGraph later for the ordering assistant.
• We lock the versions, and we upgrade once a month, on purpose, with the 40 test questions as a safety net.

Meera nodded. “Ninety minutes today. Don’t let it be ninety minutes during Diwali.” She tapped the whiteboard where the three kitchens were. “Good. Now I want the real thing. The whole handbook, the HR policies, the hybrid search, the ‘I don’t know’ rules, nothing personal leaking, and that scorecard on top. Something I could show the store managers at the quarterly meeting.”

Priya looked at Arjun. “That’s your project.”

Go deeper

• LlamaIndex documentation: https://docs.llamaindex.ai
• LangChain documentation: https://python.langchain.com
• LangGraph documentation: https://langchain-ai.github.io/langgraph/
• Building effective agents (Anthropic, on starting simple): https://www.anthropic.com/research/building-effective-agents