OpenWebUI

Meera kept a tally on the back of an envelope for one week. Store managers phoned head office 74 times with questions whose answers were somewhere in the 200-page store operations handbook. “What do I do if the walk-in fridge reads 7 degrees at opening?” “How many days of leave can a part-timer take?” “Who signs off a cash shortage over ₹500?” Each call took one of three senior people away from their work for five to ten minutes.

“I watched Arjun use the AI on Friday,” she said. “He typed into a black window. I’m not doing that, and neither is Farhan at the Kothrud store. I want a page like ChatGPT. Our page, on our server, with our handbook in it.”

Priya smiled. “Someone has already built that page. We’re going to install it, and then we’re going to be careful with it, because the moment it’s useful, it’s also a database of everything our staff ask.”

A shop counter for the kitchen

Ollama, from the first two lessons, is a kitchen with no counter. It cooks anything you hand it through the back door, if you know the exact format of the order slip. That is fine for programs. It is useless for Farhan.

Open WebUI is the shop counter: a website your staff open in a browser, with a sign-in page, a box to type questions, a dropdown to pick a model, a history of past chats, and a way to attach documents. Behind the counter it passes every order through to the kitchen, Ollama, and brings the answer back. It looks and feels much like ChatGPT, which means nobody needs training.

Where the analogy breaks: a shop counter forgets each customer when they leave. Open WebUI remembers everything. It stores every account, every chat and every uploaded document in its own database. That makes it convenient, and it makes the Open WebUI server one of the most sensitive machines in the building.

What you get out of the box:

• Accounts and roles. Each person signs in. The first account ever created becomes the admin, who manages users, models and settings. Other users get the “user” role or wait as “pending” until the admin approves them.
• A model picker. Every model Ollama has pulled appears in a dropdown. You can also add any OpenAI-compatible API (the request format many providers copy) as a connection, so cloud and local models sit side by side.
• Chat history, searchable, per user.
• Knowledge. Upload documents into a named collection, and the chat can answer from them. This is RAG (retrieval-augmented generation, Module 09): find the relevant passages, paste them into the prompt, let the model answer from them.
• Model presets. A named setup that bundles a base model, a system prompt, settings and attached knowledge, like the Modelfile from lesson 1 but made with a form instead of a text file.

Arjun’s first attempt

Arjun had it running on the GPU server in ten minutes on Saturday. He wanted managers to try it from their stores over the weekend, so he did two things to make it easy. He set up a port forward on the office router so the page was reachable from the internet at the office’s address on port 3000, and he left new sign-ups open with the default role set to “user”, so managers could create their own accounts. He posted the link on the managers’ WhatsApp group.

Then he uploaded the whole 200-page handbook PDF into a chat and asked the fridge question. The answer was confident and well formatted: “Keep the walk-in fridge below 8°C. If it is above 8°C, call maintenance.”

Two things were wrong, and he found them on Monday.

First, the admin panel listed 23 accounts. Fourteen were store staff. Nine were not: names like “test”, “admin2” and “x”, created from addresses outside India in the early hours of Sunday. Automated scanners sweep the internet for exactly this kind of page, and an open sign-up form is an invitation. Those strangers had been chatting with Crumb’s GPU, and could see any model and knowledge collection shared with all users.

Second, the handbook’s actual rule, in section 7.3, is “Walk-in fridge: 1 to 4°C. If above 5°C at opening, discard cream-based fillings made the previous day and record it in the cold-chain log.” The “below 8°C” came from Appendix C, an old 2019 equipment specification that was never removed from the PDF. The model had answered faithfully from the wrong page.

“Two separate lessons,” Priya said. “Who can reach it. And whether you should believe it.”

How the pieces fit together

Before fixing anything, Priya sketched what was running.

flowchart LR
    B["Browser on a store laptop"] -->|"office network or VPN"| W["Open WebUI container, port 8080"]
    W --> D["Its data volume: users, chats, uploads"]
    W --> E["Embedding model and vector store"]
    W -->|"OLLAMA_BASE_URL"| O["Ollama container"]
    O --> G["GPU"]
    W -.->|"optional"| C["Cloud API via OpenAI-compatible connection"]

The browser only ever talks to Open WebUI. Open WebUI talks to Ollama on the server’s private network, using an address you give it in the OLLAMA_BASE_URL setting. Everything Open WebUI remembers lives in one folder inside its container, /app/backend/data, which you mount as a Docker volume (storage that outlives the container, from the last lesson). By default that folder holds a small SQLite database file for users and chats, the uploaded files, and a vector store for the knowledge feature.

Show the answer

The model runs on the GPU server in the back office, inside the Ollama container. The chat is stored on the same server, in the open-webui volume. Farhan’s laptop only shows a web page, so the history is not on it (though a logged-in browser on a stolen laptop could still open the page, which is why sessions should expire and the admin should be able to disable his account). The flip side: the server holds everyone’s chats, so it is the machine to protect and back up.

Setting it up the careful way

If Ollama runs directly on the host (for example natively on a Mac), the one-command start from Open WebUI’s docs is:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -e WEBUI_SECRET_KEY="$(openssl rand -hex 32)" \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

Piece by piece:

• -p 3000:8080: Open WebUI listens on port 8080 inside the container; you open http://localhost:3000 on the host.
• --add-host=host.docker.internal:host-gateway: inside a container, localhost is the container itself. host.docker.internal is a name for “the machine running Docker”. Docker Desktop provides it automatically; on Linux this flag creates it.
• -v open-webui:/app/backend/data: the volume that holds every user and chat. Never run without it.
• WEBUI_SECRET_KEY: the key used to sign login sessions. If it changes every time the container is recreated, everyone is logged out. Generate it once and store it in a .env file rather than regenerating it as above on every run.
• --restart always: Docker restarts it after a crash or a reboot.

On Crumb’s server, Ollama already runs from the compose.yaml of the last lesson, so Open WebUI joins that file as another service and reaches Ollama by its service name:

# added to compose.yaml, under services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main   # pin a release tag once you are happy
    ports:
      - "10.0.0.5:3000:8080"                    # the server's office-network address only
    volumes:
      - open-webui:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434     # service name, not localhost
      - WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY}    # read from the .env file next to compose.yaml
    depends_on:
      - ollama
    restart: unless-stopped

# and under volumes:
  open-webui:

The port line is the first fix from Monday. 10.0.0.5 stands for the server’s address on the office network (yours will differ). Stores reach the office network through a VPN, a private tunnel from each store’s laptop into the office, rather than through a hole in the router. Run docker compose up -d, open the page, and create your own account first, because the first account becomes admin. Recent versions turn open sign-up off once the admin exists; check under the admin panel’s settings that it is off, or that new users arrive as “pending” so you approve each one. Arjun deleted the nine strangers’ accounts, changed the secret key to log everyone out, and closed the router port.

Menu names in Open WebUI move between versions. If a setting is not where this lesson says, search its docs.

Following one handbook question

The second fix was the knowledge base, and the clearest way to understand it is to follow one question through it. Arjun first deleted Appendix C from the PDF, then created a knowledge collection called “Ops handbook” under the Workspace menu and uploaded the cleaned file.

At upload time (once).

1. Open WebUI extracts the text. The 200 pages hold roughly 500,000 characters. (A scanned PDF that is only pictures of pages would yield almost nothing here; it needs text extraction first.)
2. It splits the text into chunks, overlapping pieces of around 1,000 characters each (a common default at the time of writing; Module 09 covers choosing sizes). That gives roughly 550 chunks.
3. It turns each chunk into an embedding, a list of numbers that captures its meaning (Module 07). The default small embedding model at the time of writing, all-MiniLM-L6-v2, produces 384 numbers per chunk.
4. It stores the chunks and their embeddings in its vector store, inside the open-webui volume.

At question time. Farhan picks the “Handbook helper” model (set up in the next section) and types: “Walk-in fridge reading 7 degrees at opening, what do I do?”

1. The question is embedded into 384 numbers with the same embedding model.
2. The vector store returns the chunks whose embeddings are closest in meaning. Illustrative results with their similarity scores:

Chunk	From	Similarity
212	7.3 Cold chain: “Walk-in fridge: 1 to 4°C. If above 5°C at opening, discard cream-based fillings made the previous day…”	0.71
214	7.4 Cold-chain log: “Record the reading, the time, what was discarded and your initials…”	0.66
88	3.2 Cleaning roster: “Walk-in fridge shelves are wiped every Monday…”	0.52

3. Open WebUI builds the prompt: the preset’s system prompt, the three chunks, then Farhan’s question. About 950 tokens in total (a token is a chunk of text, roughly three-quarters of an English word).
4. Ollama runs llama3.1:8b on the GPU: about 0.3 s to read the 950 tokens and about 1.2 s to write a 130-token answer at roughly 110 tokens per second.
5. The answer comes back with citations, small links to the chunks it used, which Farhan can click to see the handbook text:

Your fridge is above the 5°C limit (section 7.3).
1. Discard cream-based fillings made yesterday.
2. Record the reading, time, what you discarded and your initials in the cold-chain log (7.4).
3. If it is still above 4°C after 30 minutes, call maintenance.
Sources: Ops handbook, 7.3 and 7.4

The cost. Locally, ₹0 per question beyond electricity. The same 950 tokens in and 130 out on a cloud model, at made-up round prices of $1permillioninputtokensand$5 per million output tokens, would be about $0.0016, roughly ₹0.14. Seventy-four questions a week costs almost nothing either way. The local version wins on privacy, not price. The real saving is the 74 phone calls.

Notice what made the Saturday answer wrong. Not the model: retrieval handed it a chunk from Appendix C, and it answered from what it was given. Clean sources and visible citations fix more RAG failures than a bigger model does.

Show the answer

The model can only use what it is given, so it will describe the 2023 policy, and it may say nothing changed, or it may invent a change to fill the gap. A perfect model cannot read a document it never received. The manager notices only by checking the citations: every source says 2023. That is why the preset’s system prompt should say “if the handbook does not contain the answer, say so”, and why someone owns keeping the knowledge collection up to date.

A preset called “Handbook helper”

Choosing the model, pasting instructions and attaching the handbook every time would defeat the point. Under Workspace, Models, Arjun created a preset:

• Name: Handbook helper
• Base model: llama3.1:8b
• System prompt: “You answer questions for Crumb & Co. store managers using only the attached Ops handbook. Quote section numbers. If the handbook does not contain the answer, say ‘Not in the handbook, please call head office.’ Never guess temperatures, amounts or times.”
• Knowledge: Ops handbook
• Temperature: 0.2, for steady, repeatable answers.
• Access: the “Store managers” group only.

Farhan now sees “Handbook helper” in the model dropdown and nothing else he needs to know about. You can also pull a knowledge collection into any chat by typing # in the message box and choosing it.

A week later, managers had a complaint: some questions spanned several sections (“closing checklist for a Sunday when the delivery partner hasn’t come”), and three retrieved chunks were not enough. Priya suggested a second preset for those, “Handbook (full)”, using a cloud model through an OpenAI-compatible connection, with the 60-page “core procedures” part of the handbook, about 30,000 tokens, placed in full in the system prompt. The handbook contains no customer data, so Meera was happy for it to go to the cloud. Answers improved. Remember that 30,000-token number; it comes back at the end of this module.

Where it breaks

Running it yourself means owning what a SaaS company would normally own:

• Backups. Losing the volume loses every account, chat and collection. Back it up on a schedule and test a restore:

docker run --rm -v open-webui:/data -v "$(pwd)":/backup alpine \
  tar czf /backup/open-webui-$(date +%F).tgz -C /data .

• Updates. The main tag changes often. Pin a release tag, read the release notes, back up, then upgrade. Database changes between versions are usually automatic, which is exactly why you want a backup first.
• Built-in RAG is a good start, not a finished product. It is excellent for trying an idea in an afternoon. For the answers staff will act on, check the citations, test with real questions, and know that Modules 09 to 10 and 23 teach you to build and measure this properly.
• One GPU, many managers. At 9 a.m. eight managers open the page together. Ollama works through requests one by one by default, so the eighth waits several seconds. Lesson 4 looks at concurrency.
• The licence. Open WebUI was BSD-licensed until version 0.6.5. From version 0.6.6 (April 2025) its licence adds a branding clause: you may not remove or change the “Open WebUI” branding unless you stay under a small user count, have permission, or buy an enterprise licence. Using it internally as-is is fine; if you plan to rebrand it as “Crumb Assistant”, check the current licence on its docs first.

Try it yourself

1. Start Ollama and Open WebUI (either the single docker run above or the compose file). Pull llama3.2 if you have not.
2. Sign up: you are the admin. Check the sign-up setting in the admin panel.
3. Create a three-page PDF of your own: page 1 a cake menu, page 2 delivery areas, page 3 one oddly specific rule, such as “Orders over 4 kg need 48 hours’ notice”. Upload it as a knowledge collection.
4. In a chat, type #, attach the collection and ask: “How much notice for a 5 kg cake?” Then ask: “Do you deliver to Mumbai?” when page 2 lists only Pune areas.

Show the answer

The first question should come back with “48 hours” and a citation pointing at page 3. If it does not, open the citation: retrieval probably returned the menu page instead, which shows how small changes in wording affect what is found. The second question is the honesty test. A good answer says Mumbai is not in the listed areas. A weak one invents a delivery charge. If you get the weak one, add “say when the documents do not contain the answer” to a preset’s system prompt and try again.

Common mistakes

• Exposing it to the internet with open sign-ups. Use a private network or VPN, approve each account, and create the admin account before anyone else can.
• Running without the data volume. Recreate the container and every user and chat is gone.
• Letting WEBUI_SECRET_KEY change on each start. Everyone gets logged out on every restart. Set it once in .env.
• Trusting answers without opening the citations. Retrieval can hand the model the wrong page, as Appendix C did. Clean the sources and train staff to click the source.
• Uploading scanned PDFs. Images of text produce no text. Check that a few chunks contain real words.
• Pointing OLLAMA_BASE_URL at localhost. Inside the container that is Open WebUI itself. Use host.docker.internal or the compose service name.

Recap

• Open WebUI is a self-hosted chat site in front of Ollama or any OpenAI-compatible API: accounts, history, a model picker, knowledge collections and presets.
• The first account becomes admin; keep sign-ups off or pending, and keep the site on a private network.
• Its data volume holds every user, chat and document, so it needs a fixed secret key, backups and careful updates.
• Its knowledge feature is RAG: extract, chunk, embed, retrieve the closest chunks, answer with citations. Wrong answers usually trace back to what was retrieved.
• Presets bundle a model, a system prompt, settings and knowledge into one dropdown entry for non-technical staff.

At the end of the month Arjun gave Meera the numbers. “Phone calls about the handbook went from 74 a week to 19. The helper answers from the handbook and shows the page it used, so if it’s wrong, the manager can see why. It runs on our server, behind the office network. We back it up every night.”

“And the nine strangers?”

“Gone, and they can’t get back in.”

That afternoon, Kabir, who runs the WhatsApp ordering line, stopped by Arjun’s desk. “Meera says you’ve got an AI in the back room now. We get 1,800 WhatsApp messages a day, and someone reads every one to sort orders from complaints from allergy questions. Can our system send each message to it automatically? I don’t need a chat page. I need something our software can call.”

Priya, passing, answered for him. “He needs an API. Let’s build one.”

Go deeper

• Open WebUI documentation, including the quick start, features and licence pages: https://docs.openwebui.com
• Docker documentation, for volumes, networking and Compose: https://docs.docker.com
• The Ollama repository and its docs: https://github.com/ollama/ollama
• A refresher on what the knowledge feature is doing under the hood: