Pinecone end to end

Meera comes back from a meeting with news. Three independent cafés in Pune, Brew Lane, Kettle & Co and Mocha Junction, already buy Crumb’s cakes wholesale. They have seen Crumb’s support assistant answer customer questions on the website, and they want one for themselves. Each café would get an assistant that answers from its own menu, prices, opening hours and policies.

“They’ll pay a monthly fee,” Meera says. “But the contract has one line their owners all insisted on: no café’s data is ever shown to another café’s customers. Kettle & Co’s staff discount code appearing on Brew Lane’s website would end the deal.”

Arjun thinks about the Chroma setup on his laptop, then about running Chroma servers for three outside businesses, with backups, uptime and someone to restart things at 2 a.m. Crumb has one engineer, and it is him.

“This is the moment for a managed service,” Priya says. “Someone else runs the servers. You get an API key and a bill. And it has a feature built for exactly this contract.”

Arjun’s first multi-café prototype

Arjun signs up for Pinecone, creates an index (Pinecone’s name for a searchable collection of vectors), and writes the obvious design: all three cafés’ documents in one pile, each record tagged with a cafe field, and every query filtered by café.

Three things go wrong before lunch.

The dimension error. He created the index with dimension=1536, copying a tutorial, then embedded the documents with his local all-MiniLM-L6-v2 model, which produces 384 numbers. The first upsert fails: Vector dimension 384 does not match the dimension of the index 1536. An index’s dimension is fixed when you create it.

The giant request. He switches to OpenAI’s text-embedding-3-small (1,536 numbers) and tries to upsert all 1,200 records in one call. It is rejected for being too large. Requests have a size limit.

The leak. He fixes both, builds the search, and writes a test: for each café, ask “is there a staff discount?” The Brew Lane and Mocha Junction tests pass. Then he finds that the “suggested questions” feature, which he had written separately on Friday, calls the search function without the café filter. Asked “any discounts?”, Brew Lane’s suggestions include Kettle & Co’s line: “Staff discount code KETTLE20 for 20% off.”

No customer ever saw it. It was caught in testing. But it was one forgotten line of code away from ending the deal.

Flats in one building

Priya draws an apartment building. “Think of the Pinecone index as one building. Pinecone is the management company: they handle plumbing, electricity, security guards and repairs. You never touch the boiler.”

“Fine. And the cafés?”

“Your current design is one big hall where everyone’s post is dumped on a table, and each café is trusted to only pick up envelopes with its own name on them. The metadata filter is that rule. It works perfectly, until one person forgets, or is in a hurry.”

“Like my Friday code.”

“Like your Friday code. The alternative is flats. Each café gets its own flat with its own lock. Post for Brew Lane goes into Brew Lane’s flat. When Brew Lane’s customer searches, the search can only happen inside Brew Lane’s flat. It’s not a rule people must remember. It’s a wall.”

“And Pinecone has flats?”

“They’re called namespaces. One index, many namespaces. Every upsert and every query names exactly one namespace. And when a café leaves, you delete their whole flat in one call.”

“What does the building charge?”

“The serverless version charges for what you use: how much you store, how much you write, how much you read. Like a taxi rather than owning a car. Nothing to pay for idle servers at night.”

Following one question from a café’s website to an answer

Trace one record and one query through the whole system.

Step 1: the source text. Brew Lane’s FAQ, entry 12: “Do you have plant-based milk? Yes. We offer oat and almond milk for any coffee at no extra charge. Soy milk is available on request at the Baner outlet only.”

Step 2: embed it. OpenAI’s text-embedding-3-small turns the 32-word entry (about 40 tokens) into 1,536 numbers, already normalised to length 1. Cost: about 0.0000008 US dollars.

Step 3: build the record. A Pinecone record has an ID, the vector (called values), and optional metadata:

{"id": "brewlane-faq-12#c0",
 "values": [0.0213, -0.0087, 0.0441, "... 1,533 more numbers ..."],
 "metadata": {"source": "faq", "entry": 12, "outlet": "all",
              "text": "Do you have plant-based milk? Yes. We offer oat and almond milk..."}}

Pinecone stores vectors and metadata, not your documents as such, so the text that you want to show later goes into metadata. The #c0 suffix marks it as chunk 0 of entry 12, which helps later when you need to find and delete every chunk of one entry.

Step 4: upsert into the café’s namespace. Upsert means “insert, or replace if this ID exists”. The call names the namespace brew-lane. Pinecone writes it to durable storage and indexes it. Within a few seconds it is searchable. A record of 1,536 four-byte numbers plus about 300 bytes of metadata is roughly 6.4 KB; Brew Lane’s whole knowledge base of 400 chunks is under 3 MB.

Step 5: a customer asks a question. On Brew Lane’s website: “can I get oat milk in my latte?” The assistant’s server embeds the question with the same model: another 1,536 numbers.

Step 6: query, inside one namespace.

{"vector": [0.0198, -0.0102, "..."], "top_k": 3, "include_metadata": true,
 "namespace": "brew-lane"}

Step 7: the response.

{"matches": [
   {"id": "brewlane-faq-12#c0", "score": 0.83, "metadata": {"source": "faq", "text": "Do you have plant-based milk?..."}},
   {"id": "brewlane-menu-latte#c0", "score": 0.61, "metadata": {"source": "menu", "text": "Latte: espresso with steamed milk..."}},
   {"id": "brewlane-faq-3#c0", "score": 0.44, "metadata": {"source": "faq", "text": "Do you have sugar-free syrups?..."}}],
 "namespace": "brew-lane"}

The index was created with the cosine metric, so score is the cosine similarity: higher is better, and 0.83 is a strong match. Kettle & Co’s records were never candidates. They live in a different namespace, and this query could not reach them even if Arjun’s code had a bug.

Step 8: what this round trip cost. One embedding call (fractions of a paisa), one small write when the FAQ was loaded, and one read for the query. Pinecone’s serverless pricing counts storage (per gigabyte per month), write units (consumed by upserts, updates and deletes) and read units (consumed by queries and fetches, more for bigger namespaces and larger top_k). The exact rates change, so check the pricing page. At three cafés with a few hundred chunks each and a few thousand questions a month, Arjun’s estimate is comfortably inside the free starter tier. The WhatsApp archive with its 3.2 million records would be a paid workload, and the first thing to estimate is its storage: about 20 GB of vectors.

Opening the hood: Pinecone’s pieces

Projects and API keys. Your Pinecone account holds projects; each project has API keys. Keep the key in an environment variable, never in code.

Indexes. An index has a name, a dimension (the number of values per vector) and a metric (cosine, dotproduct or euclidean). Both are fixed at creation. A serverless index also names a cloud and region. Pinecone handles the index structure, sharding, replication and backups.

Records. Each record has an id, values (the dense vector), optional metadata (a dictionary of strings, numbers, booleans and lists of strings) and, optionally, sparse values for keyword-style hybrid search, which you will meet in the RAG module. Metadata per record has a size limit (around 40 KB), so store passages, not whole documents.

Namespaces. A namespace is a partition inside an index. Every upsert, query, fetch and delete operates on exactly one namespace. If you do not name one, you get the default namespace, whose name is the empty string. Namespaces are created automatically on first upsert, and deleting all records in a namespace removes it. Pinecone recommends one namespace per tenant for multi-tenant apps: queries are faster because they search less data, and isolation does not depend on remembering a filter.

Metadata filters. Within a namespace you can still filter: {"source": {"$eq": "faq"}}, with operators such as $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin and $exists, combined with $and and $or.

Freshness. Pinecone’s serverless indexes are eventually consistent: a record you upsert becomes searchable after a short delay, usually seconds. That matters in tests that upsert and immediately query.

Three ways to do multitenancy. Multitenancy means serving several separate customers (tenants) from one system.

Building the café assistant’s search, step by step

Step 1: install and create the index

uv add pinecone openai python-dotenv

Add PINECONE_API_KEY=... and OPENAI_API_KEY=... to your .env file.

# cafe_index.py
import os
import time
from dotenv import load_dotenv
from openai import OpenAI
from pinecone import Pinecone, ServerlessSpec

load_dotenv()
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
oai = OpenAI()

INDEX = "cafe-assistant"
EMBED_MODEL = "text-embedding-3-small"      # 1,536 dimensions: must match the index

if INDEX not in pc.list_indexes().names():
    pc.create_index(
        name=INDEX,
        dimension=1536,
        metric="cosine",
        spec=ServerlessSpec(cloud="aws", region="us-east-1"),
    )
    while not pc.describe_index(INDEX).status["ready"]:     # wait until it can take data
        time.sleep(1)

index = pc.Index(INDEX)

You should now see the index in the Pinecone console, empty, with dimension 1536 and metric cosine.

Step 2: embed and upsert in batches

def embed(texts):
    resp = oai.embeddings.create(model=EMBED_MODEL, input=texts)
    return [d.embedding for d in resp.data]

def load_cafe(cafe_ns, chunks, batch_size=100):
    """chunks: list of dicts with id, text, source, entry, outlet."""
    for start in range(0, len(chunks), batch_size):
        batch = chunks[start:start + batch_size]
        vectors = embed([c["text"] for c in batch])
        index.upsert(
            vectors=[{"id": c["id"], "values": v,
                      "metadata": {"text": c["text"], "source": c["source"],
                                   "entry": c["entry"], "outlet": c["outlet"]}}
                     for c, v in zip(batch, vectors)],
            namespace=cafe_ns,
        )

load_cafe("brew-lane", brew_lane_chunks)
load_cafe("kettle-co", kettle_chunks)
load_cafe("mocha-junction", mocha_chunks)

Batches of 100 keep each request well under the size limit and match OpenAI’s batching sweet spot. Pinecone’s docs give the exact request limits.

Step 3: check what is stored

stats = index.describe_index_stats()
print(stats["total_vector_count"])
for ns, info in stats["namespaces"].items():
    print(f"{ns:15s} {info['vector_count']}")

1203
brew-lane       412
kettle-co       377
mocha-junction  414

Run this after every load. It is the quickest way to spot a missing café or a doubled upload.

Step 4: query one café, with an optional filter

def search(cafe_ns, question, k=3, source=None):
    q = embed([question])[0]
    res = index.query(
        vector=q, top_k=k, include_metadata=True, namespace=cafe_ns,
        filter={"source": {"$eq": source}} if source else None,
    )
    return [(round(m["score"], 2), m["id"], m["metadata"]["text"][:60]) for m in res["matches"]]

for row in search("brew-lane", "can I get oat milk in my latte?"):
    print(row)

(0.83, 'brewlane-faq-12#c0', 'Do you have plant-based milk? Yes. We offer oat and almond')
(0.61, 'brewlane-menu-latte#c0', 'Latte: espresso with steamed milk, 240 ml. Rs 180. Any mil')
(0.44, 'brewlane-faq-3#c0', 'Do you have sugar-free syrups? We stock sugar-free vanilla')

Step 5: fetch, update and delete

# Read specific records by ID, no similarity involved.
got = index.fetch(ids=["brewlane-faq-12#c0"], namespace="brew-lane")

# Update metadata in place (for example, the Baner outlet stops offering soy milk).
index.update(id="brewlane-faq-12#c0", set_metadata={"outlet": "all"}, namespace="brew-lane")

# Delete every chunk of one FAQ entry: list IDs by prefix, then delete them.
for ids in index.list(prefix="brewlane-faq-12#", namespace="brew-lane"):
    index.delete(ids=ids, namespace="brew-lane")

# Offboard a café completely: one call removes their whole namespace.
index.delete(delete_all=True, namespace="mocha-junction")

The list by prefix is why the IDs were designed as entry#chunk. Library APIs change quickly. If a name has moved, check the Pinecone docs linked below.

Step 6: make the safe path the only path

The real fix for Friday’s leak is not “remember the namespace”. It is code where forgetting is impossible:

class CafeSearch:
    """All search for one café. There is no way to query without a namespace."""
    ALLOWED = {"brew-lane", "kettle-co", "mocha-junction"}

    def __init__(self, cafe_ns: str):
        if cafe_ns not in self.ALLOWED:
            raise ValueError(f"unknown café: {cafe_ns}")
        self.ns = cafe_ns

    def search(self, question, k=3, source=None):
        return search(self.ns, question, k, source)

brew = CafeSearch("brew-lane")         # created once per request, from the logged-in café
print(brew.search("any staff discounts?"))

Every feature, including the “suggested questions” box, now receives a CafeSearch object, never the raw index. The café is decided once, from the website’s login, not from anything a customer types.

Where it breaks

The assistants go live. Within a week, Arjun’s monitoring shows each search taking about 650 milliseconds. Customers notice the pause.

He breaks it down: about 300 ms for the OpenAI embedding call, about 250 ms for the Pinecone query, and the rest in his own code. The index lives in us-east-1, in Virginia. A network round trip from Pune to Virginia and back takes around 200 milliseconds before any work is done. Each request crosses the world twice, once to OpenAI and once to Pinecone.

Arjun moves the café assistant’s server to a cloud region near the index, so the two hops to OpenAI and Pinecone become short, and only the final answer crosses to Pune. Search time drops to about 250 ms.

Other things that bite in production:

• Dimension or metric mismatch. Both are fixed at creation. A new embedding model with a different dimension needs a new index.
• Testing too fast. Upserting and then querying in the next line can return nothing, because of eventual consistency. Wait briefly, or poll describe_index_stats, in tests.
• Stale chunks. As in Chroma, an edited FAQ entry that produces fewer chunks leaves old ones behind. Delete by ID prefix before re-loading an entry.
• Paying for what you do not use. Large top_k values and returning the vectors themselves (include_values=True) cost more read capacity and bandwidth. Ask for what you need.
• Metadata as a database. Metadata is for filtering and display. Prices and stock that change hourly belong in your main database, looked up by ID after retrieval.
• Keys in the browser. The Pinecone API key gives full access to the project. It must live only on your server, never in website JavaScript.

Arjun explains it to Meera

Meera has the café contract open. “Tell me how the wall works, in words I can repeat to the café owners.”

“All three cafés’ information lives in one database run by a company called Pinecone,” Arjun says. “Inside it, each café has its own separate space. When a Brew Lane customer asks a question, the search happens only inside Brew Lane’s space. It isn’t a setting someone might forget. The search can’t reach the other spaces.”

“And if a café leaves?”

“One command deletes their whole space, and I can show them the count going to zero.”

“What does it cost us?”

“At our size, almost nothing. Pinecone charges for storage, writing and searching. Three cafés with a few hundred passages each is tiny. The bigger cost was speed: our database was on the other side of the world. We moved our server next to it, and searches went from about 0.65 seconds to 0.25.”

“And we don’t have to run anything at 2 a.m.?”

“Pinecone runs the database. We run one small app.”

Meera signs the contract that afternoon.

Try it yourself

1. Create a free Pinecone index with dimension 1536 and metric cosine.
2. Load two fake “tenants” into two namespaces, each with ten short FAQ entries, including one question both tenants answer differently (“what time do you close?”).
3. Query the shared question in each namespace and confirm each returns its own answer.
4. Print describe_index_stats(), delete one tenant with delete_all=True, and print it again.

Expected result: each namespace answers with its own closing time; the stats show two namespaces, then one.

import time
def wait_for_count(ns, expected, timeout=30):
    for _ in range(timeout):
        info = index.describe_index_stats()["namespaces"].get(ns)
        if info and info["vector_count"] >= expected:
            return
        time.sleep(1)
    raise TimeoutError(f"{ns} never reached {expected} records")

Common mistakes

• Index dimension that does not match the embedding model. Decide the model first, then create the index with its exact dimension.
• Tenant isolation by filter alone. Use one namespace per tenant and decide the namespace from authentication, not from user input.
• Huge single upserts. Batch around 100 records per request.
• Reading scores the wrong way round. With the cosine metric, Pinecone’s score is a similarity: higher is better.
• Forgetting where the index lives. Measure end-to-end latency from your real users’ location.

Recap

• Pinecone is a managed vector database: you create an index with a fixed dimension and metric, upsert records (ID, values, metadata), and query over the internet.
• Serverless indexes charge for storage, writes and reads, and are eventually consistent, so new records appear after a short delay.
• Namespaces partition one index; every operation targets exactly one, which makes them the natural wall between tenants.
• Multitenancy can use an index per tenant, a namespace per tenant, or a shared filter; the filter approach is only as safe as every line of your code.
• Batch upserts, design IDs for deletion by prefix, keep keys on the server, and measure end-to-end latency, including region distance.

Back at Crumb, Priya has been reading about a third option. “Chroma on a laptop, Pinecone as a service. There’s also Milvus, which is what big companies run when they have billions of vectors. And there’s a question I want you to be able to answer for any project: which one should we use, and why?”

Go deeper

• Pinecone documentation
• OpenAI API documentation (embedding models and limits)
• OWASP Top 10 for LLM Applications (data leakage between users is on the list)