Module 08
Vector Databases
When you have millions of embeddings, comparing a query with every one of them is too slow and too memory-hungry, so vector databases organise vectors into indexes and compress them, trading a little recall for a lot of speed. You will learn how flat, IVF and HNSW indexes and quantisation work, then use Chroma, Pinecone, Milvus and pgvector end to end, and finish by building a Pinecone-backed chatbot that answers questions about YouTube videos with clickable timestamp citations.
By the end of this module you can
- explain flat, IVF and HNSW indexes, scalar and product quantisation, and measure the recall-versus-speed trade-off
- build, filter, update and cost a Chroma collection, and keep it in sync with changing documents
- run Pinecone end to end, from index creation to batched upserts and queries, and isolate customers with namespaces
- choose between Chroma, Pinecone, Milvus and pgvector for a workload, with reasons tied to size, operations, filtering and cost
- store and search embeddings inside PostgreSQL with pgvector, including filters, joins and index tuning
- ship a grounded RAG chatbot over video transcripts that cites the exact moment each claim came from
Lessons
6 in this module- 01How vector databases work: indexing and compressionExplain how flat, IVF and HNSW indexes and scalar and product quantisation work, and measure the recall-versus-speed trade-off on real vectors.
- 02ChromaDB end to endBuild a searchable handbook in Chroma with chunking, metadata filters, full CRUD, your own embeddings and a cost estimate, and keep it correct as documents change.
- 03Pinecone end to endCreate a Pinecone serverless index, batch-upsert and query records, and use namespaces to keep each customer’s data walled off in a multi-tenant app.
- 04Milvus and choosing a vector databaseUse Milvus from a local file to a production schema, and choose between Chroma, Pinecone, Milvus and pgvector by working through size, operations, data location, filtering and full cost.
- 05pgvector: vectors in the database you already haveAdd embeddings to existing PostgreSQL tables with pgvector, search them with SQL filters and joins, build and verify HNSW indexes, and fix filtered searches that return too few rows.
- 06Project: RAG chatbot over YouTube transcriptsShip a Pinecone-backed chatbot that answers questions about YouTube videos using only what was said, with clickable citations that open each video at the right second.