ChromaDB

Overview

ChromaDB is an open-source vector database for storing, searching, and managing embeddings. It provides a simple API for document ingestion, semantic similarity search, and metadata filtering, supporting both Python and JavaScript/TypeScript clients with embedded, server, and cloud deployment options.

Instructions

• Install with pip install chromadb (checked against 1.5.9, October 2026) or npm install chromadb for JavaScript.
• Initialize with get_or_create_collection for idempotent setup. Use PersistentClient(path="./chroma_data") for local work, HttpClient(host="localhost", port=8000) against a server (chroma run --path ./chroma_data, or the chromadb/chroma Docker image, which keeps data in /data), and CloudClient for Chroma Cloud. list_collections() returns Collection objects since 1.0.
• Adding documents: batch add()/upsert() calls below client.get_max_batch_size() (5,461 in the 1.5 line, so 5,000 per call is safe), always store source metadata (filename, URL, page) for citations, and use upsert() for re-ingestion.
• Embeddings: with no embedding_function, Chroma downloads a small ONNX copy of all-MiniLM-L6-v2 on first use (about 80 MB) and runs it locally. Pass an OpenAI, Cohere or other embedding function for production, or pass your own embeddings= vectors and set embedding_function=None. Query with the same function you ingested with: it is stored with the collection configuration.
• Querying: collection.query(query_texts=[...], n_results=5, where={...}, where_document={"$contains": "refund"}). Results are lists of lists, one per query. Metadata operators: $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, combined with $and/$or. Use get() to fetch by ids or filter without similarity.
• Distance and index tuning go in configuration={"hnsw": {"space": "cosine", "ef_construction": 200, "max_neighbors": 16, "ef_search": 100}} at creation (space is l2 by default; cosine and ip also exist). The old metadata={"hnsw:space": ...} style is replaced by configuration. The space cannot be changed after creation.
• Returned distances are distances, lower is closer; for cosine, similarity is 1 - distance.

A minimal local server and client check:

chroma run --path ./chroma_data --host 127.0.0.1 --port 8000

import chromadb
client = chromadb.HttpClient(host="127.0.0.1", port=8000)
print(client.heartbeat())   # nanosecond timestamp when the server is up

Examples

Example 1: Build a document Q&A pipeline

User request: "Set up a RAG pipeline with ChromaDB for answering questions about our docs"

import chromadb
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction

client = chromadb.PersistentClient(path="./chroma_data")
docs = client.get_or_create_collection(
    "product-docs",
    embedding_function=OpenAIEmbeddingFunction(model_name="text-embedding-3-small"),  # reads OPENAI_API_KEY
    configuration={"hnsw": {"space": "cosine"}},
)
docs.upsert(
    ids=["billing-p1-c0", "billing-p1-c1"],
    documents=["Invoices are issued on the 1st of each month.", "Refunds take 5 business days."],
    metadatas=[{"source": "billing.pdf", "page": 1}, {"source": "billing.pdf", "page": 1}],
)
hits = docs.query(query_texts=["How long do refunds take?"], n_results=3)
print(hits["documents"][0][0], hits["metadatas"][0][0])

Pass the retrieved chunks and their source/page to the LLM as context and cite them.

Example 2: Filtered semantic search

User request: "Implement product search that combines text similarity with category filters"

products = client.get_or_create_collection("products")
res = products.query(
    query_texts=["noise cancelling headphones"],
    n_results=5,
    where={"$and": [{"category": "electronics"}, {"price": {"$gte": 50}}, {"price": {"$lte": 300}}]},
)
for pid, dist in zip(res["ids"][0], res["distances"][0]):
    print(pid, round(dist, 3))

The output is up to five product ids with distances, smallest first, all inside the category and price range.

Guidelines

• Use get_or_create_collection so restarts are safe; it ignores a changed configuration for an existing collection.
• Collection names must be 3-512 characters from [a-zA-Z0-9._-], starting and ending with a letter or digit; a name like x raises InvalidArgumentError.
• Keep ids stable (for example file-page-chunk) so upsert() replaces instead of duplicating.
• Store source metadata for every chunk; metadata values must be strings, numbers or booleans, not nested objects.
• Pick n_results from the LLM context budget: 5-10 for most RAG pipelines.
• Use cosine for text embeddings from OpenAI or Cohere models; the default l2 ranks unnormalized vectors differently.
• Built-in server authentication was removed in 1.0: put the server behind a reverse proxy with auth, or use Chroma Cloud. Never expose port 8000 to the internet.
• Chroma suits prototypes through mid-size collections on one node; for very large or multi-writer workloads evaluate a dedicated vector service.