Second Brain OS

Status: v0, in progress. This is a thin vertical slice that exercises every core piece of the intended architecture end to end — hybrid retrieval, LLM-driven graph extraction, and a traced reasoning pipeline — rather than the long-term vision. Upload a PDF and query it; that path works today.

The pitch for most "AI assistant" products is a chatbot with your documents bolted on. The problem is that a model answering from generic web knowledge can't tell you what to learn next, because it doesn't know what you already know. Ask it what to focus on for a senior AI role and it will describe a generic candidate, not you.

Second Brain OS is the opposite arrangement: a local-first system that builds a knowledge graph out of documents you feed it — CV, notes, project write-ups — and answers strictly from that. It's aimed at being something that accumulates over years rather than a session that forgets you.

Ingestion: chunking and graph extraction in one pass

Uploading a PDF (POST /doc/upload) runs a pipeline where the interesting decision is doing two jobs at once:

1. The original file goes to MinIO as-is.
2. Text is extracted with PyMuPDF and handed to Gemini, which splits it into semantically coherent chunks and extracts knowledge-graph relationships from each chunk in the same pass, both as schema-validated structured output.
3. Each chunk becomes a Chunk node in Neo4j; entities mentioned in it link via MENTIONED_IN, and extracted relationships become edges between entity nodes.
4. Chunk text is embedded twice — dense (Ollama nomic-embed-text, for semantic similarity) and sparse (FastEmbed BM25, for exact keyword matches) — and upserted into Qdrant as a hybrid collection.

Combining chunking and extraction into a single LLM call matters because the model is already reading the text to decide where the semantic boundaries are. Splitting first and extracting second pays for that comprehension twice and throws away the context the boundaries were chosen from.

The graph schema is deliberately small: User, Skill, Project, Goal, Note, Chunk, connected by KNOWS, LEARNING, REQUIRES, RELATED_TO, and MENTIONED_IN.

Retrieval: vectors find the door, the graph walks through it

A query (POST /query) runs two things in parallel:

• a hybrid search against Qdrant — dense and sparse, fused with RRF
• a local LLM call that picks which relationship types, if any, are relevant to this particular question

The retrieved chunk IDs then anchor a Neo4j lookup: find entities mentioned in those chunks, expand 1–2 hops along only the relationship types the model picked. If nothing expands — no linked entities, or no relevant relation types — it falls back to the chunk's own stored content rather than returning nothing.

That fallback is the part worth pointing at. Graph retrieval degrades badly when the graph is sparse, which it always is early on, and a system that returns an empty answer because a traversal found no edges is worse than one that quietly behaves like ordinary RAG.

A local Ollama model then answers using only that graph-derived context. The whole retrieve → reason → respond flow is a three-node LangGraph graph, traced in LangSmith, so a bad answer can be attributed to a stage rather than guessed at.

Local-first, with one honest exception

Everything that touches your documents at query time runs on your machine: Ollama for embeddings and reasoning, Neo4j, Qdrant and MinIO in Docker Compose. Gemini is the one remote call, used for chunking and graph extraction at ingest.

There's also a known rough edge, documented rather than hidden: the docker-compose.yml defines an app service, but you should run the API on the host for now — a containerised app can't reach Ollama on localhost:11434 without extra Docker networking that isn't wired up yet.

Structure

Each top-level directory is an independent package, Django-app style, with plain absolute imports across them (from core.settings import settings) rather than relative-import gymnastics:

main.py         FastAPI entrypoint
core/           settings, prompts, shared graph + embedding logic
database/       Neo4j / Qdrant / Postgres / Redis clients
storage/        MinIO client + file parsing
ingestion/      upload pipeline: chunking, graph writes, vector upserts
retrieval/      query pipeline: hybrid search, relation picking, graph reads
pipeline/       the LangGraph state machine
routers/        FastAPI routes

Postgres and Redis are already stood up in Compose for relational data and short-term memory, but aren't wired into the v0 path yet.