RAG Platform

Retrieval-augmented generation is easy to demonstrate and hard to trust. A demo answers three questions well; a corpus of real documents produces answers nobody can check, from passages nobody can see, at a cost nobody has counted.

So the platform was built around the parts a demo skips: where a citation actually came from, what a change to the pipeline did to the answers, and what the whole thing costs per question.

Seven formats - PDF, DOCX, XLSX, HTML, Markdown, JSON, plain text - and each loader returns the structure the format actually declares rather than a wall of text. A PDF has no headings, only type, so headings are read from the type size the document itself uses. Word has heading styles. HTML has h1, and main beats page chrome.

Chunks are then packed to a token budget instead of cut at every boundary, because a heading says the subject changed, not that there is enough text to answer a question. Every chunk carries the page and heading path it can honestly claim, which is what lets a citation name a page instead of an ordinal.

pgvector for meaning and Postgres full-text for exact terms, fused by reciprocal rank. Each document is indexed in its own language, detected at ingest, so Russian stemming is applied to Russian and English to English.

The query is built to match any of its meaningful words with the stopwords of every configured language stripped first - joining them all with AND, which is what the naive Postgres helper does, matches nothing at all on a real question. A multilingual cross-encoder reranker ships inside the image and runs on CPU, so the first question is not also the first model download.

A retrieval pipeline is a JSON document of nodes and edges that the runtime executes and a React Flow canvas edits. Seven node types - dense retrieval, lexical retrieval, fusion, rerank, context builder, generate, agent - and each one generates its configuration form from its own JSON Schema, so the form and the validation come from a single declaration.

Versions are immutable snapshots with labels; one is current, and that is what bots serve. A draft can be run against a real bot before it is saved, and a graph whose pieces are not joined up is refused rather than quietly answering with no context at all.

A golden set is questions whose answers are already known, with the document each should come from. It runs through the real pipeline - the same executor a chat turn uses - and scores seven metrics: four deterministic and free, three judged by a model and marked as costing money.

A run can be pinned to a specific pipeline version, so two graphs are compared on the same questions with the same bot, and any two runs can be diffed with the regressions named. On live traffic, an A/B experiment splits conversations between the current version and a candidate, sticky per person, with turns, tokens and spend per arm coming from the same usage events billing reads.

Langfuse knows what the model saw, answered and cost. Jaeger knows where the time went inside one request, including the SQL and HTTP that Langfuse never sees. Prometheus is the only one that answers questions about no particular request - queue depth, error rate this hour, p95 last Tuesday. Grafana is what that looks like, and when somebody should be woken up.

Per-organization spend is deliberately not a metric: labels never carry tenant identity, because a time series per customer is how a metrics backend dies. That question is answered from the usage table in Postgres, and the reasoning is in an ADR.