Postgres + pgvector rather than a separate vector DB
Relational matter scoping, lexical search, vector search, citations, audit records, and application data can remain inside one transactional system for this scale.
Trace
Document AI built around evidence, with hybrid retrieval and source-validated citations.
overview / problem
Document AI becomes unreliable when retrieval, source isolation, and citations are treated as afterthoughts. Trace was designed around a stricter flow: ingest → parse → chunk → embed → retrieve → assemble evidence → generate → validate.
what I built
Multi-format ingestion, matter-scoped documents, PostgreSQL full-text search, pgvector similarity, reciprocal-rank fusion, asynchronous ingestion, and source-document viewer links.
Grounded Q&A, chronology workflows, entity workflows, contradiction analysis, issue memo generation, citation validation, audit trails, and an eval harness.
architecture
Upload → MinIO → Async worker → Parsing / chunking → PostgreSQL + pgvector → Lexical + semantic retrieval → RRF → Evidence assembly → Model → Citation validation → Grounded output
interesting decisions
Hybrid retrieval rather than vectors alone
Legal and document evidence depends on names, dates, IDs, clauses, exact language, and paraphrases, so lexical and semantic paths fail differently.
Reciprocal-rank fusion
Lexical and semantic rankings are fused rather than trusting one retrieval mode to identify the right evidence every time.
Deterministic citation validation
A model may propose an answer, but source validation decides whether evidence can be trusted before the output is surfaced.
verification
The public repository includes API regression coverage across authentication, ingestion, retrieval, viewer behavior, AI workflows, exports, evals, and semantic retrieval paths.
limitations
- Portfolio project.
- Production-style architecture, not a claim of production deployment.
- Local-first default environment.
- Stronger embeddings and reranking would be natural future improvements.