The foundational search and retrieval engine for the entire ecosystem. BGE-M3 embeddings (dense + sparse + ColBERT), PostgreSQL with pgvector, hybrid search with Reciprocal Rank Fusion, adaptive semantic chunking, and a Dreaming module that periodically reorganizes knowledge — building cross-references, detecting contradictions, and strengthening semantic links. Open source, MIT license.
- Status
- Live
- Autonomy (AAL)
- L1 → L3
- Stack
- PostgreSQL + pgvector
- Licence
- Open source, MIT
01
Three embeddings in one
BGE-M3 trinity embeddings: dense, sparse and ColBERT.
02
Hybrid search
Reciprocal Rank Fusion merges the rankings, and adaptive semantic chunking avoids naive fixed-size cuts.
03
An index that tidies itself
The Dreaming module self-organizes the index over time. 1148+ chunks are indexed in production.
Capabilities
Working now
- BGE-M3 trinity embeddings (dense + sparse + ColBERT) on port 8300
- Hybrid search with Reciprocal Rank Fusion (port 8310)
- Adaptive semantic chunking (no naive fixed-size cuts)
- Dreaming module for index self-organization
- PostgreSQL + pgvector storage
- MIT-licensed, public repo (Arcanada-one/scrutator)
- 1148+ production chunks across the ecosystem knowledge base
Current autonomy level
Weakest link
BGE-M3 worker health is not externally monitored; if the embedding service is down, search returns empty and consumers see no error. Index drift between PG vectors and source documents is not auto-detected.
Roadmap to L3
- L3 lift — /health endpoint per worker, fallback to a lighter embedding model on outage, structured pino traces.
- L3 polish — drift detection job comparing PG vector count vs source manifest; classified errors propagated to consumers.
- Verification gate — kill embedding worker mid-query and assert classified 503 + Ops Bot fatal event within 5 s.
Next step
Explore the available product and its current limits
LIVE means the project is available now; the weakest link and roadmap above state the current version’s limits.
Review the path to the target level