Pull-quote: “Pure vector retrieval is the most common production-grade RAG mistake. Pure BM25 is the second most common.”

Why this matters

A pattern repeats in every RAG project that goes wrong: someone embeds the corpus, runs vector search, and ships. The system works in demos and disappoints in production. The fix is a structural architecture change: hybrid retrieval.

The components

Query
  │
  ├──► Dense (vector)   — pgvector / Weaviate / Qdrant + an embedding model
  │
  ├──► Sparse (BM25)    — Postgres FTS / Elasticsearch / OpenSearch
  │
  ├──► Optional filters — date range, source, entity tags
  │
  └──► Merge (RRF or weighted) ──► Cross-encoder re-rank ──► Top-K
                                                                │
                                                                ▼
                                                Citation-grounded generation

Why each piece matters

• Vector is excellent at semantic similarity — finding documents that are about the same topic in different words. It is bad at named entities — exact terms, IDs, dates.
• BM25 is the opposite — excellent at named entities, weaker on semantic similarity.
• Filters — when the question is bounded (“just look at 2024 reports about Boeing 737”), filters dramatically reduce the candidate set before ranking.
• Merge — Reciprocal Rank Fusion (RRF) is a clean default. Weighted merges work with calibrated scores.
• Cross-encoder re-rank — sees the query and the candidate document together and scores them jointly. More expensive than bi-encoder vector search, but the precision improvement on the top-K is large enough to pay for itself.

What changes when you do this right

• Hallucination rate drops. The model has better evidence to ground in.
• Citation precision goes up. The cited documents actually support the claim.
• Edge cases (rare entity queries, exact-quote queries) work properly.
• Generation latency stays low because the model only sees the top-K (typically 6–10), not the top-100.

Common mistakes

• No re-ranker. Top-50 from vector + top-50 from BM25 with RRF is a starting point, but without a re-ranker the top-K still contains noise.
• No filtering. Filtering before retrieval is essentially free if your data is properly indexed.
• Skip evaluation. Without a golden Q&A dataset and grounding scoring, you have no way to compare retrieval architectures.

Closing

Pure vector retrieval is the most common production-grade RAG mistake. Hybrid retrieval — vector + sparse + filters + re-rank — is the boring, reliable, production answer. Every Zorost RAG system runs this architecture.

The post Hybrid Retrieval: Why Vector Alone Isn’t Enough appeared first on Zorost Intelligence | AI, Cloud & Data Experts.

Pull-quote: “RAG works in demos. RAG that works in production requires hybrid retrieval, a re-ranker, citation grounding, and an evaluation harness.”

Why this matters

Most RAG projects pilot well and disappoint in production. The pattern is the same: embed the corpus, run vector search, ship. Production-grade RAG requires more.

The production RAG architecture

                     ┌────────────────────┐
        Question ───►│  AI Gateway        │  ← key mgmt, routing, observability
                     └──────────┬─────────┘
                                ▼
        ┌────────────────────────────────────────────┐
        │                Retrieval                    │
        │  ┌────────────────┐  ┌────────────────┐   │
        │  │ Mosaic AI      │  │ BM25 (lexical) │   │
        │  │ Vector Search  │  │ on Delta SQL   │   │
        │  │ (Delta-synced) │  │                │   │
        │  └───────┬────────┘  └────────┬───────┘   │
        │          └──── merge (RRF) ───┘           │
        │                  │                          │
        │              cross-encoder                  │
        │              re-rank                        │
        └────────────────┬─────────────────────────────┘
                         ▼
              top-K (typically 6–10)
                         │
                         ▼
              Citation-grounded generation
              (Mosaic AI Model Serving)
                         │
                         ▼
              Validated answer with source links

Why Mosaic AI Vector Search specifically

Mosaic AI Vector Search synchronizes with Delta tables. Update the source table, the index updates. No orchestration glue. Tagging, ACLs, and lineage flow through Unity Catalog. For RAG over enterprise data that changes, this matters more than people initially appreciate.

Hybrid retrieval is the pattern

Pure vector search is the most common production RAG mistake. Pure BM25 is the second most common. Hybrid — vector + BM25 + filters + re-rank — is the answer that actually works.

Citation grounding as a structural fix

Constrain the model to write with bracketed citation tokens. Validate every citation against the retrieval set. Reject answers that fail validation. This is a small structural change with a large operational impact.

Evaluation harness — non-negotiable

A production RAG system without an evaluation harness is a guess. The harness has three components:

1. Golden Q&A dataset — questions paired with the documents that should ground the answers
2. Grounding rate — what fraction of generated claims are supported by retrieved documents
3. Hallucination detection — flagging unsupported claims

The harness runs as a Databricks Job on every model or retrieval change. Regressions are caught before deployment.

Closing

Production RAG on the Lakehouse with Mosaic AI is straightforward when you adopt the architecture: hybrid retrieval, re-ranker, citation grounding, evaluation harness. The result is a RAG system analysts trust enough to use.

The post Production-Grade RAG on the Lakehouse with Mosaic AI Vector Search appeared first on Zorost Intelligence | AI, Cloud & Data Experts.

Pull-quote: “Vector search alone is not retrieval. It is one signal among several.”

Why this matters

Aviation safety knowledge sits in two enormous public-domain corpora: the U.S. NTSB accident reports and the NASA ASRS voluntary safety reports. Together, that’s 247,000+ documents of structured incident narratives. Pilots, controllers, and operations engineers have written them under the assumption that they would be searched, cross-referenced, and learned from.

Most platforms reduce this to keyword search. Better platforms add full-text search. The frontier is citation-grounded retrieval-augmented generation — the assistant retrieves, the model writes, every claim links back to the source documents.

Why hybrid retrieval

The naive approach to a RAG system is “embed everything and run a vector search.” It does not work in production. Vector search is excellent at finding semantically similar documents and bad at finding specifically named entities. BM25 is the opposite. Production retrieval needs both.

Our retrieval pipeline:

Question
   │
   ├──► dense (pgvector + BGE-large) ──► top 50
   ├──► sparse (BM25)                  ──► top 50
   │
   └──► merge + cross-encoder re-rank   ──► top 8
                            │
                            ▼
                Citation-grounded generation
                (Gemini 2.5 Flash for fast answers,
                 Claude / GPT for detailed analysis)

Why a re-ranker

The re-ranker (a cross-encoder, not a bi-encoder) sees the query and the candidate document together and scores them jointly. This is more expensive per call than vector search, but the precision improvement on the top-8 is large enough that it pays for itself — fewer retrievals, fewer hallucinations, better answers.

Why citation grounding

The default mode of an LLM is to fabricate plausible-sounding answers. The fix is structural: the model is constrained to write its answer with bracketed citation tokens, and the citation tokens must reference documents that actually exist in the retrieval set. Generation is post-processed to validate the citations and reject any answer that fails validation.

This is a small structural change with a large operational impact. It moves the system from “talking to a model that has ingested aviation knowledge” to “asking a model to summarize specific source documents.”

What this is good at

• “What are the leading causes of runway incursions for regional jets in low-visibility conditions?”
• “Show me ASRS reports that match the pattern of sudden hydraulic failure during flap retraction.”
• “What are the recurring training gaps that show up in cargo operations CRM reports?”

What it is not good at: real-time operational queries that need current schedule data — those go to the predictive and causal layers.

Closing

Vector search alone is not retrieval. It is one signal. Production-grade RAG over a regulated safety corpus requires hybrid retrieval, a real re-ranker, and structural citation grounding. The result is an assistant analysts trust enough to use — which is the only metric that matters.

The post A Retrieval Engine over the World’s Aviation Safety Corpus appeared first on Zorost Intelligence | AI, Cloud & Data Experts.

Pull-quote: “A review that is six months out of date is not a review. It is a historical artifact.”

Why this matters

The fundamental flaw of the traditional systematic review is that it is a snapshot. A team works on it for six months, freezes the literature search at a date, and publishes a result that becomes outdated the moment the next paper appears. In rapidly evolving fields — oncology, infectious disease, AI/ML methodology, certain rare-disease indications — that lag is unacceptable.

The fix is a living systematic review — a review that is continuously refreshed as new evidence appears.

What “living” actually requires

Living reviews are not just “running the search again every quarter.” They require:

1. Protocol stability — the inclusion / exclusion criteria do not change between updates
2. Federated search at scheduled cadence across the full database set
3. Delta detection — what’s new since the last update
4. Consistent screening — the same multi-agent consensus applied to new papers
5. Risk-of-bias and GRADE re-assessment — if a new high-quality study changes the certainty of evidence, that needs to surface
6. Versioned reporting — each refresh produces a versioned report with a clear changelog
7. Subscriber notification — stakeholders are alerted when something material changes

This is not a research methodology improvement. It is an engineering problem: how to do high-rigor evidence synthesis on a recurring schedule, with reproducibility and auditability preserved.

Architecture

EvidAI’s living review architecture:

Protocol (versioned) ──► Federated search (11 databases, scheduled)
                                          │
                                          ▼
                              Delta detection
                                          │
                          New papers since last refresh
                                          │
                                          ▼
                       Multi-agent consensus screening
                                          │
                          Included papers (new)
                                          │
                                          ▼
                  Risk-of-bias (RoB 2 / ROBINS-I / NOS)
                                          │
                                          ▼
                  GRADE re-assessment per outcome
                                          │
                                          ▼
                  Living report (versioned, with changelog)
                                          │
                                          ▼
                  Subscriber notifications

What changes for the team

The team’s role shifts from “run a six-month review every two years” to “monitor a continuously updated review and adjudicate the small fraction of decisions the AI escalated.” That is a fundamentally different work pattern, and it scales.

Closing

A review that is six months out of date is not a review. Living reviews are an engineering solution to a research methodology problem — and they are now operationally feasible.

The post Living Systematic Reviews: Evidence That Stays Current appeared first on Zorost Intelligence | AI, Cloud & Data Experts.

Pull-quote: “An SPC chart is not a decision. The decision is what to do about it. That’s where agents earn their keep.”

Why this matters

Quality management in regulated manufacturing has been essentially the same shape for thirty years: a set of forms (FMEA, Control Plans, MSA, NCR/CAPA, 8D), a statistical engine (control charts, capability indices, Gage R&R), and an audit trail. The forms get filled out, the charts get run, the audits pass. Operations engineers spend more time documenting than analyzing.

A multi-agent QMS is structurally different. It is not a forms engine with AI bolted on. It is an engine of cooperating agents that observe data, run analysis, recommend actions, and document what they did.

The agent architecture (eight specialized agents)

SPCio (co-developed with a manufacturing intelligence partner) ships eight specialized quality agents:

1. Process Monitor — watches SPC charts and triggers analysis on out-of-control patterns
2. Capability Analyst — runs Cp/Cpk/Pp/Ppk and interprets results in context
3. MSA Engineer — runs Gage R&R, ANOVA, and bias studies
4. FMEA Author — drafts and updates Failure Mode and Effects Analysis with severity / occurrence / detection scoring
5. Control Plan Author — drafts and updates Control Plans tied to FMEA and PPAP
6. 8D Investigator — runs Eight Disciplines problem-solving with root-cause analysis
7. NCR / CAPA Coordinator — manages non-conformance reports and corrective/preventive actions through closure
8. APQP Coordinator — orchestrates Advanced Product Quality Planning across phase gates

Each agent has a typed tool contract (inputs, outputs, side effects) and a deterministic call log. Agent-to-agent communication is mediated and recorded.

Why the architecture works

The classical QMS treats every form as an isolated artifact. The multi-agent QMS treats them as nodes in a graph: an FMEA refers to a Control Plan, which refers to an MSA, which refers to historical SPC data, which refers to the current production run. When an out-of-control pattern emerges on a chart, the Process Monitor doesn’t just raise an alert — it asks the Capability Analyst whether the process is still capable, asks the FMEA Author whether the relevant failure mode is documented, and asks the 8D Investigator to start a structured investigation if the pattern persists.

The result is a system that continuously maintains the QMS rather than waiting for the team to maintain it during audit prep.

Tool counts and the RAG corpus

SPCio’s eight agents share a tool catalog of fifty-seven callable tools ranging from statistical computations to chart generation to FMEA cross-referencing to PPAP documentation. The RAG layer is built over a 765,000-chunk quality knowledge corpus covering IATF 16949, ISO 9001, AIAG core tools, and customer-specific quality manuals.

Closing

A multi-agent QMS is not a UI improvement on the old model. It is a different model. The implication for quality engineers is significant: less time documenting, more time analyzing — and a continuously updated system that audits don’t catch up to, because it never falls behind.

The post Multi-Agent Quality: a New Architecture for the QMS appeared first on Zorost Intelligence | AI, Cloud & Data Experts.