Build Your Own Institutional Reasoning Engine
A complete guide for non-technical founders — every tool, subscription, workflow, and flowchart needed to build and deploy this architecture. Plain English throughout.
This Builder Guide is the execution document. It answers: "Can we actually build this, and how?" It is designed for developers, technical co-founders, and implementation teams.
The companion IRE Whitepaper (IRE_v1.2_Final.pdf) is the strategic and credibility document. It defines the category, establishes the architectural foundation, and answers: "Should this exist, and is it credible?" It is designed for investors, CXOs, regulators, and senior advisors.
Use the whitepaper to establish why. Use this guide to establish how.
What Is an Institutional Reasoning Engine?
An IRE is not a chatbot. Not automation. It is an AI system that runs a structured investigation workflow — forming a hypothesis, iterating through evidence, and requiring human approval at every finding before it enters the record.
A professional investigation workflow encoded in software — AI accelerates the analysis, humans remain accountable for every conclusion.
The Three Things That Make It Different
🔒 Methodology as Engineering
Investigation rules are baked into the system — not left to human discretion. The AI cannot skip a step any more than a calculator can divide by zero.
👤 Human Gate at Every Finding
No AI output enters the official record without a named investigator approving it. The approval IS the record — not a step before the record.
🔗 Tamper-Evident Audit Chain
Every action is cryptographically linked. Alter one log entry and every subsequent entry breaks. Mathematically detectable.
What It Is vs What It Is Not
| This IS | This IS NOT |
|---|---|
| A methodology encoding framework — process as engineering constraints | A replacement for investigator judgment |
| Modular — scales from 8 base components to 35 | A product you can buy off the shelf |
| Applicable to any regulated investigation, any industry | A compliance guarantee in your jurisdiction |
| AI-accelerated with mandatory human gates | An autonomous AI that makes decisions independently |
| Tamper-evident by default; independently verifiable with Cluster G active | Unconditionally immutable in all configurations |
| Upgradeable incrementally — start small, grow | A system requiring full build before delivering value |
Architecture at a Glance
Two layers: a mandatory base every deployment must include, and modular clusters that activate based on your investigation context.
The 8 Mandatory Base Components
| # | Component | Plain English | Why Non-Negotiable |
|---|---|---|---|
| B1 | Case-Scoped Isolation | Each case gets its own sealed container. Evidence from Case A cannot leak into Case B — by design. | Cross-contamination makes findings legally indefensible. |
| B2 | Retrieval Verifier | Before any AI claim reaches an investigator, the system checks it exists in the actual case documents. A database lookup — not AI judgment. | Prevents hallucinated findings entering the record. |
| B3 | Evidence Grounder | Every verified claim gets a citation — document name, page number, chunk ID. Like footnotes, automatic and tamper-evident. | Regulators and courts demand traceable findings. |
| B4 | Human Review Gate | No AI finding enters the official record without a named investigator approving it. The approval event is itself logged. | Accountability. A human is responsible for every finding. |
| B5 | Audit Event Logger | Every action logged: what was queried, retrieved, which AI model used, who approved what, when. | You cannot defend a record you cannot reconstruct. |
| B6 | Hash-Chain Immutability | Each log entry cryptographically linked to the previous. Altering any entry breaks every entry after it. | Proves the record was not tampered with after the fact. |
| B7 | Model Version Pinning | Every case stores exactly which AI model version was used. Mid-investigation upgrades are blocked. | Reproducibility — you must re-run the same reasoning months later. |
| B8 | Algorithmic Bias Monitor | Monitors whether the AI is systematically over-flagging any demographic group. Active from day one. | Biased AI in regulated investigation is a liability, not a quirk. |
The 7 Modular Clusters
Documentary Evidence
Turns any document pile into searchable evidence. PDFs, spreadsheets, emails — queryable by meaning.
Behavioural & Interview
Detects contradictions between interview statements and documentary evidence. Cites both sides automatically.
Entity & Network Intelligence
Maps who is connected to whom. Surfaces circular flows and fraud rings invisible from individual documents.
Advanced Reasoning
The recursive reasoning engine. Forms hypotheses, tests them, revises them, repeats until confident or flagged.
Privacy Gateway
Replaces real identifiers with tokens before anything reaches external AI. De-tokenises only on approval.
Institutional Memory
Fine-tunes the AI on your own completed cases. The system learns your fraud typologies over time.
Integrity Elevation
Daily external anchoring of the audit chain. Elevates from institutional-grade to independently verifiable audit integrity.
Prerequisites
Three things must be true before you write a single line of code. The system encodes your methodology — if it is not documented, the build will fail at the hardest moment.
Most teams skip prerequisites and then wonder why the AI produces inconsistent outputs. The IRE encodes your investigation methodology — if that methodology is not written down, there is nothing to encode.
Document Your Investigation Methodology
Write down exactly how investigations work today: how a case is opened, what evidence types you collect, what constitutes a finding, what your output documents look like (STR, HLIR, case closure memo). This becomes the specification the system is built against. Use Claude to help draft this — describe your process and ask it to structure it as a methodology document.
Complete a Data Inventory
List every type of evidence you routinely handle: bank statement CSVs, KYC PDFs, email exports, interview recordings, transaction logs. For each type note: format, where stored, what personal data it contains. This determines which clusters you need and shapes your Cluster E design.
Get Legal & Compliance Sign-Off
Before AI touches investigation data, your legal team must sign off on: (a) human review gate requirements in your jurisdiction, (b) whether pseudonymisation is sufficient for your data protection obligations, (c) whether AI-generated outputs are admissible in your regulatory proceedings.
Technical Prerequisites
| Prerequisite | What You Need | If You Don't Have It |
|---|---|---|
| Server / Cloud Environment | Linux server or cloud VM with at least 32GB RAM for local AI models | AWS EC2 g4dn.2xlarge (~$0.75/hr, ₹63/hr). Spin up only when needed. |
| Python 3.10+ | Entire stack is Python-based. AI tools write the code — server just needs Python. | Any Ubuntu 22.04 LTS VM includes Python. |
| Docker | Most components (Qdrant, Neo4j, n8n) run as Docker containers. | Docker Desktop for dev, Docker Engine for production. |
| Git & GitHub | Version control. Enables AI coding tools most effectively. | Free. github.com. 5 commands is all you need. |
Skills You Need
With Claude Code or Cursor, you can build the majority of this system with 1 developer. The architecture is well-defined — the hard part is design, not code.
You — Domain Expert
Investigation methodology, compliance requirements, user stories, QA of outputs. Cannot be outsourced.
1 Python Developer
Builds the pipeline, integrates tools, writes orchestration code. AI tools handle 60–70% of actual code.
Legal Counsel (Part-Time)
Reviews AI-in-investigation policies, data processing agreements, admissibility. One engagement to set up.
Tools & Subscriptions
Every tool needed to build the full IRE stack. Organised by component, with free and paid options at each layer.
Vector Database — Case-Scoped Evidence Store
Stores all case documents as mathematical representations so they can be searched by meaning. One namespace per case = isolation by architecture.
| Tool | Type | Cost | Best For |
|---|---|---|---|
| Qdrant | Open Source | Free self-hosted / $25/mo cloud | Most teams — best balance. Our default recommendation. |
| ChromaDB | Open Source | Free | Development / prototyping |
| Weaviate | Open Source | Free / $25+/mo | Larger teams needing more query features |
| Pinecone | Cloud SaaS | $70+/mo | Teams avoiding infrastructure — but data leaves environment |
Graph Database — Entity & Network Intelligence
| Tool | Type | Cost | Notes |
|---|---|---|---|
| Neo4j Community | Open Source | Free | Standard choice. Mature Cypher query language. Self-hosted = data on-premise. |
| Memgraph | Open Source | Free | Faster for real-time streaming graph updates. |
| Neo4j AuraDB | Cloud SaaS | $65+/mo | Managed cloud. Check data residency before use. |
Local AI Models — On-Premise Reasoning
Investigation data is sensitive. The majority of AI reasoning should happen on local models — data never leaves your environment. External AI is used only for final narrative generation, always pseudonymised first.
| Tool | What It Does | Cost | Models | Hardware |
|---|---|---|---|---|
| Ollama | Runs local LLMs. Simplest setup available. | Free | Llama 3.1 70B (Agent), Llama 3.1 8B (Orchestration) | 70B: 40GB+ GPU RAM. 8B: 8GB GPU or fast CPU. |
| vLLM | High-performance serving for production | Free | Any HuggingFace model | NVIDIA GPU required |
| BGE-M3 Embedding | Converts text to vectors for semantic search | Free | via Ollama | Runs on CPU — no GPU needed |
External AI APIs — SOTA Reasoning (Pseudonymised Only)
| Provider | Models | Cost / M tokens | INR / M tokens |
|---|---|---|---|
| Anthropic | Claude Sonnet / Opus | $3–$15 | ₹250–₹1,250 |
| OpenAI | GPT-4o / GPT-4 Turbo | $5–$10 | ₹420–₹840 |
| Gemini 1.5 Pro | $3.50 | ₹295 | |
| Sarvam AI | Indian languages | ~$0.50 | ₹42 |
Pipeline Orchestration
| Tool | What It Does | Cost | Notes |
|---|---|---|---|
| n8n | Visual workflow builder. Connects components without writing orchestration code from scratch. | Free self-hosted | Best for non-technical teams. |
| LangGraph | Python framework for agentic reasoning loops (the recursive reasoning in Cluster D). | Free | Best option for the Planner-Analyzer-Critic loop. |
| LangChain | Python AI pipeline framework. More code, more control. | Free | Developer-facing. |
PII Detection & Pseudonymisation
| Tool | What It Does | Cost |
|---|---|---|
| Microsoft Presidio | Open-source PII detection + anonymisation. Supports custom recognisers for investigation-specific entities. 100% on-premise. | Free |
| spaCy NER | Named entity recognition. Use as a component within Presidio. | Free |
Document Processing & OCR
| Tool | What It Does | Cost |
|---|---|---|
| PyMuPDF (fitz) | Fast PDF text extraction. First choice for text-based PDFs. | Free |
| Tesseract OCR | Extracts text from scanned PDFs and images. | Free |
| unstructured.io | Handles mixed document types (PDF, DOCX, XLSX, email) with one API. | Free open source |
| AWS Textract | Managed OCR with table extraction. Best for complex bank statement layouts. | $1.50 / 1,000 pages (₹126) |
Speech-to-Text — Interview Transcripts
| Tool | What It Does | Cost |
|---|---|---|
| Whisper (Faster-Whisper) | Local speech-to-text. Runs on-premise — audio never leaves your environment. Best for sensitive recordings. | Free |
| Sarvam AI STT | Indian language speech-to-text. | ~₹1/min |
| Deepgram | Cloud STT with speaker diarisation. Only for non-sensitive recordings. | $0.0043/min (₹0.36/min) |
Audit Chain & Integrity
| Tool | What It Does | Cost |
|---|---|---|
| PostgreSQL | Stores the hash-chain audit log. Append-only table. Self-hosted. | Free |
| AWS QLDB | External notarisation ledger (Cluster G only). | ~$5–20/mo |
| Azure Confidential Ledger | Alternative to QLDB for Azure stacks. | ~$10–25/mo |
Cost Breakdown
Detailed cost figures available on request. Contact for pricing information.
The majority of the stack is free and open source. Costs scale with deployment profile and case volume. Contact us for a detailed cost model tailored to your organisation.
One-Time Build Costs
| Item | USD | INR |
|---|---|---|
| Developer time — Profile 3 MVP | **** | **** |
| Legal review — AI-in-investigation policies | **** | **** |
| Security review | **** | **** |
| Domain + SSL (optional web interface) | **** | **** |
Monthly Running Costs by Profile
| Profile | Cases/Month | USD/Month | INR/Month | Main Cost Driver |
|---|---|---|---|---|
| 1 — Minimal | **** | **** | **** | Cloud VM only. No AI API cost. |
| 2 — Interview-Led | **** | **** | **** | VM + Whisper STT compute. |
| 3 — Document Investigation | **** | **** | **** | VM + optional SOTA API. |
| 4 — Corporate Fraud | **** | **** | **** | VM with 70B GPU model + Neo4j. |
| 5 — Financial Crime | **** | **** | **** | GPU compute + SOTA API. |
| 6 — Enterprise Forensic | **** | **** | **** | Full stack + fine-tuning + notarisation. |
Use spot/preemptible instances for GPU compute — significantly cheaper. Only run the large model when needed — use the smaller model for orchestration. Batch SOTA API calls — one per section, not per claim. Cache embeddings — documents don't need re-embedding unless changed.
Build Phases
Each phase delivers standalone value — you do not need to finish everything before getting something useful.
Phase 1 — Mandatory Base (Weeks 1–6)
A working case management system with tamper-evident audit logging and human approval gates. Defensible output even with no AI reasoning yet.
What to Build
- Case creation and management (PostgreSQL + simple API)
- Case-scoped namespaces in Qdrant (B1)
- Audit event logger writing to PostgreSQL (B5)
- Hash-chain — SHA-256 chaining on every log entry (B6)
- Human review gate — approval UI writing to audit chain (B4)
- Model version pinning — store model_id + version with every AI call (B7)
- Bias monitoring dashboard — basic demographic flag rate tracker (B8)
Exit Criteria
- 10 synthetic cases run end-to-end with complete audit chains
- Hash chain verification confirms no tampering detectable
- Human review gate blocks record creation without approval event
- Zero cross-case data leakage confirmed by namespace isolation test
The Real Implementation Challenge: Data Messiness
AI reasoning failures are not your biggest early problem. Data quality is. Plan for this from day one — not as an afterthought when investigators complain that documents are "not showing up."
The Most Common Production Blockers
| Problem | What Happens | Fix |
|---|---|---|
| Scanned PDFs without OCR layer | PyMuPDF returns empty string. Ingestion pipeline produces empty chunks. Evidence silently missing from case. | Detect text layer presence before parsing. If empty → route to Tesseract automatically. Never fail silently. |
| Password-protected files | Parser throws cryptic error. File never ingests. Investigator doesn't know. | Pre-ingestion check: detect encryption before upload. Return clear error: "File is encrypted — please provide decrypted version." |
| Non-standard CSV encodings | pandas throws UnicodeDecodeError on GBK, Windows-1252, or ISO-8859-1 encoded bank statements. | Detect encoding with chardet before parsing any CSV. Normalise to UTF-8 at ingestion. |
| Inconsistent date formats | DD/MM/YYYY from one bank, MM/DD/YYYY from another, epoch timestamps from transaction logs. Chronology breaks. | Normalise all dates to ISO 8601 at ingestion. Log original format in metadata. Never assume format. |
| Garbled entity names from OCR | "Rajesh Kumar" becomes "R@jesh Kum4r" in a scanned KYC. Entity resolution fails. Graph node phantom. | OCR quality threshold: flag chunks where character confidence drops below 80%. Route to human review before entity extraction. |
| Inconsistent column headers across bank statement providers | "Amount" vs "Debit" vs "CR/DR" vs "Transaction Value" — same data, different labels across providers. | Build a column normalisation map per provider. Maintain it as a config file, not hardcoded logic. |
| Handwritten annotations on documents | OCR accuracy drops to 40–60% on handwriting. Worse on Indian scripts. | Flag handwritten content separately. Route to human transcription. Never auto-ingest without QC. |
Every case should have a rejected_documents log — every file that failed ingestion and exactly why. This serves two purposes: (1) the investigator knows their evidence is incomplete; (2) you have a systematic record of preprocessing failures to fix in the next sprint.
Phase 2 — Cluster A: Document Ingestion (Weeks 7–14)
Investigators can upload any document type and query it by meaning. Evidence retrieval under 1 second. Every claim cites its source and page.
- Document parser — PyMuPDF for PDFs, openpyxl for Excel, python-docx for Word
- Document-type-aware chunker — different strategies per file type
- Local embedding model — BGE-M3 via Ollama
- Qdrant indexer — writes to case-scoped namespace
- Retrieval Verifier — checks claims against namespace (B2)
- Evidence Grounder — attaches chunk_id + source + page (B3)
Phase 3 — Cluster B: Behavioural (Months 3–4)
Interview transcripts ingested and searchable. Contradiction detection live.
- Whisper STT — local transcription with speaker diarisation
- Transcript chunker — speaker-turn level with timestamp + speaker metadata
- Extended PII recognisers for Presidio — spoken-form names, informal references
- Contradiction Detector — cross-references transcript claims vs document corpus
- Transcript Approval Gate — separate HITL flag for statement-derived findings
Phase 4 — Clusters C + D: Graph + Reasoning (Months 4–6)
Entity network mapping live. Recursive reasoning loop operational. Evidence scoring running. Profile 4 capability achieved.
- Entity extractor — NER pipeline feeding resolution queue
- Three-tier entity resolution — Tier 1 auto, Tier 2/3 blocking HITL screens
- Neo4j graph builder — entities as nodes, transactions as weighted edges
- Graph query layer — circular flow, community detection, path analysis
- LangGraph reasoning loop — Planner → Analyzer → Verifier → Grounder → Critic → Score
- Calibrated evidence scorer — 4-component deterministic score
Phase 5 — Clusters E + G: Privacy + Integrity (Months 6–10)
Cluster E activates external AI APIs. Get legal review of pseudonymisation approach and data processing agreements first.
- Presidio pseudonymisation pipeline with custom recognisers
- Token map store — in-memory, encrypted at rest, session-scoped, never leaves environment
- De-tokenisation gate — restores identifiers only post-approval, on-premise
- External SOTA API integration — receives pseudonymised payload only
- G1 — AWS QLDB integration for daily audit chain anchoring
- G2 — Recording file hash → transcript linkage in audit chain
Phase 6 — Report Generation + Go Live (Month 10+)
Full draft report in 60–90 seconds. Every claim cited. Human review protocol live. New system is primary investigation record.
- Query template library — one RAG + graph query pair per report section
- Parallel section generator — all sections generated simultaneously
- Section review UI — finding + citation map side by side, edit + approve
- Citation Map appendix — auto-generated index of all findings to sources
- Final report lock — approved sections assembled, report hash in audit chain
Workflow: End-to-End Case Flow
How a case moves through the full IRE system — from opening to locked report.
Investigator Accountability Schema
Logging that a human approved a finding is necessary but not sufficient. For outputs to be genuinely audit-ready, the approval record must capture the quality and basis of the human judgment — not just the fact that it occurred.
| Field | Type | Required | Purpose |
|---|---|---|---|
investigator_id | UUID | Yes | Identity binding — who approved |
action | approve / edit_approve / reject / escalate | Yes | Nature of decision |
diff_hash | SHA-256 | Yes | Hash of any edits made before approval — proves what changed |
confidence_level | high / medium / low / uncertain | Yes | Investigator's stated confidence |
rationale | Free text, min 20 chars | If confidence = low | Forces articulation of doubt when confidence is low |
disagreement_flag | Boolean | Yes | True if investigator disagrees with AI finding but approves for procedural reasons |
disagreement_note | Free text | If flag=true | Documents the nature of disagreement for the record |
elapsed_review_time | Seconds | Yes | How long spent on this section — detects rubber-stamping |
An investigator approving 95%+ of findings without edits and with review times under 10 seconds per section is rubber-stamping — not reviewing. elapsed_review_time below a minimum threshold (suggest 30 seconds per section) triggers a quality review flag. This makes the human gate architecturally meaningful rather than ceremonial.
What Happens at Each Stage
| Stage | System Does | Investigator Does | Logged |
|---|---|---|---|
| Case Open | Creates isolated namespace, initialises hash chain | Names case, sets scope | case_open event |
| Evidence Upload | Parses, chunks, embeds, indexes all files | Uploads files, then waits (async) | ingestion event per file with file_hash |
| Entity Resolution | Proposes merges with evidence for/against | Approves/rejects Tier 2/3 decisions | resolution decision + evidence presented |
| Graph Build | Constructs entity network automatically | Reviews graph for obvious errors | graph_build event |
| Reasoning Loop | Runs 3–7 iterations of hypothesis → evidence → verify → score | Can interrupt or redirect at any time | hypothesis_update per iteration |
| Report Generation | Generates all sections in parallel with citations | Waits 60–90 seconds | report_generation event |
| Human Review | Presents finding + citation map per section | Reads, edits, approves each section | approval event per section with diff_hash |
| Case Close | Locks report, stores hash, appends Citation Map | Signs off final report | case_close event with report_hash |
Workflow: Document Ingestion
How a document goes from upload to queryable evidence in the case namespace.
Scanned PDFs without OCR layer: If PyMuPDF returns no text, auto-route to Tesseract. Password-protected files: Require investigator to decrypt before upload. Non-standard CSV encodings: Detect encoding with chardet before parsing.
Workflow: Entity Resolution
How the system determines "Rajesh Kumar", "R. Kumar", and "RJSH_KMR" are the same person — and what happens when it is not sure.
A 10% entity duplication rate in a 500-node graph produces 50 phantom nodes — enough to break circular flow detection entirely. Build Tier 2/3 resolution before building the graph layer.
Workflow: Recursive Reasoning Loop
How the AI forms a hypothesis, tests it against evidence, and iterates until confident or flagged. This is Cluster D — the reasoning engine.
Evidence Score Thresholds
0.80 — Section Lock
Individual report sections lock at this score. Still requires minimum 3 citations.
0.90 — Final Finding Lock
Final finding sections require this higher threshold.
High Score + Thin Evidence
Score high but fewer than 3 citations? System flags: "High confidence on thin evidence — manual review required." Never auto-lock.
Workflow: Privacy Gateway
How PII is removed before data reaches any external AI, and restored only after investigator approval — inside your environment.
Pseudonymisation controls what enters the AI — not what it infers. A model given pseudonymised data may produce output that, combined with other information, re-identifies a data subject. The Human Checkpoint is the primary control. Get legal counsel to review your approach before going live.
Workflow: Audit Chain
How every action is permanently recorded and cryptographically linked — making tampering mathematically detectable.
What Gets Logged
| Event | Key Fields | Why It Matters |
|---|---|---|
| File ingestion | file_name, file_hash, chunk_count, namespace_id | Proves exactly what evidence was in the case at what time |
| RAG retrieval | query_text, chunk_ids returned, timestamp | Shows what the AI was told, not just what it said |
| AI model call | model_id, model_version, temperature, input_hash, output_hash | Reproducibility — can re-run same call months later |
| Retrieval verification | claims verified, claims stripped, unverified_register_hash | Documents the anti-hallucination layer ran |
| Human approval | investigator_id, section, diff_hash, timestamp | The accountability record — who approved what |
| Final report generation | report_hash, citation_map_hash, model_versions_used | Tamper-evident seal on the final output |
Cluster A — Documentary Evidence
Turns any document pile into queryable evidence. Upload a contract, spreadsheet, email chain, or scanned report — every sentence becomes retrievable by meaning, not keyword. An investigator asking "what did the subject know and when?" gets an answer in seconds, with source and page number attached.
A1 Ingestion
Parses all file types: PDF, Excel, CSV, email, Word, audio transcripts.
A2 Chunking
Splits documents using document-type-aware strategies.
A3 Embedding
Converts chunks to vectors using BGE-M3 locally. No data leaves environment.
A4 Extraction
Pulls entities, dates, amounts, relationships from each chunk.
A5 Templates
Pre-built RAG queries optimised per report section and evidence type.
A6 Parallel Gen
All report sections generated simultaneously. 60–90 seconds total.
A7 Citation Map
Auto-generated index mapping every claim to its source document, chunk, and page.
Tools Required
| Component | Tool | Cost |
|---|---|---|
| PDF parsing | PyMuPDF (fitz) | Free |
| OCR for scanned PDFs | Tesseract | Free |
| Excel/CSV parsing | pandas + openpyxl | Free |
| Email parsing | extract-msg + mailparser | Free |
| Embedding model | BGE-M3 via Ollama | Free |
| Vector store | Qdrant self-hosted | Free |
Cluster B — Behavioural & Interview
Detects when someone says something in an interview that contradicts the evidence. The system checks every document and transaction record in the case and flags contradictions automatically — citing both the statement and the contradicting evidence.
B1 Transcript Chunking
Splits by speaker turn. Each chunk tagged with speaker ID, timestamp, session ID.
B2 Contradiction Detector
Cross-references every transcript claim against document corpus and graph. Cites both sides.
B3 Transcript Approval Gate
Separate approval flag for findings from interview statements — beyond standard section approval.
Tools Required
| Component | Tool | Cost |
|---|---|---|
| Speech-to-text (local) | Whisper via Faster-Whisper | Free |
| Speaker diarisation | pyannote-audio | Free (HuggingFace token required) |
| PII scrubbing | Presidio with custom recognisers | Free |
| Contradiction detection | LangChain + Qdrant retrieval | Free |
Cluster C — Entity & Network Intelligence
Answers the question no document search can: who is connected to whom? Six independently filed insurance claims look unrelated until a graph shows four share two witnesses.
Tools Required
| Component | Tool | Cost |
|---|---|---|
| Graph database | Neo4j Community self-hosted | Free |
| Graph queries | Neo4j Python driver + Cypher | Free |
| NER for entity extraction | spaCy + custom models | Free |
| Fuzzy matching (Tier 2) | rapidfuzz (Jaro-Winkler) | Free |
| Graph algorithms | Neo4j Graph Data Science plugin | Free (community) |
Cluster D — Advanced Reasoning
The reasoning engine. Forms a hypothesis, searches for evidence that tests it, challenges its own reasoning, revises, repeats. A 340-page document set that takes 3 days manually produces a draft finding in 90 seconds.
The Adversarial Critic — Active Hypothesis Invalidation
A reasoning loop that only refines toward a conclusion — without genuinely testing alternative explanations — is not an investigation tool. It is a confirmation bias engine. The Adversarial Critic is the architectural control that prevents this.
For every iteration where the evidence score exceeds 0.70, the Adversarial Critic fires an explicit counter-query: what evidence, if present in the corpus, would contradict or explain away the current hypothesis? It retrieves that counter-evidence and forces the reasoning loop to account for it before proceeding.
| Query Type | What It Asks | What It Surfaces |
|---|---|---|
| Alternative explanation | What legitimate business reason exists for the observed pattern? | Legitimate invoices, documented approval chains, regulatory filings |
| Counter-entity | Does any evidence suggest a different entity — not the subject — is responsible? | Third-party records, alternate beneficiary documents, entity resolution edge cases |
| Timeline invalidation | Is there evidence contradicting the proposed chronology? | Timestamp mismatches, document dating anomalies, contradicting records |
| Typology mismatch | Does the pattern match any known non-suspicious pattern from your institutional corpus? | Cluster F holdout patterns, industry-standard profiles, regulatory safe harbour descriptions |
A hypothesis that cannot withstand the Adversarial Critic is flagged as "contested hypothesis — alternative explanation not ruled out" and escalated with both supporting and counter-evidence presented side by side for the investigator to adjudicate.
How to Build It
- Fire after every reasoning iteration where evidence score exceeds 0.70
- Use same retrieval mechanism as main loop — RAG query against case namespace
- Pass adversarial query results to Critic alongside supporting evidence
- Block evidence score from exceeding 0.85 until adversarial check passes or investigator explicitly overrides
- Prompt approach: "Given the current hypothesis [X], retrieve all evidence in the case corpus that could support an alternative explanation. Focus on: legitimate business context, third-party responsibility, timeline contradictions."
Tools Required
| Component | Tool | Cost |
|---|---|---|
| Reasoning loop | LangGraph | Free |
| Local 8B model (orchestration) | Llama 3.1 8B via Ollama | Free |
| Local 70B model (analysis) | Llama 3.1 70B via Ollama | Free + GPU compute |
| Schema validation | Pydantic | Free |
| Hypothesis state persistence | Redis or PostgreSQL | Free |
Cluster E — Privacy Gateway
Replaces every personal identifier with an anonymous token before anything leaves your systems. The AI reasons about tokens, not people. Real identifiers restored only after investigator approval, only inside your environment.
Tools Required
| Component | Tool | Cost |
|---|---|---|
| PII detection | Microsoft Presidio + custom recognisers | Free |
| Token map store | Redis (in-memory, encrypted) | Free |
| External AI API | Anthropic Claude / OpenAI GPT-4 | $3–15 per M tokens |
Cluster F — Institutional Memory
After 200+ cases, train the AI on your own investigation history. It learns your fraud typologies, your reporting standards, what you flag and what you clear.
Do not fine-tune before 200 completed cases. A model trained on 80 cases overfits. The general model outperforms an under-trained specific one.
Tools Required
| Component | Tool | Cost |
|---|---|---|
| Fine-tuning compute | AWS/GCP GPU instances (A100) | $3–8/hr (₹252–672/hr) |
| Training framework | Axolotl or LLaMA-Factory | Free |
| Holdout evaluation | Custom evaluation harness | Free |
Cluster G — Integrity Elevation
Daily external anchoring of your audit chain. A third party can mathematically verify every entry — without trusting your systems. "We say it wasn't altered" becomes "it's mathematically provable."
Tools Required
| Component | Tool | Cost |
|---|---|---|
| External ledger anchor (G1) | AWS QLDB or Azure Confidential Ledger | ~$5–20/month |
| Recording → transcript hash link (G2) | Custom Python — SHA-256 + PostgreSQL | Free |
| Chain verification tool | Custom Python verification script | Free |
Building With AI Tools
Which AI tools to use for which parts of the build — and how to prompt them effectively.
You provide the architecture and domain expertise. AI tools write the code. You design each component's inputs, outputs, and constraints in plain English — AI coding tools implement it. Never ask AI to design the architecture from scratch.
Which AI Tool for Which Task
| Task | Best Tool | Sample Prompt Approach |
|---|---|---|
| Write pipeline code (ingestion, chunking, embedding) | Claude Code or Cursor | "Build a document ingestion pipeline: takes PDF, extracts with PyMuPDF, chunks by paragraph with 100-char overlap, stores in Qdrant namespace {case_id} with metadata {source, page_ref, chunk_id}" |
| Design database schemas | Claude (this interface) | "Design a PostgreSQL schema for an append-only audit log with SHA-256 hash chaining where each row links to the previous row's hash." |
| Write Cypher queries for Neo4j | Claude or ChatGPT | "Write a Cypher query that detects circular fund flows where the same amount passes through 3+ entities and returns to origin within 30 days." |
| Build the HITL review UI | Claude Code or v0.dev | "Build a section review interface: finding text on left, citation map on right, edit button, approve button. Approve writes to audit log." |
| Write investigation query templates | Claude (this interface) | "Write a RAG query for the Red Flags section of a financial crime investigation. Should surface: structuring, layering, velocity spikes, round-tripping." |
| Debug errors | Claude Code or Cursor | Paste the full error + relevant code. "This is failing at the Qdrant write step. Here is the error and the ingestion code." |
AI Tool Subscriptions You Need
| Tool | What It Is | Cost | INR | Priority |
|---|---|---|---|---|
| Claude Pro | Architecture design, query templates, documentation, review. | $20/month | ₹1,680/month | Essential |
| Claude Code | Terminal-based agentic coding. Reads your repo, writes and runs code, fixes errors. | ~$30–100/month | ₹2,520–₹8,400/month | Essential for build phase |
| Cursor | IDE with AI coding built in. Alternative to Claude Code. | $20/month | ₹1,680/month | Choose: Cursor OR Claude Code |
| LangSmith | Traces every LLM call — essential for debugging the reasoning loop. | Free / $39/month | ₹3,276/month | Add in Phase 4 |
Prompting Principles
Specify Inputs and Outputs Exactly
Don't say "build a chunker." Say "Build a chunker that takes a string and returns a list of dicts with keys: chunk_id (UUID), text (string), page_ref (int), source_doc (string), char_count (int). Use 500-char chunks with 100-char overlap."
Include the Non-Negotiable Constraint
Always add the architectural constraint: "The namespace_id must match the case_id — never query across namespaces" or "Temperature must be 0 — no randomness in extraction tasks."
Ask for Tests Alongside Code
Add "also write 3 pytest test cases" to every code request. Test cases clarify what the code should do and catch errors before production.
Review Before Running
Read the generated code before running in production. You don't need to understand every line — but you should understand what it does. Ask Claude to explain anything unclear.
Deployment Profiles
Six reference configurations — from minimal compliant to full enterprise forensic. Start where your needs are, grow from there.
| Profile | Components | Use Case | Monthly Cost | Build Time |
|---|---|---|---|---|
| 1 — Minimal Compliant | 8 | Simple HR, low-volume internal investigations | $20–50 / ₹1,680–4,200 | 4–6 weeks |
| 2 — Interview-Led | 11 | HR investigations, whistleblower, testimony-primary cases | $30–80 / ₹2,520–6,720 | 6–8 weeks |
| 3 — Document Investigation | 20 | Legal due diligence, procurement review, regulatory exam prep | $80–200 / ₹6,720–16,800 | 10–14 weeks |
| 4 — Corporate Fraud | 28 | FCPA/ABAC, bribery, beneficial ownership, multi-party fraud | $200–500 / ₹16,800–42,000 | 16–24 weeks |
| 5 — Financial Crime Full | 31 | AML, STR generation, transaction monitoring follow-up | $600–2,000 / ₹50,400–1,68,000 | 24–32 weeks |
| 6 — Enterprise Forensic | 35 | Enforcement agency, Big 4 forensic, court-bound outputs | $2,000–6,000 / ₹1,68,000–5,04,000 | 32–40 weeks |
Documents primary → Profile 3. Interviews primary → Profile 2. Entity network mapping needed → Profile 4. Regulated financial institution with external AI → Profile 5. Start at the lowest profile covering your current needs — upgrade incrementally without rebuilding.
Frequently Asked Questions
You need at least one person who can write Python and work with APIs. With Claude Code or Cursor, a single developer can build Profile 3 in 8–12 weeks. You as the founder provide the domain knowledge — what the system needs to do, what constitutes a finding. AI tools handle most of the actual code writing.
Yes, with a critical caveat: investigation data is sensitive. The vector database, graph database, audit log, and PII token map must stay on-premise or in a jurisdiction-compliant private cloud. Only pseudonymised data may reach external AI APIs (Cluster E). For Profiles 1–4, everything can run on a private cloud VM without any external AI.
Phase 1 (Mandatory Base) delivers defensible audit-logged case management in 4–6 weeks — before any AI reasoning is active. Phase 2 delivers evidence search in under 1 second by week 14. Phase 4 delivers draft findings in 60–90 seconds. You do not need the full platform to start getting value.
Profiles 1–3 (no 70B model): cloud VM with 8GB RAM and 4 vCPUs (~$30–50/month on AWS/GCP). Profile 4+ (70B model): GPU instance with at least 40GB GPU RAM. AWS g5.12xlarge or GCP a2-highgpu-1g. Budget $150–400/month for compute. Alternatively, an RTX 4090 workstation (24GB VRAM) can run 70B in two instances.
The architecture is designed for maximum defensibility — tamper-evident audit chain, human approval at every finding, evidence citations on every claim. With Cluster G active, integrity is independently verifiable. However, admissibility is a legal question that depends on jurisdiction and proceeding type. Engage qualified legal counsel before relying on IRE outputs in any formal proceeding.
The architecture is jurisdiction-agnostic — map your specific regulatory framework into the investigation methodology document. For RBI/PMLA: your STR templates, threshold amounts, and filing windows become the query template library (A5) and framework coverage checklist (D4). For data residency: all data stores must be on Indian infrastructure — use AWS Mumbai, Azure India, or domestic providers (Jio Cloud, Tata TCS). Engage a PMLA-specialist counsel to sign off before production.
Institutional-grade (Base B5+B6): the audit chain is tamper-evident — any modification breaks the cryptographic links, detectable by anyone with access to your systems. Court-grade (+ Cluster G): the daily chain hash is anchored to an independent external ledger. A third party can verify integrity without accessing your systems. "We say it wasn't altered" vs "a third party can prove it wasn't altered."
Don't build this if: fewer than 5 investigations per month (overhead exceeds value at that scale); investigation methodology is not documented (nothing to encode); you cannot allocate one developer for 3+ months. Start with a simpler case management system and return to IRE when volume and complexity justify it.
Investigator UX — Decision Screens
The backend is only half the system. This is where product success actually happens — the investigator-facing interfaces that make the architecture usable in practice.
Both the whitepaper and most technical architectures are backend-heavy. But investigators interact with screens, not APIs. If the decision screens are confusing, slow, or disconnected from the evidence, investigators will bypass them — defeating the human gate entirely.
The Five Critical Screens
Screen 1 — Entity Resolution Decision Screen
Appears when Tier 2 or Tier 3 entity match is detected. The most important HITL screen in the system — graph quality lives or dies here.
| Left Panel | Right Panel | Bottom Bar |
|---|---|---|
| Entity A — all known attributes, source documents, frequency of appearance | Entity B — all known attributes, source documents, frequency of appearance | Three buttons: MERGE (green) / KEEP SEPARATE (gray) / LINK AS RELATED PARTY (yellow). Confidence score shown prominently. Timer visible. |
Design principle: Never show just names. Show the evidence that led to the match proposal — field comparison table, similarity scores, co-occurrence in documents. The investigator is making an evidentiary decision, not a clerical one.
Screen 2 — Finding Review Screen (Section-by-Section)
The primary review interface. Appears after report generation for each section.
| Left Panel — Finding | Right Panel — Citation Map |
|---|---|
| Full finding text with inline citation tags highlighted. Edit button. Word count. Iteration count (how many reasoning cycles produced this). | Every citation expanded: document name, page number, chunk text preview. Click to open source document at exact location. |
Design principle: The investigator should never have to leave this screen to verify a citation. Everything needed to approve or reject is visible in one view.
Screen 3 — Contradiction Alert Screen
Appears when Cluster B detects a statement-evidence contradiction. Requires separate approval before entering the record.
| Top Half — Statement | Bottom Half — Contradicting Evidence |
|---|---|
| Exact transcript quote with speaker ID, timestamp, session number, and interview context. | The contradicting document chunk, highlighted to the exact contradicting sentence, with source and page. |
Design principle: Both sides presented simultaneously. Investigator must explicitly assess the contradiction — not just note that it exists. Approval options: MATERIAL CONTRADICTION / MINOR DISCREPANCY / NOT A CONTRADICTION (with rationale required for the last option).
Screen 4 — Graph + Evidence Unified View
The most complex screen. Shows entity network alongside the documents that support it.
- Graph panel (left 60%): Interactive entity network. Click any node → documents referencing that entity appear in right panel. Click any edge → transactions supporting that relationship appear.
- Document panel (right 40%): Scrollable evidence feed filtered to selected node/edge. Each chunk shows source, page, date, and citation count.
- Timeline bar (bottom): Drag to filter both graph and documents to a date range.
Design principle: Graph and documents are not separate tabs — they are linked views. Selecting a graph element immediately updates the document panel. This is the screen that makes network intelligence accessible to non-technical investigators.
Screen 5 — Adversarial Review Screen
Appears when the Adversarial Critic flags a contested hypothesis.
| Left Column — Supporting Evidence | Right Column — Counter-Evidence |
|---|---|
| Evidence supporting current hypothesis with citations and evidence score. | Evidence supporting alternative explanation with citations and source. |
Investigator must explicitly choose: HYPOTHESIS STANDS / REVISE HYPOTHESIS / ESCALATE FOR SENIOR REVIEW. Cannot skip this screen when adversarial flag is raised.
Build Approach for Non-Technical Founders
Wireframe First with Claude
Describe each screen to Claude: "Build an HTML mockup of a finding review screen with: left panel showing finding text with highlighted citations, right panel showing citation map with source preview, approve/edit/reject buttons at the bottom." Claude will generate a working HTML prototype in one pass.
Test With a Real Investigator Before Coding the Backend
Get an HTML mockup in front of an actual investigator (even yourself) before connecting it to the backend. The most expensive UI bugs are architectural — discovered after the backend is built.
Graph Visualisation: Use an Existing Library
Do not build a graph renderer from scratch. Use: vis.js Network (free, straightforward) or Cytoscape.js (more powerful, still free). Connect to your Neo4j query layer via a simple REST API. The library handles the rendering.
Every Approval Writes to Audit Chain
The UI is the interface to the audit chain. Every button click that constitutes an investigator decision must write an immutable log entry. This is not optional — build it into the frontend as a requirement, not an afterthought.
Graph Visualisation Libraries
| Library | Best For | Cost | Neo4j Integration |
|---|---|---|---|
| vis.js Network | Simple to medium complexity graphs. Quick to set up. Good for most investigation use cases. | Free | Fetch Cypher results as JSON, feed directly to vis.js dataset |
| Cytoscape.js | Complex graphs with custom layouts and styling. Better performance at scale. | Free | Same as vis.js — JSON data format |
| Neo4j Bloom | Built-in Neo4j graph explorer. Zero frontend code. Good for internal teams. | Free (Community) / $65+/mo (Cloud) | Native — no integration needed |
| D3.js Force Graph | Maximum customisation. Steep learning curve. | Free | Manual data transformation required |