RAG Customer Support Telegram Bot

Enterprise-grade Retrieval-Augmented Generation (RAG) customer support assistant featuring semantic document chunking, strict source citations, SQLite session memory, and deterministic anti-hallucination guardrails.

Key Features • Architecture • Quick Start • Telemetry & Cost • Design Decisions • Offline Evaluation

---

Overview

This repository contains a reference implementation of an AI customer support bot for a direct-to-consumer hardware brand (Apex Gear & Electronics).

The bot grounds every customer answer in 15 structured knowledge base documents (store policies, warranty conditions, product specifications) using semantic vector retrieval and enforces file-level citations to eliminate hallucinations.

┌─────────────────────────────────────────────────────────────────────────────┐
│  Customer: "What is your return policy and do I have to pay for shipping?"  │
├─────────────────────────────────────────────────────────────────────────────┤
│  Bot: Here is our verified return policy for Apex Gear & Electronics:       │
│  • Return Window: 30 days from carrier delivery date.                       │
│  • Condition: Original packaging and all accessories intact.                │
│  • Return Shipping Fee: Prepaid label provided; flat $6.00 fee deducted.    │
│  • Refund Timeline: 2-3 business days inspection + 5-7 banking days.        │
│                                                                             │
│  [Source: return_and_refund_policy.md]                                      │
└─────────────────────────────────────────────────────────────────────────────┘

---

Key Features

• 📑 Semantic Document Chunking: Uses MarkdownNodeParser to split documents along logical headings rather than naive character-count cuts.
• 🎯 Grounding & Mandatory Citations: Prompts enforce explicit source tags ([Source: filename.md]) on every factual statement.
• 🛡️ Zero-Hallucination Fallback: Out-of-domain queries (e.g. competitors, unrelated items) trigger polite predefined fallback messages.
• 🧠 Bounded Multi-Turn Memory: Persists up to 10 recent messages (5 dialogue turns) in SQLite to maintain conversation context without prompt bloat.
• 📊 Deterministic Offline Evaluator: Includes an offline dry-run test suite (offline_eval.py) measuring exact prompt assembly, token growth, and API cost without requiring active API keys.

---

Architecture

---

Repository Structure

rag-support-bot/
├── knowledge-base/                  # 15 structured domain markdown documents
│   ├── faq_general.md               # Common customer FAQs & discount rules
│   ├── payment_and_financing.md     # Payment methods, Klarna, tax info
│   ├── return_and_refund_policy.md  # 30-day return window & $6 fee policy
│   ├── shipping_delivery_policy.md  # SLA, courier rules, free shipping threshold
│   ├── warranty_and_repairs.md      # 1-2 year warranty terms & claims
│   └── product_*.md                 # 10 comprehensive product spec files
├── src/
│   ├── __init__.py
│   ├── bot.py                       # aiogram 3.x bot entrypoint and message router
│   ├── rag_chain.py                 # LlamaIndex query engine, retriever & prompt
│   ├── database.py                  # SQLite conversation memory manager
│   └── ingest.py                    # Knowledge base parser & ChromaDB loader
├── offline_eval.py                  # Offline dry-run evaluation suite
├── .env.example                     # Environment configuration template
├── requirements.txt                 # Pinned dependencies
└── LICENSE                          # MIT License

---

Verification Status

Component	Status	Verification Method
Document Ingestion	Verified	src/ingest.py chunks 15 markdown files into semantic vector embeddings
Prompt Assembly & Memory	Verified	offline_eval.py validates dynamic prompt assembly and 10-message SQLite history
Token & Cost Telemetry	Verified	Measured exact token distribution across in-domain and out-of-domain evaluation turns
Live OpenAI API Call	Reference	Live API responses depend on user-provided OPENAI_API_KEY
Live Telegram Polling	Reference	Interactive chat requires setting TELEGRAM_BOT_TOKEN in .env

---

Quick Start

1. Prerequisites & Installation

# Clone the repository
git clone https://github.com/therealfullmetal55555/rag-support-bot.git
cd rag-support-bot

# Create and activate a virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Offline Dry-Run (No API Key Required)

Run the deterministic evaluation benchmark to inspect prompt assembly, token count accounting, and source citation logic locally:

python3 offline_eval.py

🔍 Click to view Sample Benchmark Output

====================================================================================================
RAG SUPPORT BOT: OFFLINE DRY-RUN (PROMPT ASSEMBLY & TOKEN ACCOUNTING)
====================================================================================================

[1/5] Question [In-Domain (Store Policy)]:
     "What is your return policy, how many days do I have, and do I have to pay for return shipping?"
  Expected source: `return_and_refund_policy.md`
  Reference answer (pre-written):
     Here is our verified return policy for Apex Gear & Electronics:
     • Return Window: 30 days from carrier delivery date.
     • Condition: Original packaging and all accessories must be intact.
     • Return Shipping Fee: Prepaid label provided; flat $6.00 processing fee deducted.
     • Refund Timeline: 2-3 business days for warehouse inspection, 5-7 banking days for credit.
     
     [Source: return_and_refund_policy.md]
  Measured Telemetry:
     • Embed Tokens: 23
     • Prompt Tokens: 677 (System: 256 | Context: 378 | History: 1)
     • Completion Tokens: 99
     • Calculated Query Cost: $0.000161 USD
----------------------------------------------------------------------------------------------------

OFFLINE DRY-RUN SUMMARY:
  • Queries evaluated: 5
  • Average Prompt Tokens: 791.2
  • Average Completion Tokens: 77.2
  • Average Embed Tokens: 20.6
  • Average Calculated Cost / Query: $0.000165 USD
  • Cumulative 5-Turn Session Cost: $0.000827 USD
====================================================================================================

3. Live Deployment

1. Set up your environment variables:

   cp .env.example .env
   # Edit .env with your OPENAI_API_KEY and TELEGRAM_BOT_TOKEN

2. Build vector embeddings in ChromaDB:

   python3 -m src.ingest

3. Start the Telegram bot:

   python3 -m src.bot

---

Token Telemetry & Cost Breakdown

Pricing baseline based on OpenAI standard rates (text-embedding-3-small @ $0.020 / 1M tokens, gpt-4o-mini @ $0.150 / 1M input, $0.600 / 1M output):

Per-Query Unit Economics (Measured)

$$\text{Query Cost} = \left(\frac{21}{10^6} \times $0.02\right) + \left(\frac{791}{10^6} \times $0.15\right) + \left(\frac{77}{10^6} \times $0.60\right) \approx \mathbf{$0.000165 \text{ USD}}$$

Metric	Measured Average	Description
Embedding Query	~21 tokens	Input query vectorization
System Prompt	~256 tokens	Grounding rules, citation formatting & fallback instructions
Context Chunks	~340 tokens	Top-3 relevant markdown node chunks
SQLite History	~195 tokens	Bounded sliding window across recent turns
Total Prompt Input	~791 tokens	Total LLM context window load
Completion Output	~77 tokens	Grounded bulleted answer + citation tag

Monthly Projection Matrix

Monthly Volume	Embedding Cost	Prompt Input Cost	Completion Cost	Total LLM Cost	Vector DB Infra	Total Cost / Mo
1,000 queries	$0.0004	$0.1186	$0.0462	$0.165	$0.00 (Local ChromaDB)	~$0.17 USD
10,000 queries	$0.0042	$1.1865	$0.4620	$1.653	$0.00 (Local ChromaDB)	~$1.65 USD
100,000 queries	$0.0420	$11.8650	$4.6200	$16.527	$15.00 (Managed Cloud)	~$31.53 USD

---

Engineering Design Decisions

Decision	Implementation	Why it matters
Heading-Aware Parsing	MarkdownNodeParser	Preserves product specs, warranties, and tables as unified semantic entities rather than cutting mid-sentence.
Deterministic Citations	Explicit Prompt Guardrail	Forces every factual claim to reference [Source: filename.md], allowing instant auditing by human supervisors.
Sliding Memory Window	limit_turns * 2 messages	Prevents unbounded context growth in long sessions while maintaining multi-turn conversational coherence.
Embedded Vector Storage	Local ChromaDB	Eliminates cloud database dependencies during local development and testing.
Synthetic Dataset	Fictional Store Fixtures	Clean demonstration dataset (Apex Gear & Electronics) avoiding proprietary data exposure.

---

Roadmap & Production Enhancements

• Similarity Score Cutoff: Add a strict cosine threshold (e.g. similarity_cutoff = 0.65) before passing retrieved nodes to the synthesizer.
• Hybrid Retrieval (BM25 + Dense): Combine keyword matching with vector embeddings for exact SKU code and order number lookups.
• Human-in-the-Loop Escalation: Implement automatic operator dispatch when fallback triggers two consecutive times.

---

License

Distributed under the MIT License. See LICENSE for full details.