Build an FAQ RAG System for Customer Support

End-to-end workflow for building a production FAQ chatbot using Retrieval-Augmented Generation (RAG). Covers knowledge base preparation, hybrid retrieval pipeline (BM25 + bi-encoder + cross-encoder reranking), LLM generation with input/output guardrails, multilingual golden dataset creation, and automated evaluation.

When to use: You need an AI-powered FAQ system that retrieves answers from an existing knowledge base (help center, docs, wiki) and generates natural-language responses with safety guardrails.

Preconditions:

• Existing FAQ or help center content (articles, docs, or structured Q&A)
• Access to an embedding model and an LLM (open-source or API-based)
• A vector database or search index
• Domain volunteers or annotators for golden dataset labeling (if multilingual)

Key tuning points:

• Chunk size and overlap (step 1) — adjust based on article length distribution
• Model selection (step 3) — open-source multilingual (e.g. Mistral <8B) vs. proprietary tradeoff
• Retrieval weights (step 5) — BM25 vs. semantic score balance
• Evaluation thresholds (step 7) — minimum aggregate score for go/no-go
• Target languages (step 4) — which languages to include in golden dataset

Evaluation formula: Aggregate score = (answer relevancy + faithfulness + no hallucination + prompt alignment) / 4

Reference: Based on patterns from Amex GBT Egencia's FAQ RAG system (470K+ conversations/year). See the 3-part series on the Amex GBT Technology blog for detailed implementation context.