Turning Civil Engineering Documents into Conversations: Design and Evaluation of a Retrieval-Augmented Generation Chatbot

This thesis presents the design and evaluation of a Retrieval-Augmented Generation (RAG) chatbot developed to support internal document search at Fhecor Ingenieros Consultores, a structural engineering consulting company based in Madrid. The system, named Fhecor ChatBot, allows engineers to query the company's technical documentation in natural language. The project is built around a hybrid and adaptive document processing pipeline, which applies distinct text preparation strategies based on the source file type. For non-text-based formats such as images, videos, and spreadsheets, a custom Python library, called Document Summarizer, was developed. This library enables automatic summarization of document content from formats for which standard text extraction methods are insufficient or unavailable, combining Optical Character Recognition (OCR) and Natural Language Processing (NLP) techniques with Large Language Models. For text-native documents such as PDF and Word files, a custom LangChain-based semantic chunking library was implemented, segmenting text into semantically coherent blocks, preserving the fine-grained technical detail that summarization tends to discard. The system was evaluated through a systematic experimental study on a corpus of 100 civil engineering documents, comparing the two preprocessing pipelines on a general query set of 30 queries and conducting an ablation study across four retrieval configurations on a technical query set of 50 queries validated by domain experts. Evaluation combines standard retrieval metrics, including Precision@10, Recall@10, and MRR, with automated generation quality assessment via the RAGAS framework. The results demonstrate that semantic chunking substantially outperforms summarization on technical retrieval tasks, with MRR improving from 0.1333 to 0.4222 and Recall@10 from 0.1333 to 0.4667. The legacy summarization-based system failed completely on technical queries, scoring zero across all metrics, while the proposed architecture achieves a faithfulness of 0.6577 and an answer relevancy of 0.8463. The reranking component provides consistent improvements in ranking quality, while the validator introduces a deliberate safety trade-off between response completeness and factual grounding. A web application built with FastAPI and Vue.js serves as the deployment interface. These results confirm that adaptive preprocessing is a critical factor in optimizing RAG performance for specialized engineering domains.