From Proto-Insights to Validated Business Insights: An LLM-Based Multi-Agent Architecture for Insurance Analytics

This thesis explores the most appropriate way to build and implement the architecture of an agent system, aiming to transform preliminary statistical signals into validated and meaningful insights in the MTPL sector. Preliminary signals are the result of pattern mining algorithms, such as association rule mining, and provide useful insights for agents to explore, interpret, and contextualize, and then validate quantitatively. To achieve this goal, the second part of the work, on which this thesis focuses, concerns Insight Refinement, based on LLM-based agents. The agent performing this task is called the Insight Hunter Agent and receives as input a batch of preliminary signals, called proto-insights. This is followed by the formulation of an initial hypothesis, generated from a process of reasoning, tool calls, and validation by a second agent, called the Data Analyst Agent. The work went through several iterations: starting with a single agent, then moving to multiple agents, and the architecture design paradigms also changed accordingly. The final logic primarily concerns the Insight Hunter, which accesses two YAML files representing the Data Catalog via tools, keeping the agent grounded in reality and the available data schema. This is followed by the formulation of a hypothesis in natural language, which is passed as input to the Data Analyst Agent. The latter uses specific insurance indicators, such as loss ratio, claim frequency, severity, and premium, to validate the assertion and produce an output report that the Insight Hunter will use for refinement. The work carried out, in addition to research and design, also focused heavily on prompt engineering and reasoning design. Every interaction between the Insight Hunter and the other architecture elements had to be carefully described in the Insight Hunter prompt to avoid ambiguity. This also includes a robust traceability system, which was necessary to implement for proper development and optimization. The entire work discussed in this thesis began as a POC (Proof of Concept), aimed at demonstrating the project's feasibility using intuitive tools and frameworks, such as the Strands Agents SDK, so we could assess whether the final insights generated by the agent system were truly business-relevant.