Multi-agent LLM-based system for tabular question answering with OrQA dataset

Natural Language Processing, a branch of artificial intelligence (AI), has evolved to the point where machines have begun to communicate like humans. This aspect was further highlighted with the advent of large language models (LLMs), vast boxes of artificial neural connections whose speaking behaviors are virtually indistinguishable from those of humans. Although this opened the door to a variety of applications, there were still limitations to overcome: the ability to efficiently keep internal knowledge updated and the ability to interact with external entities. These challenges were effectively addressed with the advent of retrieval-augmented generation (RAG), which allowed models to access constantly updated external knowledge, and tool augmentation, a clever way to let an LLM manipulate external objects by generating special tokens, enabling it to interact with its surrounding environment like an autonomous agent. Despite significant research progress in recent years, there are still many areas that require further work. Among these, the effectiveness of language models in manipulating structured data such as tables is demonstrated, especially those from a different distribution than that used for training. In this regard, a system based on the interaction of multiple agents impersonated by a language model was tested for the retrieval of tabular data from Open Data portals with the aim of answering natural language questions from users. The system has shown weaknesses, especially in the retrieval part, where it struggles to retrieve the correct tables; this is because the corpus of tables is accessed in real time and the indexing is clearly different from that of traditional retrieval systems. To try to mitigate this problem, the retrieval model has been fine-tuned to suit the characteristics of Open Data portals. Despite these efforts, this did not yield the desired results, suggesting that isolated improvements within a distributed system do not promote collaborative behaviors between the various agents.