Learning Stigmergic Communication Strategies for Multi-Agent Systems

The objective of this thesis is to study the possibility of learning how to use pheromone-based communication to achieve collective goals in multi-agent systems, by analyzing an application on a predator-prey scenario, and another on a robot warehouse. Designing an effective communication system between agents in a multi-agent system, to achieve effective coordination, is a key challenge. This work aims to evaluate whether stigmergic communication can be learned and if it leads to good performances. The system is implemented using a Python framework called Mesa, that provides an easy and flexible way to simulate multi-agent systems where agents can interact in a shared environment. Reinforcement learning is applied to agents by using Q-learning, to enable agents to learn effective coordination strategies through experience without the need to share information---hence, fully autonomously and independently. Results show that agents are able to learn a better policy in comparison to a static baseline defined at design time, this can be attributed to the balance of two complementary behaviors: learning to follow or avoid pheromones (that is, to react appropriately to messages) and learning to deposit pheromones (that is, to send messages). In the predator-prey model, this trade-off leads to better spatial coverage and cooperation, recreating the natural phenomenon of predation, where predators surround the prey. When pheromone release is treated as a learnable action, predators are able to develop cooperative tactics, resulting in an improvement in their overall efficiency in capturing prey. In the robot warehouse scenario, pheromone-based communication enables agents to coordinate their actions more efficiently, resulting in faster package retrieval. By learning when to release pheromone as a request of help and when to respond to such requests, agents are able to collaborate to achieve a global goal (lifting heavy packages) together. These results show the potential of learning stigmergic communication in decentralized agent systems and open promising directions for adaptive coordination in swarm intelligence and robotics.