Design and Sim-to-Real Learning of a Tactile and ToF-Sensorized Two-Finger Gripper: From Embedded Admittance Control to Reinforcement-Learned Grasping Progettazione e apprendimento Sim-to-Real di un gripper a due dita dotato di sensori tattili e ToF: dal controllo di ammettenza embedded alla presa con Reinforcement Learning

This thesis presents the design, implementation, and preliminary validation of an end-to-end mechatronic and learning pipeline for a sensorized two-finger robotic gripper. Robust autonomous manipulation across varied object geometries and mechanical properties remains a fundamental challenge in robotics. While classical control methods struggle to generalize across diverse object profiles, learned grasping strategies often rely on idealized simulation environments that fail to mirror the physical realities of real-world tactile sensing. To bridge this gap, this work establishes a hardware and simulation platform where classical and learning-based control strategies can be realized, compared, and deployed using identical sensor modalities. The physical platform comprises a custom two-finger parallel gripper equipped with a dual Force-Sensing Resistor (FSR) matrix for tactile feedback and a multi-zone Time-of-Flight (ToF) sensor (VL53L7CX) for pre-grasp spatial localization. Low-level processing and control are executed on an STM32F446 microcontroller, integrating a real-time admittance control loop to enable compliant grasping behavior. High-throughput multi-sensor telemetry and motor commands are structured via a CANopen communication, decoupling local real-time control constraints from off-board logging. In parallel, a dedicated electronic hardware architecture was designed and fabricated as a custom PCB to consolidate sensing, power regulation, and signal processing into the gripper body. To complement the physical hardware, a sensor-faithful virtual model was developed in NVIDIA Isaac Sim. The virtual environment models the structural kinematics of the end effector while replicating the response characteristics of both the tactile FSR matrices and the multi-zone ToF distance sensors. Utilizing this simulated framework, a reinforcement learning (RL) policy was formulated and trained in NVIDIA Isaac Lab. The policy operates over combined tactile and proximity observation spaces to learn compliant grasping behaviors. The primary contributions of this work are fourfold: - Mechatronic Co-Design: Mechanical, electronic, and firmware co-design of a two-finger sensorized end effector featuring onboard real-time tactile and ToF processing. - Real-Time Communication Architecture: A deterministic CANopen communication framework balancing real-time motor control with high-rate sensor streaming. - Embedded Compliant Control: An embedded admittance control implementation evaluated on prototype hardware to demonstrate compliant grasping dynamics. - Sensor-Faithful Virtual Model and RL Pipeline: Construction of a sensor-matched virtual model in Isaac Sim along with an RL policy training pipeline in Isaac Lab. Preliminary evaluations confirm real-time closed-loop admittance performance and functional sensor streaming on the prototype platform, alongside stable simulation dynamics during RL training. A structured roadmap outlines immediate follow-up milestones, including hardware bring-up of the custom PCB, quantitative admittance benchmarking, converged policy optimization, and eventual transfer to the physical system.