Projects

Despite remarkable advances in artificial intelligence, current foundation models still struggle with basic forms of physical commonsense that humans acquire early in development, such as object permanence, gravity, and motion continuity. In contrast, infants learn these concepts through continuous interaction with their environment, active exploration, and causal learning. This project investigates whether adopting a developmental and embodied learning approach can enable the emergence of physical commonsense in foundation models.

Drawing on insights from developmental psychology, cognitive science, and robotics, we propose a framework in which agents learn through curiosity-driven exploration, sensorimotor interaction, and self-caused actions within stable and continuous environments. The project integrates three complementary components: (1) cognitive models of curiosity-driven learning implemented on an embodied robot interacting with physical objects, (2) experimental studies of how different sensorimotor environments influence the development of physical reasoning, and (3) large-scale simulation environments for training computational models that can be integrated into foundation models.

By focusing on core physical principles, including object permanence, support, gravity, inertia, momentum, and motion continuity, the project aims to demonstrate how developmental mechanisms can significantly improve physical reasoning in AI systems. The results will contribute to more robust, reliable, and realistic AI models with applications in robotics, video understanding, simulation, and autonomous systems.

---

Recent advances in Transformer-based Large Language Models (LLMs) have proven to be revolutionary in generating coherent conversations. However, their lack of symbol grounding and their sample inefficiency limits their applicability in robotics where adaptation to physical and social environments is essential. This project aims to address these limitations by grounding LLMs in an intrinsically motivated robotic agent that interacts socially, drawing inspiration from human language acquisition. Our approach is built on a dual model of symbol grounding that integrates individual sensorimotor grounding with social grounding of concepts. By combining our prior work, which grounded symbols in intrinsic motivations, with an LLM, we hope to create a system that learns from social cues (facial expressions, gaze, etc). Our four-fold work plan intends to: (1) integrate our current model with an LLM to cater to abstract language structures; (2) develop an intrinsically motivated social reward mechanism to adapt robot language output to the social interaction context; (3) deeply analyze the embedding space aspect of our model to better align them with human language learning mechanisms, using typical infant-caregiver strategies for sample efficiency and continuous learning and (4) evaluation that will comprise a comparison of our grounded model against traditional LLMs in both simulated and in a real-world environments and reproduction of cognitive studies.

---

The objective of this project is to design and evaluate a robot-student (RS)
capable of learning in real time through interaction with pupils, using reinforcement learning (RL) techniques and neural networks. Building on the learning-by-teaching paradigm (the “protégé effect”), this system aims to foster engagement, skills development, and metacognitive strategies among students, particularly those experiencing academic difficulties. The project is structured around three main axes:

1. Modeling an online-learning robot-student;
2. Extending its capacities to diverse and realistic tasks (multiple-choice questions, geometric constructions, open-ended questions), including the integration of generative AI;
3. Evaluating the system’s impact both in laboratory settings and in classrooms, in collaboration with Prof En Poche, an EduTech startup.
   
4. The project has a dual ambition: (i) to support students’ learning through active engagement, and (ii) to provide teachers with a novel tool for observing their pupils’ metacognitive strategies, as revealed through the knowledge acquired by the robot. The proposed solution will be integrated
   in the MathIA solution (Prof en Poche) and deployed for upper primary and lower secondary pupils (ages 9 to 11) in the context of the P2IA Partenariat d’Innovation Intelligence Artificielle of the French Ministry of Education.
   

Présentation du Projet P2IA @Educatech

Présentation vidéo

---

This research challenges the traditional passive-learner approach in computational models of language development, where artificial agents are taught language as an isolated goal. Instead, it proposes a functional perspective, viewing communication as a means to achieve broader objectives, such as satisfying needs or fostering social bonds, reflecting observations in infant language development. While current models capture different aspects of language acquisition, they largely overlook the role of affect, emotions, and social interaction. This project aims to bridge this gap by integrating affective and motivational systems into language acquisition models for robots, enabling them to learn language in a socially interactive and goal-directed manner, similar to human infants. The research further seeks to explore how caregiver responses and emotional signals can shape language development, proposing an experimental framework to model and test these functions using reinforcement learning.

The funding is for a PhD student, Zakaria Lemhaouri and the environment.

---