Nadine: A large language model‐driven intelligent social robot with affective capabilities and human‐like memory

Abstract

AbstractIn this work, we describe our approach to developing an intelligent and robust social robotic system for the Nadine social robot platform. We achieve this by integrating large language models (LLMs) and skillfully leveraging the powerful reasoning and instruction‐following capabilities of these types of models to achieve advanced human‐like affective and cognitive capabilities. This approach is novel compared to the current state‐of‐the‐art LLM‐based agents which do not implement human‐like long‐term memory or sophisticated emotional capabilities. We built a social robot system that enables generating appropriate behaviors through multimodal input processing, bringing episodic memories accordingly to the recognized user, and simulating the emotional states of the robot induced by the interaction with the human partner. In particular, we introduce an LLM‐agent frame for social robots, social robotics reasoning and acting, serving as a core component for the interaction module in our system. This design has brought forth the advancement of social robots and aims to increase the quality of human–robot interaction.