Online Seminar Computer Science - Dr. Abdel Makkeh, ETH Zurich

Title: A quantitative approach to understanding information processing within intelligent systems

Abstract:

Intelligent systems, spanning from primates and the neocortex to artificial intelligence, exhibit a remarkable level of complex cognitive functionality. This functionality has played a pivotal role in human society's technological advancements and the creation of sophisticated artificial systems. Despite these remarkable achievements, there remains a profound gap in our understanding of how such complex functionality emerges and the underlying mechanisms. This knowledge gap raises critical questions, including concerns about social polarization and AI safety. To enhance our understanding of this intricate functionality, we focus on the information processing within intelligent systems. Information processing provides a framework to describe their functionality, offering insights into its emergence and mechanisms. Specifically, we establish a mathematical foundation for information processing using Information Theory and its recently developed branch, Partial Information Decomposition (PID). Information Theory's domain-agnostic generality enables us to ignore the intelligent systems idiosyncrasies and examine a variety of systems, while PID refines our understanding of information processing. Leveraging these tools, we have not only analyzed information processing in these systems but also introduced novel artificial intelligent systems known as Infomorphic Networks. In these networks, individual neurons employ information processing at the local level to achieve a wide range of complex functionalities and unveiled which local information processings promote supervised learning, feature extraction, and auto-associative memorization. Additionally, our work has contributed to a better understanding of information distribution in feedforward Deep Neural Networks and a preliminary tool to examine autonomy in reinforcement learning agents. By combining these two research approaches—studying and building—we have gained valuable insights into the emergence of complex functionality. These approaches also hold promise for developing interpretable and safe AI.