Jürgen Schmidhuber is a leading researcher in the field of artificial intelligence (AI) and is often referred to as the “father of neural networks.” He is renowned for his groundbreaking contributions to deep learning, which have had a profound impact on the development of AI and machine learning as we know it today. Schmidhuber is the co-founder and scientific director of the Swiss AI Lab IDSIA. He is a co-director of the Dalle Molle Institute for Artificial Intelligence Research and has received numerous awards and honors for his work, including the Helmholt Award from the International Neural Network Society, the IEEE Neural Network Pioneer Award, and the Neural Networks Pioneer Award from the IEEE Computational Intelligence Society. With over 600 scientific publications and a deep passion for exploring the frontiers of AI, Schmidhuber continues to inspire and challenge researchers around the world to push the boundaries of what’s possible in this rapidly evolving field.
Jurgen Schmidhuber spoke at the History of AI Roundtable of AIWS on February 17, 2023, moderator by Francesco Lapenta, Representative of Boston Global Forum in Rome. Here are some notes from his talk:
Some of his key accomplishments include:
- Development of LSTM: Schmidhuber, along with his colleagues at the Swiss AI lab IDSIA, developed the long short-term memory (LSTM) algorithm in the early 1990s. This is a type of recurrent neural network that can process and predict sequences of data. LSTMs have become a fundamental building block of modern deep learning systems and have been used in many applications, including speech recognition, natural language processing, and image captioning. 2.
- Early work on neural networks and reinforcement learning: Schmidhuber has been at the forefront of research on artificial neural networks and reinforcement learning since the 1990s. He has made significant contributions to the field of deep learning, including the development of deep neural networks and the exploration of reinforcement learning methods for training them.
- Founding of NNAISENSE: In 2014, Schmidhuber co-founded NNAISENSE, a company focused on developing practical applications of artificial intelligence. The company has since become a leader in the field of AI-powered automation and has worked with a variety of industries, including manufacturing, logistics, and finance. NNAISENSE’s goal is to create intelligent machines that can learn and adapt to new tasks and environments in real-time.
Some key themes discussed in the dialogue.
Credit assignment
During the debate Schmidhuber developed a different interpretation of Credit assignment pointing to the practice of finding patterns in historic data and figuring out how certain events were enabled by previous events. Historians do it. Physicists do it. AIs do it, too. He looked at AI in the broadest historical context of all time since the Big Bang and found a pattern of evolution that will lead to the year of the Omega (2040). Credit assignment is a concept associated with his work and for which he is well known. In this more technical context the problem of credit assignment is determining how much credit or blame should be assigned to the various components of a learning system for achieving a specific outcome. Credit assignment in the context of machine learning and artificial neural networks refers to the process of determining how much credit each neuron or connection in the network should receive for producing a specific output. According to Jürgen Schmidhuber, credit assignment is a fundamental challenge for intelligent systems. An agent or system must be able to assign credit or blame to the actions and components that contributed to a specific outcome in order to learn and improve. This is especially difficult in complex environments where many factors may contribute to a given outcome and the consequences of an action may not be immediately apparent. According to Schmidhuber, the problem of credit assignment can be solved by combining gradient-based learning algorithms and reinforcement learning techniques. Agents can learn to assign credit or blame to the system components that are most responsible for achieving a specific outcome by using feedback from the environment to guide the learning process. This method has proven to be effective in a wide range of applications, including image recognition, natural language processing, and robotics.
Synthetic data
Data would be collected from a variety of sources, including synthetic data through simulations, experiments, or models, or use data labeling and augmentation techniques to increase the quality and quantity of data available for training AI models. There are several models that could be used by AI applications based on the process of collecting data from a variety of sources, including synthetic data through simulations, experiments, or models, or using data labeling and augmentation techniques to increase the quality and quantity of data available for training AI models. Schmidhuber’s Long Short-Term Memory (LSTM) network, a recurrent neural network that processes sequential data like text, speech, and video, is one of his most important contributions. Language translation, speech recognition, and image captioning use LSTMs. Reinforcement learning algorithms, which reward and punish an AI agent for good and bad decisions, are another area where Schmidhuber has contributed. Game-playing, robotics, and autonomous driving use reinforcement learning. Schmidhuber believes AI systems should design and improve their own algorithms and architectures. This “self-referential” approach could lead to the development of artificial general intelligence (AGI), which could perform a wide range of intellectual tasks at a human-level or beyond.
The year of the Omega (2040).
Jürgen Schmidhuber has proposed the concept of «Omega,» which he defines as the limit of the fastest possible self-improvement of an intelligent agent, Omega represents the point at which an AI system is improving itself pushing towards the limit of what is possible, Trough a series of patterns recognitions in the history of the universe Schmidhuber has speculated that the development of advanced AI systems could lead to a singularity event, in which the pace of technological progress accelerates rapidly and fundamentally changes the nature of human civilization. This singularity event could occur in 2040 as In fact, history seems to converge in an Omega point in the year 2040 or so., and its outcome is uncertain and what might happen after that is uncertain.
Francesco Lapenta
Jurgen Schmidhuber and Francesco Lapenta at the History of AI roundtbale