DeepMind’s latest research at ICLR 2023 . Research towards AI models that can generalise, scale, and accelerate science

Research towards AI models that can generalise, scale, and accelerate science

Next week marks the start of the 11th International Conference on Learning Representations (ICLR), taking place 1-5 May in Kigali, Rwanda. This will be the first major artificial intelligence (AI) conference to be hosted in Africa and the first in-person event since the start of the pandemic. 

Researchers from around the world will gather to share their cutting-edge work in deep learning spanning the fields of AI, statistics and data science, and applications including machine vision, gaming and robotics. We’re proud to support the conference as a Diamond sponsor and DEI champion. 

Teams from across DeepMind are presenting 23 papers this year. Here are a few highlights:

Open questions on the path to AGI

Recent progress has shown AI’s incredible performance in text and image, but more research is needed for systems to generalise across domains and scales. This will be a crucial step on the path to developing artificial general intelligence (AGI) as a transformative tool in our everyday lives.

We present a new approach where models learn by solving two problems in one. By training models to look at a problem from two perspectives at the same time, they learn how to reason on tasks that require solving similar problems, which is beneficial for generalisation. We also explored the capability of neural networks to generalise by comparing them to the Chomsky hierarchy of languages. By rigorously testing 2200 models across 16 different tasks, we uncovered that certain models struggle to generalise, and found that augmenting them with external memory is crucial to improve performance.

Another challenge we tackle is how to make progress on longer-term tasks at an expert-level, where rewards are few and far between. We developed a new approach and open-source training data set to help models learn to explore in human-like ways over long time horizons.

Innovative approaches

As we develop more advanced AI capabilities, we must ensure current methods work as intended and efficiently for the real world. For example, although language models can produce impressive answers, many cannot explain their responses. We introduce a method for using language models to solve multi-step reasoning problems by exploiting their underlying logical structure, providing explanations that can be understood and checked by humans. On the other hand, adversarial attacks are a way of probing the limits of AI models by pushing them to create wrong or harmful outputs. Training on adversarial examples makes models more robust to attacks, but can come at the cost of performance on 'regular' inputs. We show that by adding adapters, we can create models that allow us to control this tradeoff on the fly.

Reinforcement learning (RL) has proved successful for a range of real-world challenges, but RL algorithms are usually designed to do one task well and struggle to generalise to new ones. We propose algorithm distillation, a method that enables a single model to efficiently generalise to new tasks by training a transformer to imitate the learning histories of RL algorithms across diverse tasks. RL models also learn by trial and error which can be very data-intensive and time-consuming. It took nearly 80 billion frames of data for our model Agent 57 to reach human-level performance across 57 Atari games. We share a new way to train to this level using 200 times less experience, vastly reducing computing and energy costs.