Machine learning × Physics

Benjamin Scellier

I am a machine learning research scientist.

Portrait of Benjamin Scellier

Research Interests

My research interests lie at the interface of machine learning (ML) and physics. I am more particularly interested in physics-based computation and learning.

Much of my current research revolves around equilibrium propagation (EP), a learning framework grounded in physical principles. Unlike the more conventional ML framework based on automatic differentiation ("backpropagation"), EP performs inference and extracts gradient using the same physical laws, which makes it a potentially useful framework for the design of energy-efficient hardware for ML.

For a very brief mathematical introduction to EP, see these notes. (For a more detailed introduction to EP, Chapter 2 of my PhD thesis may be relevant.) For EP-related code, click here. If you are interested in my research, feel free to reach out.

Recent Articles and Preprints

  1. Training of physical neural networks A review paper of training methods for physical systems.
  2. Energy-based learning algorithms for analog computing: a comparative study Clarifies the differences between EP and other contrastive learning algorithms, and compares them in theory and in numerical experiments.
  3. EP in nonlinear resistive networks: Training end-to-end analog neural networks with equilibrium propagation, Universal approximation theorem for nonlinear resistive networks, A fast algorithm to simulate nonlinear resistive networks.
  4. Quantum equilibrium propagation: gradient-descent training of quantum systems A version of EP for quantum systems.
  5. Training a convergent energy transformer with equilibrium propagation Applying EP to transformer-like models.
  6. Training a predictive coding network on ImageNet with equilibrium propagation Scaling EP to full-size ImageNet classification.
  7. Agnostic physics-driven deep learning Introduces Agnostic EP, a version of EP for black box systems in which the weights are updated directly via physical dynamics.