Denny Wu

About Me

I am a faculty fellow at the Center for Data Science, New York University and the Flatiron Institute.

I obtained my Ph.D. in computer science from the University of Toronto and the Vector Institute, under the supervision of Jimmy Ba and Murat A. Erdogdu. Before that, I was an undergraduate student at Carnegie Mellon University advised by Ruslan Salakhutdinov. I also frequently visit the University of Tokyo and RIKEN AIP, where I am now working with Taiji Suzuki.

My current research focuses on developing a theoretical understanding (e.g., optimization and generalization) of modern machine learning systems, especially overparameterized models such as neural networks, using tools from high-dimensional statistics.

[CV] [Google Scholar]

Selected Publications

• Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit.
  Jason D. Lee*, Kazusato Oko*, Taiji Suzuki*, and Denny Wu*.
  Preprint.

• Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinations.
  Kazusato Oko*, Yujin Song*, Taiji Suzuki*, and Denny Wu*.
  COLT 2024.

• Nonlinear spiked covariance matrices and signal propagation in deep neural networks.
  Zhichao Wang, Denny Wu, and Zhou Fan.
  COLT 2024.

• Learning in the presence of low-dimensional structure: a spiked random matrix perspective.
  Jimmy Ba*, Murat A. Erdogdu*, Taiji Suzuki*, Zhichao Wang*, and Denny Wu*.
  NeurIPS 2023.

• Convergence of mean-field Langevin dynamics: time-space discretization, stochastic gradient, and variance reduction.
  Taiji Suzuki, Denny Wu, and Atsushi Nitanda.
  NeurIPS 2023.

• High-dimensional asymptotics of feature learning: how one gradient step improves the representation.
  Jimmy Ba*, Murat A. Erdogdu*, Taiji Suzuki*, Zhichao Wang*, Denny Wu*, and Greg Yang*.
  NeurIPS 2022.

• Convex analysis of the mean-field Langevin dynamics.
  Atsushi Nitanda, Denny Wu, and Taiji Suzuki.
  AISTATS 2022.

• When does preconditioning help or hurt generalization?
  Shun-ichi Amari*, Jimmy Ba*, Roger Grosse*, Xuechen Li*, Atsushi Nitanda*, Taiji Suzuki*, Denny Wu*, and Ji Xu*.
  ICLR 2021.

• On the optimal weighted $\ell_2$ regularization in overparameterized linear regression.
  Denny Wu* and Ji Xu*.
  NeurIPS 2020.

• Stochastic Runge-Kutta accelerates Langevin Monte Carlo and beyond.
  Xuechen Li, Denny Wu, Lester Mackey, and Murat A. Erdogdu.
  NeurIPS 2019.

Misc.

I like Wes Anderson.

I also like Japanese food.