Gal Vardi

I am a postdoc at TTI-Chicago and the Hebrew University, hosted by Nati Srebro and Amit Daniely as part of the NSF/Simons Collaboration on the Theoretical Foundations of Deep Learning. Prior to that, I was a postdoc at the Weizmann Institute, hosted by Ohad Shamir, and a PhD student at the Hebrew University advised by Orna Kupferman.

My research focuses on theoretical machine learning, with an emphasis on deep-learning theory.

Email: galvardi at ttic dot edu

Papers

(* = Equal contribution or alphabetical order)

Noisy Interpolation Learning with Shallow Univariate ReLU Networks

Nirmit Joshi, Gal Vardi and Nathan Srebro

To appear in ICLR 2024 (Selected as Spotlight)

An Agnostic View on the Cost of Overfitting in (Kernel) Ridge Regression

Lijia Zhou, James B. Simon, Gal Vardi and Nathan Srebro

To appear in ICLR 2024

Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data

Zhiwei Xu, Yutong Wang, Spencer Frei, Gal Vardi and Wei Hu

To appear in ICLR 2024

A preliminary version appeared in M3L Workshop @ NeurIPS 2023 (Selected as Oral)

Most Neural Networks Are Almost Learnable

Amit Daniely*, Nathan Srebro* and Gal Vardi*

NeurIPS 2023

The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks

Spencer Frei*, Gal Vardi*, Peter L. Bartlett and Nathan Srebro

NeurIPS 2023

Computational Complexity of Learning Neural Networks: Smoothness and Degeneracy

Amit Daniely*, Nathan Srebro* and Gal Vardi*

NeurIPS 2023

Adversarial Examples Exist in Two-Layer ReLU Networks for Low Dimensional Data Manifolds

Odelia Melamed, Gilad Yehudai and Gal Vardi

NeurIPS 2023

Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses

Gon Buzaglo*, Niv Haim*, Gilad Yehudai, Gal Vardi, Yakir Oz, Yaniv Nikankin and Michal Irani

NeurIPS 2023

A preliminary version appeared in TrustML-(un)Limited Workshop @ ICLR 2023

Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization

Spencer Frei*, Gal Vardi*, Peter L. Bartlett and Nathan Srebro

COLT 2023

On the Implicit Bias in Deep-Learning Algorithms (a survey)

Gal Vardi

Communications of the ACM (CACM), 2023

Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data

Spencer Frei*, Gal Vardi*, Peter L. Bartlett, Nathan Srebro and Wei Hu

ICLR 2023 (Selected as Spotlight)

Implicit Regularization Towards Rank Minimization in ReLU Networks

Nadav Timor, Gal Vardi and Ohad Shamir

ALT 2023

On Convexity and Linear Mode Connectivity in Neural Networks

David Yunis, Kumar Kshitij Patel, Pedro Savarese, Gal Vardi, Jonathan Frankle, Matthew Walter, Karen Livescu and Michael Maire

OPT Workshop @ NeurIPS 2022

On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias

Itay Safran*, Gal Vardi* and Jason D. Lee

NeurIPS 2022

Reconstructing Training Data from Trained Neural Networks

Niv Haim*, Gal Vardi*, Gilad Yehudai*, Ohad Shamir and Michal Irani

NeurIPS 2022 (Selected as Oral)

See project page

Gradient Methods Provably Converge to Non-Robust Networks

Gal Vardi*, Gilad Yehudai* and Ohad Shamir

NeurIPS 2022

The Sample Complexity of One-Hidden-Layer Neural Networks

Gal Vardi, Ohad Shamir and Nathan Srebro

NeurIPS 2022

On Margin Maximization in Linear and ReLU Networks

Gal Vardi, Ohad Shamir and Nathan Srebro

NeurIPS 2022

Width is Less Important than Depth in ReLU Neural Networks

Gal Vardi*, Gilad Yehudai* and Ohad Shamir

COLT 2022

On the Optimal Memorization Power of ReLU Neural Networks

Gal Vardi*, Gilad Yehudai* and Ohad Shamir

ICLR 2022 (Selected as Spotlight)

Learning a Single Neuron with Bias Using Gradient Descent

Gal Vardi*, Gilad Yehudai* and Ohad Shamir

NeurIPS 2021

From Local Pseudorandom Generators to Hardness of Learning

Amit Daniely* and Gal Vardi*

COLT 2021

Size and Depth Separation in Approximating Benign Functions with Neural Networks

Gal Vardi, Daniel Reichman, Toniann Pitassi and Ohad Shamir

COLT 2021

Implicit Regularization in ReLU Networks with the Square Loss

Gal Vardi and Ohad Shamir

COLT 2021

Neural Networks with Small Weights and Depth-Separation Barriers

Gal Vardi and Ohad Shamir

NeurIPS 2020

Hardness of Learning Neural Networks with Natural Weights

Amit Daniely* and Gal Vardi*

NeurIPS 2020

Perspective Games