Workshop on Theory of Deep Learning: Where next?

The event was livestreamed.

Organizers:

The workshop was organized by Sanjeev Arora (IAS/Princeton University), Joan Bruna (IAS/NYU), Rong Ge (IAS/Duke), Suriya Gunasekar(IAS/Toyota Technical Institute), Jason Lee (IAS/USC), Bin Yu (IAS/UC Berkeley)

This workshop sought to bring together deep learning practitioners and theorists to discuss progress that has been made on deep learning theory, and to identify promising avenues where theory is possible and useful. There were several invited talks each day and also spotlight talks by young researchers.

The workshop was free of charge thanks to the support from the Institute for Advanced Study and the Schwab Charitable Fund made possible by the generosity of Eric and Wendy Schmidt.

Invited Speakers who confirmed participation:

Anima Anandkumar, Raman Arora, Sanjeev Arora, Mikhail Belkin, Léon Bottou, Joan Bruna, Michael Collins, Simon Du, Gintare Karolina Dziugaite, Surya Ganguli, Rong Ge, Suriya Gunasekar, Stefanie Jegelka, Chi Jin, Sham Kakade, Yann LeCun, Jason Lee, Ke Li, Tengyu Ma, Aleksander Madry, Chris Manning, Behnam Neyshabur, Dan Roy, Nathan Sbrero, Rachel Ward, Bin Yu

Registration: was required

If you have any problems with registration or other practical questions, please write to Michelle Huguenin (huguenin@ias.edu).

Registration for the meals and workshop are required. (Speakers do not need to register)

Although there is no registration fee, because seating is limited both in the seminar room and in our dining hall, you are required to register if you wish to attend all or part of the workshop. In addition, in order for our chef to prepare enough food, we need to have a headcount for the meals. Please note the workshop attendees are expected to pay for their lunch. Approximate range of prices are 4.00 for a salad to 9.00 for an entree. Vegetarian lunches will be available. Only cash is accepted at the cash registers. Credit cards and personal checks are not accepted.

Agenda:(all talks were in Wolfensohn Hall)

October 15, 2019

REGISTRATION: Opening at 8:30 am - 9:30 am in Wolfensohn Hall and will remain open until 3 pm

Workshop on Theory of Deep Learning: Where next?
Topic: Emergent linguistic structure in deep contextual neural word representations slides video
Speaker: Chris Manning, Stanford University
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
Topic: Explaining landscape connectivity of low-cost solutions for multilayer nets video
Speaker: Rong Ge, Duke University; Member, School of Mathematics
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic: Fixing GAN optimization through competitive gradient descent video
Speaker: Anima Anandkumar, Caltech
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic: Tightening information-theoretic generalization bounds with data-dependent estimates with an application to SGLD video
Speaker: Daniel Roy, University of Toronto
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
Topic: Spotlight Talks: Yuanzhi Li, Soham De, Mahyar Fazlyab, Maithra Raghu, Valentin Thomas video
Speaker: Various
Time/Room: 12:20pm - 1:00 pm
Topic: Is optimization the right language to understand deep learning? video
Speaker: Sanjeev Arora, Princeton University; Distinguished Visiting Professor, School of Mathematics
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
Topic: Spotlight Talks: Amir Asadi, Dimitris Kalimeris video
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
Topic: PAC-Bayesian approaches to understanding generalization in deep learning video slides
Speaker: Gintare Karolina Dziugaite, Simons Institute for the Theory of Computing
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
Topic: Overcoming the Curse of Dimensionality and Mode Collapse video slides
Speaker: Ke Li, University of California, Berkeley
Time/Room: 4:30pm - 5:00pm/Wolfensohn Hall
Topic: Are All Features Created Equal? video slides
Speaker: Aleksander Madry, Massachusetts Institute of Technology
Time/Room: 5:00pm - 5:40pm/Wolfensohn Hall

October 16, 2019

Topic: Energy-based Approaches to Representation Learning slides video
Speaker: Yann LeCun, NYU and Facebook AI
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
Topic: On Large Deviation Principles for Large Neural Networks video slides
Speaker: Joan Bruna, New York University
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic: Neural Models for Speech and Language: Successes, Challenges, and the Relationship to Computational Models of the Brain video
Speaker: Michael Collins, Columbia University
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic: On the Connection between Neural Networks and Kernels: a Modern Perspective video
Speaker: Simon Du, Member, School of Mathematics
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
Topic: Hike in Institute Woods
12:20 pm-1:00 pm
Topic: From Classical Statistics to Modern ML: the Lessons of Deep Learning video slides
Speaker: Mikhail Belkin, Ohio State University
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
Topic: Spotlight Talks: Vaishnavh Nagarajan, Preetum Nakkiran, Xiaowu Dai, Weijie Su video
Speaker: Various
Time/Room: 3:10pm-3:40pm/Wolfensohn Hall
Topic: Towards a theoretical foundation of neural networks video
Speaker: Jason Lee, Princeton University; Member, School of Mathematics
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
Topic: Panel Session
Time/Room: 4:30pm - 5:30pm/Wolfensohn Hall

October 17, 2019

Topic: Learning Representations Using Causal Invariance video
Speaker: Leon Bottou, Facebook AI Research
Time/Room: 9:30am - 10:10am/Wolfensohn Hall
Topic: Understanding the inductive bias due to dropout video
Speaker: Raman Arora, Johns Hopkins University; Member, School of Mathematics
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic: Interpreting Deep Neural Networks video slides
Speaker: Bin Yu, University of California, Berkeley
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic: Designing explicit regularizers for deep models video slides
Speaker: Tengyu Ma
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
Topic: Spotlight Talks: Arjun Nitin Bhagoji, Jiaoyang Huang, Rosemary Ke, Or Sharir, Omar Shehab
Speaker: Various video
Time/Room: 12:20pm - 1:00pm/Wolfensohn Hall
Topic: Kernel and Rich Regimes in Deep Learning video
Speaker: Nati Srebro, TTIC
Time/Room: 2:30pm - 3:10pm/Wolfensohn Hall
Topic: Spotlight Talks: Sebastian Goldt video
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
Topic: Provably Efficient Reinforcement Learning with Linear Function Approximation video
Speaker: Chi Jin, Member, School of Mathematics
Time/Room: 4:00pm - 4:30pm/Wolfensohn Hall
Topic: Poster Session
Speaker: Various
Time/Room: 4:30pm - 5:30pm/Wolfensohn Hall

October 18, 2019

Topic: Reinforcement Learning, Deep Learning,
and the Role of Policy Gradient Methods video
Speaker: Sham Kakade, University of Washington
Time/Room: 9:30am - 10:10 am/Wolfensohn Hall
Topic: Statistical Mechanics of Machine Learning video
Speaker: Surya Ganguli, Stanford University
Time/Room: 10:10am - 10:40am/Wolfensohn Hall
Topic: Concentration inequalities for random matrix products
Speaker: Rachel Ward, The University of Texas at Austin; von Neumann Fellow, School of Mathematics
Time/Room: 11:10am - 11:50am/Wolfensohn Hall
Topic: Representational Power of Graph Neural Networks video slides
Speaker: Stefanie Jegelka, Massachusetts Institute of Technology
Time/Room: 11:50am - 12:20pm/Wolfensohn Hall
Topic: Spotlight Talks: Zhiyuan Li, John Zarka, Stanislav Fort video
Speaker: Various
Time/Room: 12:20pm - 1:00pm/Wolfensohn Hall
Topic: Toward a Causal Analysis of Generalization in Deep Learning video
Speaker: Behnam Neyshabur, Google
Time/Room: 2:30pm - 3:00pm/Wolfensohn Hall
Topic: Spotlight Talks: Zhifeng Kong, Daniel Paul Kunin, Omar Montasser video
Speaker: Various
Time/Room: 3:10pm - 3:40pm/Wolfensohn Hall
Topic: Informal discussion sessions
Time/Room: 3:40pm - 5:30pm/Wolfensohn Hall

~end

Contributed talks: Deadline was Sept 2

A few shorter slots were given to showcase late-breaking results and work by young researchers (grads and postdocs).

Selected papers were invited as either talks or posters. Notification deadline was Sept 10.

Travel Information:

Directions to IAS (driving, train and plane): https://www.ias.edu/about/maps-directions

Places to Stay:

  • The Peacock Inn (1.5 miles from IAS) 20 Bayard Lane, Pirnceton, NJ 08540 - 609-924-1707
  • Nassau Inn (1.6 miles from IAS) 10 Palmer Square, Princeton, NJ 08542 - 609-921-7500
  • Hyatt Regency (3.1 miles from IAS) 102 Carnegie Center Drive, Princeton, NJ 08540 - 609-987-1234
  • Marriott Residence Inn (3.7 miles from IAS) 3563 US Route 1, Princeton, NJ 08540 - 609-799-0550
  • Hyatt Place (3.8 miles from IAS) 3565 US Highway 1, Princeton, NJ 08540 - 609-720-0200
  • Extended Stay America (4.4 miles from IAS) 3450 US Highway 1, Princeton, NJ 08540 - 609-919-9000
  • Courtyard Marriott (5.3 Miles from IAS) 3815 US Highway 1 at Mapleton Road, Princeton, NJ 08540 - 609-716-9100
  • Homewood Suites (5.7 miles from IAS) 3819 US Highway 1, Princeton, NJ 08540 - 609-720-0550
  • DoubleTree (5.9 miles from IAS) 4355 US Highway 1, Princeton, NJ 08540 - 609-452-2400

Please inquire with the hotel if they offer shuttle service to the local Princeton area.

Date & Time

October 15, 2019 | 9:00am – October 18, 2019 | 6:00am

Categories