Special Year on Optimization, Statistics, and Theoretical Machine Learning
The special year was led by Sanjeev Arora, Charles Fitzmorris Professor of Computer Science at Princeton University, with a dual appointment at IAS in 2017-2019 (Visiting Professor) and 2019-2020 (Distinguished Visiting Professor).
Special Year Calendar of Events
The program hosted 18 members (some of them postdocs) in each term. They represented various strands of research under the broad umbrella of Optimization, Statistics and Machine Learning. A key goal of the program was to explore how these traditional disciplines are changing as they respond to modern challenges, especially deep learning.
Members were selected in Winter 2018 and organization of various workshops and some key colloquia started in Spring 2019.
Monday: Lunch together in the Dining Hall
Tuesday and Wednesday (Fall) or Tuesday and Thursday (Spring): Group seminars with lunch served. Tuesdays usually featured an outside speaker and/or were hosted off-campus (see joint seminars below).
Monday (Fall) or Tuesday (Spring): Afternoon informal study group/ “open blackboard” session. This rotated among a few topics and settled in spring (thanks to good organizing by Samory Kpotufe of Columbia) on a study of how to develop theory for domain adaptation.
Of course, Special-Year Members would also meet daily over cookies at the famous afternoon tea in Fuld Hall.
The seminar series was organized fantastically by postdoc Chris Maddison from October 2019 - March 2020, and then postdoc Ke Li from April - August 2020.
Monthly Joint Seminar Series:
Two joint seminar series were held joint with talks roughly once per month. Usually Tuesdays, in lieu of the usual seminar talk.
(a) Joint IAS/PNI Seminar (PNI =Princeton Neuroscience Institute) on talks at the interface of Neuroscience and Machine Learning. There were four talks in fall and three in spring before the Covid shutdown. (Unfortunately, no separate webpage for these talks; they are on the list of special year talks above.)
(b) Joint seminar with Princeton Center for Theoretical Science (PCTS), on “Deep learning for physics.”
Sanjeev Arora led off in September with a survey talk on deep learning and its mysteries. Then each month there were two speakers giving one hour talks each.
A crowd-sourced e-book on theory of deep learning. Since many of the best experts were in town for various durations in Fall, we decided to do a seminar course at the university on the topic and collect lecture notes as an e-book. The draft is here
A subset of this group also did a 1-week workshop in Barbados in February 2020 hosted by McGill University and the famous Montreal group in deep learning.This was a true meeting of ideas from theory and practice (Bengio and LeCun were two leading lights from practice) and the practitioners were very intrigued by some of the latest theories being developed. It did make them interested in looking further, and LeCun has already used one of the new theory ideas in a new paper (personal communication).
Scientific Advisory Committee: Michael Jordan (UC Berkeley), Yann LeCun (NYU and Facebook), Yoram Singer (Princeton University and Google Brain), Bin Yu (UC Berkeley)
The following researchers participated:
Fall: Raman Arora, Laura Balzano, Guang Cheng, Yu Cheng, Sanjoy Dasgupta, Simon Du, Rong Ge, Anna Gilbert, Suriya Gunasekar, Chi Jin, Jason Lee, Christopher Maddison, Boaz Nadler, Sushant Sachdeva, Robert Schapire, Zhao Song, Rachel Ward, Jonathan Weed
Spring: Raman Arora, Laura Balzano, Joan Bruna, Costis Daskalakis, Simon Du, Bianca Dumitrascu, Gintare Karolina Dziugaite, Roger Grosse, Adam Klivans, Samory Kpotufe, Ke Li, Christopher Maddison, Daniel Roy, Zhao Song, Jonathan Weed, Bin Yu
Researchers from industry who were unable to commit for a full semester were welcome for shorter visits.
Theoretical Machine Learning Group at Princeton University