Abstract : Machine learning is an area characterized by rapid growth, broader impact and diverse audience. It is changing applied mathematics in a fundamentally way. At the same time, there is still a lack of proper venues for publishing research work in machine learning with an applied math orientation. Journal of Machine Learning (JML) is a newly launched journal to provide such a venue. It strives to accommodate both the special features of machine learning mentioned above, as well as the long-held tradition of applied mathematics. In this minisymposium, we invite authors who have published papers in JML to present their work.
Organizer(s) : Weinan E, Bin Dong, Arnulf Jentzen, Zhiqin Xu
[02598] Embedding Principle: A Hierarchical Structure of Loss Landscape of Deep Neural Networks
Format : Talk at Waseda University
Author(s) :
Yaoyu Zhang (Shanghai Jiao Tong University)
Abstract : This talk is about the Embedding Principle of loss landscape of deep neural networks ((NNs)), i.e., loss landscape of an NN "contains" all critical points of all the narrower NNs. We will introduce a general class of embedding operators which map any critical point of a narrower NN to a critical point of the target NN preserving the output. Our results uncover a hierarchical structure of loss landscape special to the deep learning models.
[02247] Perturbational Complexity and Reinforcement Learning in Reproducing Kernel Hilbert Space
Format : Talk at Waseda University
Author(s) :
Jihao Long (Princeton University)
Abstract : This talk will offer some fresh insight into the challenge for analyzing reinforcement learning (RL) in a general reproducing kernel Hilbert space (RKHS). We define a quantity called “perturbational complexity by distribution mismatch” and show that the perturbational complexity gives both the lower bound and upper bound of the error for the RL problem in RKHS. We will provide some concrete examples and discuss whether the complexity decays fast or not in these examples.
[02258] The Random Feature Method for Solving Partial Differential Equations
Format : Talk at Waseda University
Author(s) :
Jingrun Chen (University of Science and Technology of China)
Abstract : In this presentation, we will give a description of the random feature method for solving partial differential equations, including its basic formulation for both static and time-dependent problems and the application for three dimensional problems with complex geometries.
[02957] DeePN$^2$: A deep learning-based non-Newtonian hydrodynamic model
Format : Talk at Waseda University
Author(s) :
Huan Lei
Lidong Fang (Michigan State University)
Pei Ge (Michigan State University)
Lei Zhang (Shanghai Jiao Tong University)
Weinan E (Peking University)
Abstract : A long standing problem in the modeling of non-Newtonian hydrodynamics of polymeric flows is the availability of reliable and interpretable hydrodynamic models that faithfully encode the underlying micro-scale polymer dynamics. We developed a deep learning-based non-Newtonian hydrodynamic model, DeePN$^2$, that enables us to systematically pass the micro-scale structural mechanics information to the macro-scale hydrodynamics for polymer suspensions. The model retains a multi-scaled nature with clear physical interpretation, and strictly preserves the frame-indifference constraints.
[02982] Generalization ability and memorization phenomenon of distribution learning models
Format : Talk at Waseda University
Author(s) :
Hongkang Yang (Princeton University, Program in Applied and Computational Mathematics)
Abstract : Generative models and density estimators suffer from the memorization phenomenon (i.e. convergence to the finite training samples) as training time goes to infinity. This deterioration is in contradiction to the empirical success of models such as StableDiffusion and GPT-3. We resolve this paradox by proving that distribution learning models enjoy implicit regularization during training. Specifically, prior to the onset of memorization, their generalization errors at early-stopping escape from the curse of dimensionality.
[04035] Approximation of Functionals by Neural Network without Curse of Dimensionality
Format : Talk at Waseda University
Author(s) :
Yahong Yang (The Hong Kong University of Science and Technology)
Tianyu Jin (The Hong Kong University of Science and Technology)
Yang Xiang (Hong Kong University of Science and Technology)
Abstract : In this paper, we establish a neural network to approximate functionals, which are maps from infinite dimensional spaces to finite dimensional spaces. The approximation error of the neural network is $O(1/\sqrt{m})$ where m is the size of networks, which overcomes the curse of dimensionality. Then, the proposed method is employed in several numerical experiments, such as evaluating the energy functionals and solving Poisson equations by the aforementioned network at one or a few given points.
[02256] Approximation of Functionals by Neural Network without Curse of Dimensionality
Format : Talk at Waseda University
Author(s) :
Yahong Yang (Hong Kong University of Science and Technology)
Yang Xiang (Hong Kong University of Science and Technology)
Abstract : In this paper, we establish a neural network to approximate functionals, which are maps from infinite dimensional spaces to finite dimensional spaces. The approximation error of the neural network is $O(1/\sqrt{m})$ where $m$ is the size of networks. In other words, the error of the network is no dependence on the dimensionality respecting to the number of the nodes in neural networks. The key idea of the approximation is to define a Barron space of functionals.
[02998] Ab-Initio Study of Interacting Fermions at Finite Temperature with Neural Canonical Transformation
Format : Talk at Waseda University
Author(s) :
Hao Xie (Institute of Physics, Chinese Academy of Sciences)
Linfeng Zhang (DP Technology/AI for Science Institute)
Lei Wang (Institute of Physics, Chinese Academy of Sciences)
Abstract : We present a variational density matrix approach to the thermal properties of interacting fermions in the continuum. The variational density matrix is parametrized by a permutation equivariant many-body unitary transformation together with a discrete probabilistic model. The unitary transformation is imple- mented as a quantum counterpart of neural canonical transformation, which incorporates correlation effects via a flow of fermion coordinates. As the first application, we study electrons in a two-dimensional quan- tum dot with an interaction-induced crossover from Fermi liquid to Wigner molecule. The present approach provides accurate results in the low-temperature regime, where conventional quantum Monte Carlo methods face severe difficulties due to the fermion sign problem. The approach is general and flexible for further ex- tensions, thus holds the promise to deliver new physical results on strongly correlated fermions in the context of ultracold quantum gases, condensed matter, and warm dense matter physics.
