[02644] Black box methods for efficient learning in high-dimensional scientific computing
Session Time & Room : 1D (Aug.21, 15:30-17:10) @F310
Type : Proposal of Minisymposium
Abstract : The past decade has seen an explosion of interest in the use of black box modeling techniques in scientific computing. Driven by advances in hardware and software, researchers have begun to harness the power of these techniques to solve a wide range of complex and high-dimensional problems arising in computational science and engineering. This minisymposium aims to bring together experts from various fields including data science, high-dimensional approximation, deep learning, optimization, control, and scientific computing to share their latest research and developments in this exciting and rapidly evolving area of computational science.
Organizer(s) : Nick Dexter, Clayton Webster, Guannan Zhang
Abstract : When approximating smooth, high-dimensional functions from limited samples using polynomials, it is common to use Monte Carlo (MC) sampling to lessen the curse of dimensionality. However, it is well known that MC is theoretically suboptimal. This has led to a concerted effort to design improved strategies with near-optimal sample complexities. In this work we demonstrate, both theoretically and numerically, that MC is actually an eminently suitable strategy in sufficiently high dimension despite its apparent suboptimality.
[05196] CAS4DL: Christoffel Adaptive Sampling for Deep Learning in data-scarce applications
Format : Online Talk on Zoom
Author(s) :
Ben Adcock
Juan M. Cardenas (Simon Fraser University)
Nick Dexter (Florida State University)
Abstract : Many problems in computational science and engineering require the approximation of a high-dimensional function from data. In many such applications, data is costly to generate: for example, it each sample may require a costly PDE solve. Therefore, it is imperative to develop highly sample efficient algorithms. Recently, deep neural networks and deep learning have shown great promise to provide breakthrough performance in challenging function approximation tasks. In this work, we propose an adaptive sampling strategy, CAS4DL (Christoffel Adaptive Sampling for Deep Learning) to increase the sample efficiency of DL. Our novel approach is based on interpreting the second to last layer of a DNN as a dictionary of functions defined by the nodes on that layer. With this viewpoint, we then define an adaptive sampling strategy motivated by adaptive sampling schemes recently proposed for linear approximation schemes, wherein samples are drawn randomly with respect to the Christoffel function of the subspace spanned by this dictionary. We present numerical experiments comparing CAS4DL with standard Monte Carlo (MC) sampling. Our results demonstrate that CAS4DL often yields substantial savings in the number of samples required to achieve a given accuracy, particularly in the case of smooth activation functions. These results therefore are a promising step towards fully adapting DL towards scientific computing applications.
[05283] Exploiting the local parabolic landscapes of adversarial losses to accelerate black-box adversarial attack
Format : Talk at Waseda University
Author(s) :
Hoang Anh Tran (Oak Ridge National Laboratory)
Abstract : Machine learning models, and convolutional neural networks (CNNs) in particular, have demonstrated remarkable performance in many classification tasks. However, deep learning technology also exposed certain security risks, as they are susceptible to malicious inputs, which are small, human-imperceptible perturbations to the inputs designed to fool the model prediction. In this talk, we present an investigation into the vulnerability of CNN classifiers from the shape of the loss’s landscape perspective. We theoretically and experimentally justify that the adversarial losses of many standard and robust image classifiers behave like parabolas with respect to perturbations in the Fourier domain, but not in the pixel domain. Then, we exploit the parabolic landscape to design a new black-box adversarial attack methods with improved query efficiency, compared to the other state-of-the-art baselines. We demonstrate the efficiency of our method on MNIST, CIFAR-10 and ImageNet datasets for various standard and robust models.
[05315] A Mathematical Approach Towards Physical Law Learning
Format : Online Talk on Zoom
Author(s) :
Gitta Kutyniok (LMU Munich)
Philipp Scholl (LMU Munich)
Aras Bacho (LMU Munich)
Holger Boche (TU Munich)
Abstract : For most of human history, scientists had to derive physical laws by hand. Recently, due to the data deluge, several learning-based approaches to infer the governing laws from experimental data have been suggested. However, a theoretical foundation is at present missing. In this talk, we will discuss our first stage of a mathematical framework for physical law learning, in particular, how to derive well-definedness of the learning problem, both theoretically and numerically.
