Abstract : Scientific Machine Learning is a new discipline that integrates traditional scientific computing and modern machine learning. It has grown explosively in recent years and is recognized as a fundamental research field that develops computational tools enabling Digital Twins. This mini-symposium will highlight recent progress in scientific machine learning techniques for Digital Twins and gather experts working on theory, algorithms, and applications to discuss and identify urgent current agendas and challenges.
[03527] Digital Twins and Machine Learning from an Inverse Problem Perspective
Author(s) :
Mark Asch (Université de Picardie Jules Verne)
Abstract : Digital Twins that are exchanging data with their real-world counterparts can be considered as instances of inverse
problems—either of parameter identification type (static, or quasi-static), or of data assimilation type (dynamic).
Many classical methods exist for solving inverse problems, but inverse problems remain complex, time-consuming
and difficult to solve, especially with limited resources. However, machine learning can also be viewed as a parameter
identification inverse problem, where for example in a neutral network we seek to identify the weights (parameters) of
the network. Moreover, the machine learning community has developed extremely efficient frameworks and tools for
solving their inverse problems, such as stochastic gradient, backpropagation, and others. In this talk I will review
machine learning methods with respect to the solution of inverse problems and I will present some examples of
Digital Twins that have been developed on this basis.
[03528] Physics-guided data-driven simulations for a digital twin
Author(s) :
Youngsoo Choi (Lawrence Livermore National Laboratory)
Abstract : A computationally expensive physical simulation is a huge bottleneck for a digital twin. Fortunately, many data-driven approaches have emerged to accelerate those simulations, thanks to the recent advance in machine learning (ML) and artificial intelligence. For example, a well-trained 2D convolutional deep neural network can predict the solution of complex Richtmyer–Meshkov instability problem with a speed-up of 100,000x (1). However, the traditional black-box ML models do not incorporate existing governing equations, which embed underlying physics, such as conservation of mass, momentum, and energy. Therefore, the black-box ML models often violate important physics law, which greatly concerns physicists, and require big data to compensate the missing physics information. Additionally, it comes with other disadvantages, such as non-structure-preserving, computationally expensive training phase, non-interpretability, and vulnerability in extrapolation. To resolve these issues, we can bring physics into data-driven framework. Physics can be incorporated in different stages of data-driven modeling, i.e., sampling stage and model-building stage. Physics-informed greedy sampling procedure minimizes the number of required training data for a target accuracy (2). Physics-guided data-driven model better preserves physical structure and more robust in extrapolation than traditional black-box ML models. Numerical results, e.g., hydrodynamics (3,4), particle transport (5), plasma physics, and 3D printing, will be shown to demonstrate the performance of the data-driven approaches. The benefits of the data-driven approaches will also be illustrated in multi-query decision-making applications, such as design optimization (6,7).
Reference:
(1) Jekel, Charles F., Dane M. Sterbentz, Sylvie Aubry, Youngsoo Choi, Daniel A. White, and Jonathan L. Belof. "Using Conservation Laws to Infer Deep Learning Model Accuracy of Richtmyer-meshkov Instabilities." arXiv preprint arXiv:2208.11477 (2022).
(2) He, Xiaolong, Youngsoo Choi, William D. Fries, Jon Belof, and Jiun-Shyan Chen. "gLaSDI: Parametric Physics-informed Greedy Latent Space Dynamics Identification." arXiv preprint arXiv:2204.12005 (2022).
(3) Copeland, Dylan Matthew, Siu Wun Cheung, Kevin Huynh, and Youngsoo Choi. "Reduced order models for Lagrangianhydrodynamics." Computer Methods in Applied Mechanics and Engineering 388 (2022): 114259.
(4) Kim, Youngkyu, Youngsoo Choi, David Widemann, and Tarek Zohdi. "A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder." Journal of Computational Physics 451 (2022): 110841.
(5) Choi, Youngsoo, Peter Brown, William Arrighi, Robert Anderson, and Kevin Huynh. "Space–time reduced order model for large-scale linear dynamical systems with application to boltzmann transport problems." Journal of Computational Physics 424 (2021): 109845.
(6) McBane, Sean, and Youngsoo Choi. "Component-wise reduced order model lattice-type structure design." Computer methods in applied mechanics and engineering 381 (2021): 113813.
(7) Choi, Youngsoo, Gabriele Boncoraglio, Spenser Anderson, David Amsallem, and Charbel Farhat. "Gradient-based constrained optimization using a database of linear reduced-order models." Journal of Computational Physics 423 (2020): 109787.
[03532] Attributing anomalies from black-box predictions
Author(s) :
Tsuyoshi Ide (IBM Research)
Abstract : One of the most important problems with digital twins is how to explain an unusual event observed as a significant discrepancy from the prediction of an AI model. Although this problem encompasses various different scenarios, we are particularly interested in the task of anomaly attribution in the black-box regression setting. The question is how we can quantify the contribution of each input variable in the face of an unexpected deviation between observation and prediction.
In this talk, I will first review existing attribution approaches recently developed in the machine learning community, including linear surrogate modeling, Shapley values, and integrated gradient. After summarizing the challenges of these methods in the particular context of anomaly explanation, I will touch upon a newer notion of likelihood compensation as one of the major counterfactual-type explanations. If time permits, I will share some experimental results, including the one conducted for IBM IoT Business Unit.
[03536] AI Based Medical Twin System: Investigation on Focused Ultrasound Therapeutics
Author(s) :
Kyungho Yoon (Yonsei University)
Abstract : In the huge paradigm shift of 4th industrial revolution driven by AI technique, the medical industry is also pursuing personalized precision medicine through digital transformation and smart transformation from analogue, standard, and empirical medical procedure. At the center of this change, IT technology is acting as key driving force. In particular, personalized digital twin model of human body and therapeutic tool will be the core engine of smart medicine. In this presentation, I will introduce research on the development of artificial intelligence-based medical twin systems for digital/smart therapeutics by convergence of computational science, medical engineering, and artificial intelligence technologies as the core technologies of the 4th medical revolution. In particular, investigation on the focused ultrasound device will be presented, which has recently been emerging as a non-invasive brain stimulation tool.
[03635] Learning reduced-order operators with Bayesian inference and Gaussian processes
Author(s) :
Mengwu Guo (University of Twente)
Abstract : Credible real-time simulation is a critical enabling factor for digital twin technology, and data-driven model reduction is a natural choice for achieving it. In this talk, we will discuss a probabilistic strategy for the learning of reduced-order representations of high-dimensional dynamical systems, with which a significantly reduced dimensionality guarantees improved efficiency, and the endowed uncertainty quantification certifies computational reliability. The strategy is based on Bayesian reduced-order operator inference, a data-driven method that inherits the formulation structure of projection-based reduced-state governing equations yet without requiring access to full-order solvers. The reduced-order operators are estimated using Bayesian inference with Gaussian priors, and two fundamentally different strategies of likelihood definition will be discussed – one formulated as linear regression, and the other through Gaussian processes. Given by posterior Gaussian distributions conditioning on solution data, the reduced-order operators probabilistically define a low-dimensional dynamical system for the predominant latent states, and provide an inherently embedded Tikhonov regularization together with a quantification of modeling uncertainties.
[04330] Online Sparse Identification of Dynamical Systems with Regime Switching by Causation Entropy Boosting
Author(s) :
Chuanqi Chen (University of Wisconsin-Madison)
Nan Chen (University of Wisconsin-Madison)
Jinlong Wu ((University of Wisconsin-Madison)
Abstract : Online nonlinear system identification with sequential data has recently become important in many applications, e.g., extreme weather events, climate change, and autonomous systems. In this work, we developed a causation entropy boosting (CEBoosting) framework for online nonlinear system identification. For each sequential data batch, this framework calculates the causation entropy that evaluates the contribution of each function in a large set of candidate functions to the system dynamics. The causation entropies based on multiple data batches are then aggregated to identify a few candidate functions that have significant impacts on the system dynamics. With the identified sparse set of functions, the framework further fits a model of the system dynamics. The results show that the CEBoosting method can capture the regime switching and then fit models of system dynamics for various types of complex dynamical based on a limited amount of sequential data.
[05255] ClimaX: A foundation model for weather and climate
Author(s) :
Johannes Brandstetter (Microsoft Research)
Abstract : Most state-of-the-art approaches for weather and climate modeling are based on physics-informed numerical models of the atmosphere. These approaches aim to model the non-linear dynamics and complex interactions between multiple variables, which are challenging to approximate. Additionally, many such numerical models are computationally intensive, especially when modeling the atmospheric phenomenon at a fine-grained spatial and temporal resolution. Recent data-driven approaches based on machine learning instead aim to directly solve a downstream forecasting or projection task by learning a data-driven functional mapping using deep neural networks. However, these networks are trained using curated and homogeneous climate datasets for specific spatiotemporal tasks, and thus lack the generality of numerical models. We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatio-temporal coverage, and physical groundings. ClimaX extends the Transformer architecture with novel encoding and aggregation blocks that allow effective use of available compute while maintaining general utility. ClimaX is pre-trained with a self-supervised learning objective on climate datasets derived from CMIP6. The pre-trained ClimaX can then be fine-tuned to address a breadth of climate and weather tasks, including those that involve atmospheric variables and spatio-temporal scales unseen during pretraining. Compared to existing data-driven baselines, we show that this generality in ClimaX results in superior performance on benchmarks for weather forecasting and climate projections, even when pretrained at lower resolutions and compute budgets.
