Abstract : Tensor decompositions are a foundational unsupervised machine learning method for data science, with applications across all of science and engineering. Traditionally, tensor decompositions seek low-rank tensors that best fit the data with respect to the least squares loss. However, other choices of loss function can be more appropriate for non-Gaussian data such as count data, binary data, and data with outliers. This minisymposium presents state-of-the-art advances in developing efficient algorithms and rigorous theory for tensor decompositions with respect to general losses.
[05616] Generalized Canonical Polyadic Tensor Decomposition: Algorithms and Applications
Format : Talk at Waseda University
Author(s) :
David Hong (University of Delaware)
Abstract : Canonical polyadic (CP) tensor decomposition is a fundamental method for finding underlying patterns in data tensors by fitting a low-rank tensor to the data with respect to the least-squares loss. This talk gives an introduction to the generalized CP (GCP) tensor decomposition, which allows general user-selected losses (i.e., non-Gaussian/non-Euclidean/non-least-squares losses). We will describe first-order algorithms for computing GCP and show various demonstrations of GCP on data from science and engineering.
[05224] Recent Improvements in CP Poisson Tensor Algorithms
Format : Online Talk on Zoom
Author(s) :
Jeremy Myers (Sandia National Laboratories)
Daniel Dunlavy (Sandia National Laboratories)
Abstract : The challenge of fitting a low-rank, non-negative canonical polyadic tensor model to Poisson-distributed multi-way data is often formulated as a nonconvex optimization problem. A common approach is to use local methods initialized from many random starting points to improve the probability of convergence to the global minimum, which is costly. Our mitigation is a heuristic that identifies and teleports away from suboptimal solutions to improve the probability of convergence and reduce computational cost.
[04808] Second-order algorithms for canonical polyadic decomposition with non-least-squares cost functions
Format : Talk at Waseda University
Author(s) :
Michiel Vandecappelle (Televic Rail NV)
Nico Vervliet (KU Leuven)
Lieven De Lathauwer (KU Leuven)
Abstract : Signal processing and data analysis applications often rely on rank-1 terms to extract meaningful information from tensor data. By using the least-squares loss when computing this canonical polyadic decomposition, one implicitly assumes normally distributed errors, which might not be suitable for, e.g., count data. Therefore, we derive a generalized Gauss-Newton-type algorithm for non-least-squares loss functions and discuss how exploiting tensor structure and randomization lead to an efficient algorithm.
[05553] Efficient Algorithms and Software for Generalized Tensor Completion
Format : Online Talk on Zoom
Author(s) :
Navjot Singh (University of Illinois Urbana-Champaign)
Edgar Solomonik (University of Illinois at Urbana-Champaign)
Abstract : We present novel algorithms and systems infrastructure which enable efficient parallel implementation of algorithms for CP tensor completion with generalized loss functions.
Specifically, we consider alternating minimization, coordinate minimization, and a quasi-Newton (generalized Gauss-Newton) method.
By extending the Cyclops library, we implement all these methods in high-level Python syntax using new sparse tensor kernels. We demonstrate the generalizability of our framework with the first large-scale implementation for Poisson loss completion.
[03326] Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses
Format : Talk at Waseda University
Author(s) :
Wenqiang PU (Shenzhen Research Institute of Big Data)
Xiao FU (Oregon State University)
Abstract : This talk considers low-rank canonical polyadic decomposition (CPD) under a class of non-Euclidean loss functions that frequently arise in statistical machine learning and signal processing. These loss functions are often used for certain types of tensor data, e.g., count and binary tensors, where the least squares loss is considered unnatural. Compared to the least squares loss, the non-Euclidean losses are generally more challenging to handle. Non-Euclidean CPD has attracted considerable interests and a number of prior works exist. However, pressing computational and theoretical challenges, such as scalability and convergence issues, still remain. This talk offers a unified stochastic algorithmic framework for large-scale CPD decomposition under a variety of non-Euclidean loss functions. Our key contribution lies in a tensor fiber sampling strategy-based flexible stochastic mirror descent framework. Leveraging the sampling scheme and the multilinear algebraic structure of low-rank tensors, the proposed lightweight algorithm ensures global convergence to a stationary point under reasonable conditions. Numerical results show that our framework attains promising non-Euclidean CPD performance. The proposed framework also exhibits substantial computational savings compared to state-of-the-art methods.
[03666] Generalized Tucker tensor estimation: An optimal statistical and computational framework
Format : Online Talk on Zoom
Author(s) :
Anru Zhang (Duke University)
Rungang Han (Duke University)
Rebecca Willett (University of Chicago)
Abstract : We describe a flexible framework for generalized low-rank tensor estimation problems. The proposed estimator consists of finding a low Tucker rank tensor fit to the data under generalized parametric models. To overcome the difficulty of nonconvexity, we introduce a unified approach of projected gradient descent. We establish both an upper bound on the statistical error and the linear rate of computational convergence. We demonstrate the superiority of the proposed framework on real data.
