[00827] Stochastic Rounding for Reduced-Precision Arithmetic in Scientific Computing
Session Time & Room : 2E (Aug.22, 17:40-19:20) @E709
Type : Proposal of Minisymposium
Abstract : The comeback that stochastic rounding has made in the last few years can be attributed to the availability of hardware implementing low-precision floating-point arithmetic, as well as to the recognition that, in some applications, this rounding mode can control the growth of rounding errors better than commonly used alternatives. Research has focused not only on obtaining efficient hardware and software implementations, but also on understanding the numerical properties of algorithms that replace round-to-nearest with stochastic rounding. In this minisymposium, we will have an opportunity to learn about recent advances in both directions.
Abstract : In this talk we will review the latest developments in implementing stochastic rounding. We will first revisit the current methods of implementing stochastic rounding in hardware and software packages, such as CPFloat and MATLAB chop. We will present the main challenges and open problems, such as the reproducibility and precision of pseudo-random numbers. We will also review the commercial hardware that currently includes stochastic rounding, such as Graphcore IPU, Amazon Trainium, and Tesla Dojo devices. Finally, we will outline the list of features required for stochastic rounding to be standardized.
[04133] Software Simulation of Stochastic Rounding
Format : Online Talk on Zoom
Author(s) :
Massimiliano Fasi (Durham University)
Mantas Mikaitis (University of Leeds)
Abstract : Implementing a stochastically rounded mathematical function requires three steps: 1) evaluating the function using a high-precision floating-point arithmetic; 2) drawing a pseudo-random number from some uniform distribution; and 3) rounding the high-precision result to the target precision. We describe how stochastic rounding can be performed using only integer arithmetic and bit-level operations, and we summarize the major challenges and open questions surrounding the implementation of this rounding mode.
[04508] Bounds on Non-linear Errors for Variance Computation with Stochastic Rounding
Format : Talk at Waseda University
Author(s) :
El-Mehdi El Arar (Paris-Saclay University-UVSQ- LI-PaRAD )
Pablo de Oliveira Castro (Paris-Saclay University-UVSQ- LI-PaRAD )
Eric Petit (Intel Corp)
Abstract : This work's main objective is to investigate non-linear errors and pairwise summation using stochastic rounding (SR) in variance computation algorithms. We estimate the forward error of computations under SR through two methods: 1 a bound of the variance and Bienaymé–Chebyshev inequality, 2 martingales and Azuma–Hoeffding inequality. We examine two algorithms, "textbook" and "two-pass", both with non-linear errors. We show that they have probabilistic bounds under SR in $O(\sqrt{n}u)$ instead of $nu$ for the deterministic bounds.
[04837] Trace estimation via asynchronous stochastic rounding
Format : Online Talk on Zoom
Author(s) :
Lior Horesh (IBM T. J. Watson Research Center)
Vasileios Kalantzis (IBM T. J. Watson Research Center)
Georgios Kollias (IBM T. J. Watson Research Center)
Shashanka Ubaru (IBM T. J. Watson Research Center)
Chai Wah Wu (IBM T. J. Watson Research Center)
Abstract : We present a framework of randomized algorithms that include stochastic rounding and asynchronous updates and apply it to randomized linear algebra algorithms. In particular, we analyze an application to trace estimation and show its efficacy on various real-world datasets.
