Abstract : Derivative-free optimization methods aim to solve optimization problems based on function values without using derivatives or other first-order information. They are motivated by problems where the first-order information is expensive or impossible to obtain. Such problems emerge frequently from industrial and engineering applications, including integrated circuit design, aircraft design, and hyperparameter tuning in artificial intelligence. This minisymposium will provide a platform for researchers and practitioners in derivative-free optimization to discuss the recent advances in theory, methods, and software in this area.
Stefan M Wild (Lawrence Berkeley National Laboratory)
Abstract : We consider finite-sum minimization problems in which the summand functions are computationally expensive, making it undesirable to evaluate all summands on every iteration. We present the idea of stochastic average model methods, which sample component functions according to a probability distribution on component functions that minimizes an upper bound on the variance of the resulting stochastic model. We present promising numerical results on a corresponding extension to the derivative-free model-based trust-region solver POUNDERS.
[01372] DFO with Transformed Objectives and a Model-based Trust-region Method
Format : Talk at Waseda University
Author(s) :
Pengcheng Xie (Academy of Mathematics and Systems Science (AMSS), Chinese Academy of Sciences (CAS))
Abstract : Derivative-free optimization, i.e., DFO, is the optimization where the derivative information is unavailable. The least Frobenius norm updating quadratic model is an essential under-determined model for derivative-free trust-region methods. We propose DFO with transformed objectives and give a model-based method with the least Frobenius norm model. We prove the existence and necessary and sufficient condition of model optimality-preserving transformations, and analyze the model, interpolation error and convergence property. Numerical results support our model and method.
[01341] COBYQA — A Derivative-Free Trust-Region SQP Method for Nonlinearly Constrained Optimization
Format : Talk at Waseda University
Author(s) :
Tom M. Ragonneau (The Hong Kong Polytechnic University)
Zaikun Zhang (The Hong Kong Polytechnic University)
Abstract : This talk introduces COBYQA, a derivative-free trust-region SQP method for nonlinear optimization. The method builds trust-region quadratic models using the derivative-free symmetric Broyden update. An important feature of COBYQA is that it always respects bound constraints. COBYQA is competitive with NEWUOA, BOBYQA, and LINCOA while being able to handle more general problems. Most importantly, COBYQA evidently outperforms COBYLA on all types of problems.
COBYQA is implemented in Python and is publicly available at https://www.cobyqa.com/.
[03192] A General Blackbox Optimization Framework for Hyperparameter Optimization in Deep Learning
Format : Talk at Waseda University
Author(s) :
Edward Hallé-Hannan (Polytechnique Montréal)
Sébastien Le Digabel (Polytechnique Montréal)
Charles Audet (Polytechnique Montréal)
Abstract : Tuning the hyperparameters of a deep model is a mixed-variable BBO problem with an unfixed structure. For instance, the number of layers (a hyperparameter) affects the number of architectural hyperparameters: meta variables are introduced to model this unfixed structure. Moreover, the hyperparameter optimization problem (HPO) may also simultaneously contain categorical, integer and continuous variables. A mathematical framework, which is compatible with direct search methods and Bayesian optimization, is proposed to tackle and model the HPO.
[01570] PRIMA: Reference Implementation for Powell's methods with Modernization and Amelioration
Format : Talk at Waseda University
Author(s) :
Zaikun Zhang (The Hong Kong Polytechnic University)
Abstract : Powell developed five widely used DFO solvers, namely COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. They were coded in Fortran 77 with a unique style, which poses a significant obstacle to maintaining, exploiting, or extending them. PRIMA is a project providing the reference implementation of these solvers in modern languages. We will present the current stage of PRIMA, including the bugs we have spotted in the Fortran 77 code and the improvements we have achieved.
