Stochastic Optimization Methods For Modern Machine Learning Problems
Download Stochastic Optimization Methods For Modern Machine Learning Problems PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Stochastic Optimization Methods For Modern Machine Learning Problems book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages.
Stochastic Optimization Methods for Modern Machine Learning Problems
Optimization has been the workhorse of solving machine learning problems. However, the efficiency of these methods remains far from satisfaction to meet the ever-growing demand that arises in modern applications. In this context, the present dissertation will focus on two fundamental classes of machine learning problems: 1) stochastic nested problems, where one subproblem builds upon the solution of others; and, 2) stochastic distributed problems, where the subproblems are coupled through sharing the common variables. One key difficulty of solving stochastic nested problems is that the hierarchically coupled structure makes the computation of (stochastic) gradients, the basic element in first-order optimization machinery, prohibitively expensive or even impossible.We will develop the first stochastic optimization method, which runs in a single-loop manner and achieves the same sample complexity as the stochastic gradient descent method for non-nested problems. One key difficulty of solving stochastic distributed problems is the resource intensity, especially when algorithms are running atresource-limited devices. In this context, we will introduce a class of communication-adaptive stochastic gradient descent (SGD) methods, which adaptively reuse the stale gradients, thus saving communication. We will show that the new algorithms have convergence rates comparable to original SGD and Adam algorithms, but enjoy impressive empirical performance in terms of total communication round reduction.
Stochastic Modeling and Optimization Methods for Critical Infrastructure Protection, Volume 1
Author: Alexei A. Gaivoronski
language: en
Publisher: John Wiley & Sons
Release Date: 2025-05-13
Stochastic Modeling and Optimization Methods for Critical Infrastructure Protection is a thorough exploration of mathematical models and tools that are designed to strengthen critical infrastructures against threats – both natural and adversarial. Divided into two volumes, this first volume examines stochastic modeling across key economic sectors and their interconnections, while the second volume focuses on advanced mathematical methods for enhancing infrastructure protection. The book covers a range of themes, including risk assessment techniques that account for systemic interdependencies within modern technospheres, the dynamics of uncertainty, instability and system vulnerabilities. The book also presents other topics such as cryptographic information protection and Shannon’s theory of secret systems, alongside solutions arising from optimization, game theory and machine learning approaches. Featuring research from international collaborations, this book covers both theory and applications, offering vital insights for advanced risk management curricula. It is intended not only for researchers, but also educators and professionals in infrastructure protection and stochastic optimization.
First-order and Stochastic Optimization Methods for Machine Learning
This book covers not only foundational materials but also the most recent progresses made during the past few years on the area of machine learning algorithms. In spite of the intensive research and development in this area, there does not exist a systematic treatment to introduce the fundamental concepts and recent progresses on machine learning algorithms, especially on those based on stochastic optimization methods, randomized algorithms, nonconvex optimization, distributed and online learning, and projection free methods. This book will benefit the broad audience in the area of machine learning, artificial intelligence and mathematical programming community by presenting these recent developments in a tutorial style, starting from the basic building blocks to the most carefully designed and complicated algorithms for machine learning.