Exploiting Structure To Efficiently Solve Large Scale Partially Observable Markov Decision Processes Microform
Download Exploiting Structure To Efficiently Solve Large Scale Partially Observable Markov Decision Processes Microform PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Exploiting Structure To Efficiently Solve Large Scale Partially Observable Markov Decision Processes Microform book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages.
Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes [microform]
![Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes [microform]](https://library.ardhindie.com/contents/assets/images/blank.png){kind=small}
Author: Pascal Poupart
language: en
Publisher: Library and Archives Canada = Bibliothèque et Archives Canada
Release Date: 2005
Partially observable Markov decision processes (POMDPs) provide a natural and principled framework to model a wide range of sequential decision making problems under uncertainty. To date, the use of POMDPs in real-world problems has been limited by the poor scalability of existing solution algorithms, which can only solve problems with up to ten thousand states. In fact, the complexity of finding an optimal policy for a finite-horizon discrete POMDP is PSPACE-complete. In practice, two important sources of intractability plague most solution algorithms: Large policy spaces and large state spaces. In practice, it is critical to simultaneously mitigate the impact of complex policy representations and large state spaces. Hence, this thesis describes three approaches that combine techniques capable of dealing with each source of intractability: VDC with BPI, VDC with Perseus (a randomized point-based value iteration algorithm by Spaan and Vlassis [136]), and state abstraction with Perseus. The scalability of those approaches is demonstrated on two problems with more than 33 million states: synthetic network management and a real-world system designed to assist elderly persons with cognitive deficiencies to carry out simple daily tasks such as hand-washing. This represents an important step towards the deployment of POMDP techniques in ever larger, real-world, sequential decision making problems. On the other hand, for many real-world POMDPs it is possible to define effective policies with simple rules of thumb. This suggests that we may be able to find small policies that are near optimal. This thesis first presents a Bounded Policy Iteration (BPI) algorithm to robustly find a good policy represented by a small finite state controller. Real-world POMDPs also tend to exhibit structural properties that can be exploited to mitigate the effect of large state spaces. To that effect, a value-directed compression (VDC) technique is also presented to reduce POMDP models to lower dimensional representations.
Constrained Markov Decision Processes
This book provides a unified approach for the study of constrained Markov decision processes with a finite state space and unbounded costs. Unlike the single controller case considered in many other books, the author considers a single controller with several objectives, such as minimizing delays and loss, probabilities, and maximization of throughputs. It is desirable to design a controller that minimizes one cost objective, subject to inequality constraints on other cost objectives. This framework describes dynamic decision problems arising frequently in many engineering fields. A thorough overview of these applications is presented in the introduction. The book is then divided into three sections that build upon each other. The first part explains the theory for the finite state space. The author characterizes the set of achievable expected occupation measures as well as performance vectors, and identifies simple classes of policies among which optimal policies exist. This allows the reduction of the original dynamic into a linear program. A Lagranian approach is then used to derive the dual linear program using dynamic programming techniques. In the second part, these results are extended to the infinite state space and action spaces. The author provides two frameworks: the case where costs are bounded below and the contracting framework. The third part builds upon the results of the first two parts and examines asymptotical results of the convergence of both the value and the policies in the time horizon and in the discount factor. Finally, several state truncation algorithms that enable the approximation of the solution of the original control problem via finite linear programs are given.
Intelligent Decision Making: An AI-Based Approach
Author: Gloria Phillips-Wren
language: en
Publisher: Springer Science & Business Media
Release Date: 2008-03-04
Intelligent Decision Support Systems have the potential to transform human decision making by combining research in artificial intelligence, information technology, and systems engineering. The field of intelligent decision making is expanding rapidly due, in part, to advances in artificial intelligence and network-centric environments that can deliver the technology. Communication and coordination between dispersed systems can deliver just-in-time information, real-time processing, collaborative environments, and globally up-to-date information to a human decision maker. At the same time, artificial intelligence techniques have demonstrated that they have matured sufficiently to provide computational assistance to humans in practical applications. This book includes contributions from leading researchers in the field beginning with the foundations of human decision making and the complexity of the human cognitive system. Researchers contrast human and artificial intelligence, survey computational intelligence, present pragmatic systems, and discuss future trends. This book will be an invaluable resource to anyone interested in the current state of knowledge and key research gaps in the rapidly developing field of intelligent decision support.