The theory of Markov decision processes focuses on controlled Markov chains in discrete time. The authors establish the theory for general state and action spaces and at the same time show its application by means of numerous examples, mostly taken from the fields of finance and operations research. By using a structural approach many technicalities (concerning measure theory) are avoided. They cover problems with finite and infinite horizons, as well as partially observable Markov decision processes, piecewise deterministic Markov decision processes and stopping problems.
The book presents Markov decision processes in action and includes various state-of-the-art applications with a particular view towards finance. It is useful for upper-level undergraduates, Master's students and researchers in both applied probability and finance, and provides exercises (without solutions).
The book presents Markov decision processes in action and includes various state-of-the-art applications with a particular view towards finance. It is useful for upper-level undergraduates, Master's students and researchers in both applied probability and finance, and provides exercises (without solutions).
From the reviews:
"This book presents Markov decision processes with general state and action spaces and includes various state-of-the-art applications that stem from finance and operations research. ... very helpful, not only for graduate students, but also for researchers working in the field of MDPs and finance. The authors do not focus only on discrete-time MDPs, but provide the description of different classes of Markov models ... . Each chapter ends with remarks, where the potential reader may find further hints concerning references." (Anna Jaskiewicz, Zentralblatt MATH, Vol. 1236, 2012)
"This book presents Markov decision processes with general state and action spaces and includes various state-of-the-art applications that stem from finance and operations research. ... very helpful, not only for graduate students, but also for researchers working in the field of MDPs and finance. The authors do not focus only on discrete-time MDPs, but provide the description of different classes of Markov models ... . Each chapter ends with remarks, where the potential reader may find further hints concerning references." (Anna Jaskiewicz, Zentralblatt MATH, Vol. 1236, 2012)