1 Controlled Markov Processes.- 1.1 Introduction.- 1.2 Stochastic Control Problems.- 1.3 Examples.- 1.4 Further Comments.- 2 Discounted Reward Criterion.- 2.1 Introduction.- 2.2 Optimality Conditions.- 2.3 Asymptotic Discount Optimality.- 2.4 Approximation of MCM's.- 2.5 Adaptive Control Models.- 2.6 Nonparametric Adaptive Control.- 2.7 Comments and References.- 3 Average Reward Criterion.- 3.1 Introduction.- 3.2 The Optimality Equation.- 3.3 Ergodicity Conditions.- 3.4 Value Iteration.- 3.5 Approximating Models.- 3.6 Nonstationary Value Iteration.- 3.7 Adaptive Control Models.- 3.8 Comments and References.- 4 Partially Observable Control Models.- 4.1 Introduction.- 4.2 PO-CM: Case of Known Parameters.- 4.3 Transformation into a CO Control Problem.- 4.4 Optimal I-Policies.- 4.5 PO-CM's with Unknown Parameters.- 4.6 Comments and References.- 5 Parameter Estimation in MCM's.- 5.1 Introduction.- 5.2 Contrast Functions.- 5.3 Minimum Contrast Estimators.- 5.4 Comments and References.- 6 Discretization Procedures.- 6.1 Introduction.- 6.2 Preliminaries.- 6.3 The Non-Adaptive Case.- 6.4 Adaptive Control Problems.- 6.5 Proofs.- 6.6 Comments and References.- Appendix A. Contraction Operators.- Appendix B. Probability Measures.- Total Variation Norm.- Weak Convergence.- Appendix C. Stochastic Kernels.- Appendix D. Multifunctions and Measurable Selectors.- The Hausdorff Metric.- Multifunctions.- References.- Author Index.
