Markov Chains and Decision Processes for Engineers and Managers - Rilegato

Sheskin, Theodore J.

Rilegato

ISBN 10: 1420051113 ISBN 13: 9781420051117

Casa editrice: CRC Press, 2010

Riassunto

Recognized as a powerful tool for dealing with uncertainty, Markov modeling can enhance your ability to analyze complex production and service systems. However, most books on Markov chains or decision processes are often either highly theoretical, with few examples, or highly prescriptive, with little justification for the steps of the algorithms used to solve Markov models. Providing a unified treatment of Markov chains and Markov decision processes in a single volume, Markov Chains and Decision Processes for Engineers and Managers supplies a highly detailed description of the construction and solution of Markov models that facilitates their application to diverse processes.

Organized around Markov chain structure, the book begins with descriptions of Markov chain states, transitions, structure, and models, and then discusses steady state distributions and passage to a target state in a regular Markov chain. The author treats canonical forms and passage to target states or to classes of target states for reducible Markov chains. He adds an economic dimension by associating rewards with states, thereby linking a Markov chain to a Markov decision process, and then adds decisions to create a Markov decision process, enabling an analyst to choose among alternative Markov chains with rewards so as to maximize expected rewards. An introduction to state reduction and hidden Markov chains rounds out the coverage.

In a presentation that balances algorithms and applications, the author provides explanations of the logical relationships that underpin the formulas or algorithms through informal derivations, and devotes considerable attention to the construction of Markov models. He constructs simplified Markov models for a wide assortment of processes such as the weather, gambling, diffusion of gases, a waiting line, inventory, component replacement, machine maintenance, selling a stock, a charge account, a career path, patient flow

Su questo libro

L'autore

Sheskin, Theodore J.

Contenuti

Markov Chain Structure and Models
Historical Note
States and Transitions
Model of the Weather
Random Walks
Estimating Transition Probabilities
Multiple-Step Transition Probabilities
State Probabilities after Multiple Steps
Classification of States
Markov Chain Structure
Markov Chain Models
Problems
References

Regular Markov Chains
Steady State Probabilities
First Passage to a Target State
Problems
References

Reducible Markov Chains
Canonical Form of the Transition Matrix
The Fundamental Matrix
Passage to a Target State
Eventual Passage to a Closed Set Within a Reducible Multichain
Limiting Transition Probability Matrix
Problems
References

A Markov Chain with Rewards (MCR)
Rewards
Undiscounted Rewards
Discounted Rewards
Problems
References

A Markov Decision Process (MDP)
An Undiscounted MDP
A Discounted MDP
Problems
References

Special Topics: State Reduction and Hidden Markov Chains
State Reduction
An Introduction to Hidden Markov
Problems
References
Index

Dettagli dell?edizione

• Editore: CRC Press
• Data di pubblicazione: 2010
• ISBN 10: 1420051113
• ISBN 13: 9781420051117
• Rilegatura: Copertina rigida
• Lingua: Inglese
• Numero edizione: 1
• Numero di pagine: 492