A textbook for students with some background in probability that develops quickly a rigorous theory of Markov chains and shows how actually to apply it, e.g. to simulation, economics, optimal control, genetics, queues and many other topics, and exercises and examples drawn both from theory and practice.