This book was born from my teaching experience in French engineering schools.In these, mathematical tools should be introduced by showing that they providemodels allowing for exact or approximate computations for relevant quantitiespertaining to realistic phenomena.

I have taught in particular a course on the Markovchains, the theory of which is already old. I had learnt a lot of it by osmosis whilestudying or doing research on more recent topics in stochastics. Teaching it forced medo delve deeper into it. This allowed me to rediscover the power and finesse ofthe probabilistic tools based on the work by Kai Lai Chung,Wolfang (or Vincent) Doeblin, Joseph Leo Doob, William Feller, and AndreiKolmogorov, which laid the ground for stochastic calculus.

I realized that Markovchain theory is actually a very active research field, both for theory and forapplications. The derivation of efficient Monte Carlo algorithms and theirvariants, for instance adaptive ones, is a hot subject, and often these are theonly methods allowing tractable computations for treating the enormousquantities of data that scientists and engineers can now acquire. This neednotably feeds theoretical studies on long-time rates of convergence for Markovchains.

This book is aimed at a public of engineering school and masterstudents and of applied scientists and engineers, wishing to acquire thepertinent mathematical bases. It is structured and animated by a few classicexamples, which are each investigated at different stages of the book andeventually studied exhaustively. These illustrate in real time the newly introducednotions and their qualitative and quantitative uses. It also elaborates on thegeneral matter of Monte Carlo approximation.

This book owes a lot tothe forementioned mathematicians, as well as the authors ofthe too small bibliography. More personal contributions to this book were provided by discussions with other teachers and researchers and by theinteraction with the students, notably those in the engineering schools withtheir pragmatic perspective on mathematics.

Feller's book [3] has alwaysimpressed me, notably by the wealth of examples of a practical nature itcontains. It represents a compendium of probability theory at his time and canreadily be adapted to a modern public. Reading it has lead me to try topush some explicit computations quite far, notably using generatingfunctions, but my efforts are pale with respect to his achievements in thisperspective.