F79SP Stochastic Processes

Prof Sergey FossDr Alistair Wallis

Course co-ordinator(s): Prof Sergey Foss (Edinburgh), Dr Alistair Wallis (Malaysia).


To introduce fundamental stochastic processes which are useful in insurance, investment and stochastic modelling, and to develop techniques and methods for analysing the long term behaviour of these processes.


In this course, we develop methods for modelling systems or quantities which change randomly with time. Specifically, the evolution of the system is described by a collection {Xt} of random variables, where Xt denotes the state of the system at time t.

Discrete-time processes studied include (renewal processes and) Markov chains. In particular we consider branching processes, random walk processes, and more general countable state-space chains.

Continuous-time processes studied include Poisson and compound Poisson processes; continuous time Markov processes; population, queueing and risk models.

Detailed Information

Course Description: Link to Official Course Descriptor.

Pre-requisite course(s): F78PA Probability and Statistics A & F78PB Probability and Statistics B .

Location: Edinburgh, Malaysia.

Semester: 1.


  • Review of independence
  • Sequences of random variables and the Markov property
  • Review of matrix algebra
  • Review of summation notation and other useful concepts.
  • Using the Markov property
  • Absorbing Markov chains with finite state space:
    • Computing probability of absorption
    • Computing expected time to absorption
  • First-step (backwards) equations
  • Basic examples: Mouse and cheese, drunkard’s walk, Ehrenfest chain, genetic models, gambling chains, etc.
  • Stationarity problem for finite space chains
  • Tricks on the computation of the stationary distribution (fluxes, reversibility, symmetry)
  • State classification
  • Periodicity
  • Convergence to stationarity
  • The strong theorem (law) of large numbers for sequences of i.i.d. random variables
  • Frequencies
  • Simple random walk (SRW) in the integers
  • Markov chains with infinite but countable state space
  • Simulation of Markov Chains and Markov Processes
  • Recurrence and transience
  • Examples: Discrete option pricing, branching processes, insurance loss, queueing for service, waiting for a bus, etc.–as time permits
  • Basic relations between exponential, gamma, Poisson and uniform distributions
  • Simple point processes, Poisson and compound Poisson processes
  • Continuous time Markov processes
  • Numerical solution of the Kolmogorov Forward Equations for a time-inhomogeneous Markov process
  • Examples

Reading list:

Useful reference books are:

Recommended textbooks on background matters:

  • D. Stirzaker (1999). Probability and Random Variables: a beginner’s guide, Cambridge University Press.
  • K.L. Chung and F. Aitsahlia (2003). Elementary Probability Theory, Springer-Verlag.
  • G. Grimmett & D. Welsh (1990). Probability: an Introduction, Oxford University Press.
  • S. M. Ross (1988). A First Course in Probability, 3rd edition, Macmillan.
  • D. Williams (2001). Weighing the Odds: A Course in Probability and Statistics, Cambridge University Press.

Assessment Methods: Due to covid, assessment methods for Academic Year 2021/22 may vary from those noted on the official course descriptor. Please see:
- Maths (F1) Course Weightings 2021/22
- Computer Science (F2) Course Weightings 2021/22
- AMS (F7) Course Weightings 2021/22

SCQF Level: 9.

Other Information

Help: If you have any problems or questions regarding the course, you are encouraged to contact the course leader.

Canvas: further information and course materials are available on Canvas