F79SP Stochastic Processes

Prof Sergey FossDr Abdul-Lateef Haji-AliDr Alastair Wallis

Course co-ordinator(s): Prof Sergey Foss (Edinburgh), Dr Abdul-Lateef Haji-Ali (), Dr Alastair Wallis (Malaysia), Mahendran Shitan (Malaysia).


To introduce fundamental stochastic processes which are useful in insurance, investment and stochastic modelling, and to develop techniques and methods for analysing the long term behaviour of these processes.


In this course, we develop methods for modelling systems or quantities which change randomly with time. Specifically, the evolution of the system is described by a collection {Xt} of random variables, where Xt denotes the state of the system at time t.

Discrete-time processes studied include (renewal processes and) Markov chains. In particular we consider branching processes, random walk processes, and more general countable state-space chains.

Continuous-time processes studied include Poisson and compound Poisson processes; continuous time Markov processes; population, queueing and risk models.

Detailed Information

Course Description: Link to Official Course Descriptor.

Pre-requisite course(s): F78PA Probability and Statistics A & F78PB Probability and Statistics B .

Location: Edinburgh, Malaysia.

Semester: 1.


  • Review of independence
  • Sequences of random variables and the Markov property
  • Review of matrix algebra
  • Review of summation notation and other useful concepts.
  • Using the Markov property
  • Absorbing Markov chains with finite state space:
    • Computing probability of absorption
    • Computing expected time to absorption
  • First-step (backwards) equations
  • Basic examples: Mouse and cheese, drunkard’s walk, Ehrenfest chain, genetic models, gambling chains, etc.
  • Stationarity problem for finite space chains
  • Tricks on the computation of the stationary distribution (fluxes, reversibility, symmetry)
  • State classification
  • Periodicity
  • Convergence to stationarity
  • The strong theorem (law) of large numbers for sequences of i.i.d. random variables
  • Frequencies
  • Simple random walk (SRW) in the integers
  • Markov chains with infinite but countable state space
  • Simulation of Markov Chains and Markov Processes
  • Recurrence and transience
  • Examples: Discrete option pricing, branching processes, insurance loss, queueing for service, waiting for a bus, etc.–as time permits
  • Basic relations between exponential, gamma, Poisson and uniform distributions
  • Simple point processes, Poisson and compound Poisson processes
  • Continuous time Markov processes
  • Numerical solution of the Kolmogorov Forward Equations for a time-inhomogeneous Markov process
  • Examples

Reading list:

Useful reference books are:

Recommended textbooks on background matters:

  • D. Stirzaker (1999). Probability and Random Variables: a beginner’s guide, Cambridge University Press.
  • K.L. Chung and F. Aitsahlia (2003). Elementary Probability Theory, Springer-Verlag.
  • G. Grimmett & D. Welsh (1990). Probability: an Introduction, Oxford University Press.
  • S. M. Ross (1988). A First Course in Probability, 3rd edition, Macmillan.
  • D. Williams (2001). Weighing the Odds: A Course in Probability and Statistics, Cambridge University Press.

Assessment Methods:

This course will be assessed by a 2-hour examination at the end of the first semester (worth 80%) and 2 pieces of course work through the semester (worth 20%). It is synoptically linked with F79SU Survival Models.

SCQF Level: 9.

Other Information

Help: If you have any problems or questions regarding the course, you are encouraged to contact the lecturer

VISION: further information and course materials are available on VISION