F79SP Stochastic Processes

Prof Sergey FossDr Alastair Wallis

Course co-ordinator(s): Prof Sergey Foss (Edinburgh), Dr Alastair Wallis (Malaysia).

Aims:

To introduce fundamental stochastic processes which are useful in insurance, investment and stochastic modelling, and to develop techniques and methods for analysing the long term behaviour of these processes.

Summary:

In this course, we develop methods for modelling systems or quantities which change randomly with time. Specifically, the evolution of the system is described by a collection {Xt} of random variables, where Xt denotes the state of the system at time t.

Discrete-time processes studied include (renewal processes and) Markov chains. In particular we consider branching processes, random walk processes, and more general countable state-space chains.

Continuous-time processes studied include Poisson and compound Poisson processes; continuous time Markov processes; population, queueing and risk models.

Detailed Information

Pre-requisite course(s): F78PA Probability and Statistics A & F78PB Probability and Statistics B .

Location: Edinburgh, Malaysia.

Semester: 1.

Syllabus:

  • Review of independence
  • Sequences of random variables and the Markov property
  • Review of matrix algebra
  • Review of summation notation and other useful concepts.
  • Using the Markov property
  • Absorbing Markov chains with finite state space:
    • Computing probability of absorption
    • Computing expected time to absorption
  • First-step (backwards) equations
  • Basic examples: Mouse and cheese, drunkard’s walk, Ehrenfest chain, genetic models, gambling chains, etc.
  • Stationarity problem for finite space chains
  • Tricks on the computation of the stationary distribution (fluxes, reversibility, symmetry)
  • State classification
  • Periodicity
  • Convergence to stationarity
  • The strong theorem (law) of large numbers for sequences of i.i.d. random variables
  • Frequencies
  • Simple random walk (SRW) in the integers
  • Markov chains with infinite but countable state space
  • Simulation of Markov Chains and Markov Processes
  • Recurrence and transience
  • Examples: Discrete option pricing, branching processes, insurance loss, queueing for service, waiting for a bus, etc.–as time permits
  • Basic relations between exponential, gamma, Poisson and uniform distributions
  • Simple point processes, Poisson and compound Poisson processes
  • Continuous time Markov processes
  • Numerical solution of the Kolmogorov Forward Equations for a time-inhomogeneous Markov process
  • Examples

Reading list:

Useful reference books are:

Recommended textbooks on background matters:

  • D. Stirzaker (1999). Probability and Random Variables: a beginner’s guide, Cambridge University Press.
  • K.L. Chung and F. Aitsahlia (2003). Elementary Probability Theory, Springer-Verlag.
  • G. Grimmett & D. Welsh (1990). Probability: an Introduction, Oxford University Press.
  • S. M. Ross (1988). A First Course in Probability, 3rd edition, Macmillan.
  • D. Williams (2001). Weighing the Odds: A Course in Probability and Statistics, Cambridge University Press.

Assessment Methods:

This course will be assessed by a 2-hour examination at the end of the first semester (worth 80%) and 2 pieces of course work through the semester (worth 20%). It is synoptically linked with F79SU Survival Models.

SCQF Level: 9.

Other Information

Help: If you have any problems or questions regarding the course, you are encouraged to contact the lecturer

VISION: further information and course materials are available on VISION