**Course co-ordinator(s):** Prof Sergey Foss (Edinburgh), Dr Alastair Wallis (Malaysia).

**Aims:**

To introduce fundamental stochastic processes which are useful in insurance, investment and stochastic modelling, and to develop techniques and methods for analysing the long term behaviour of these processes.

**Summary:**

In this course, we develop methods for modelling systems or quantities which change randomly with time. Specifically, the evolution of the system is described by a collection {*X _{t}*} of random variables, where

*X*denotes the state of the system at time

_{t}*t*.

Discrete-time processes studied include (renewal processes and) Markov chains. In particular we consider branching processes, random walk processes, and more general countable state-space chains.

Continuous-time processes studied include Poisson and compound Poisson processes; continuous time Markov processes; population, queueing and risk models.

## Detailed Information

**Pre-requisite course(s):** F78PA Probability and Statistics A & F78PB Probability and Statistics B .

**Location: **Edinburgh, Malaysia.

**Semester: **1.

**Syllabus:**

- Review of independence
- Sequences of random variables and the Markov property
- Review of matrix algebra
- Review of summation notation and other useful concepts.
- Using the Markov property
- Absorbing Markov chains with finite state space:
- Computing probability of absorption
- Computing expected time to absorption

- First-step (backwards) equations
- Basic examples: Mouse and cheese, drunkard’s walk, Ehrenfest chain, genetic models, gambling chains, etc.
- Stationarity problem for finite space chains
- Tricks on the computation of the stationary distribution (fluxes, reversibility, symmetry)
- State classification
- Periodicity
- Convergence to stationarity
- The strong theorem (law) of large numbers for sequences of i.i.d. random variables
- Frequencies
- Simple random walk (SRW) in the integers
- Markov chains with infinite but countable state space
- Simulation of Markov Chains and Markov Processes
- Recurrence and transience
- Examples: Discrete option pricing, branching processes, insurance loss, queueing for service, waiting for a bus, etc.–as time permits
- Basic relations between exponential, gamma, Poisson and uniform distributions
- Simple point processes, Poisson and compound Poisson processes
- Continuous time Markov processes
- Numerical solution of the Kolmogorov Forward Equations for a time-inhomogeneous Markov process
- Examples

**Reading list:**

Useful reference books are:

- Grinstead and Snell.
*Introduction to Probability.*American Mathematical Society - P. Bremaud (1997).
*An Introduction to Probabilistic Modeling*. Springer. - P. Bremaud (1999).
*Markov Chains*. Springer. - J. R. Norris (1998).
*Markov Chains*. Cambridge University Press. - G. R. Grimmett & D. R. Stirzaker (2001)
*Probability and Random Processes, 3rd ed.*Oxford University Press.

Recommended textbooks on background matters:

- D. Stirzaker (1999).
*Probability and Random Variables: a beginner’s guide*, Cambridge University Press. - K.L. Chung and F. Aitsahlia (2003).
*Elementary Probability Theory*, Springer-Verlag. - G. Grimmett & D. Welsh (1990).
*Probability: an Introduction*, Oxford University Press. - S. M. Ross (1988).
*A First Course in Probability*, 3rd edition, Macmillan. - D. Williams (2001).
*Weighing the Odds: A Course in Probability and Statistics*, Cambridge University Press.

**Assessment Methods:**

This course will be assessed by a 2-hour examination at the end of the first semester (worth 80%) and 2 pieces of course work through the semester (worth 20%). It is synoptically linked with F79SU Survival Models.

**SCQF Level: **9.

## Other Information

**Help:** If you have any problems or questions
regarding the course, you are encouraged to contact the lecturer

**VISION:** further information and course materials
are available on VISION