# F79SP Stochastic Processes

Course co-ordinator(s): Prof Sergey Foss (Edinburgh), Dr Abdul-Lateef Haji-Ali (), Dr Alastair Wallis (Malaysia), Mahendran Shitan (Malaysia).

Aims:

To introduce fundamental stochastic processes which are useful in insurance, investment and stochastic modelling, and to develop techniques and methods for analysing the long term behaviour of these processes.

Summary:

In this course, we develop methods for modelling systems or quantities which change randomly with time. Specifically, the evolution of the system is described by a collection {Xt} of random variables, where Xt denotes the state of the system at time t.

Discrete-time processes studied include (renewal processes and) Markov chains. In particular we consider branching processes, random walk processes, and more general countable state-space chains.

Continuous-time processes studied include Poisson and compound Poisson processes; continuous time Markov processes; population, queueing and risk models.

## Detailed Information

Course Description: Link to Official Course Descriptor.

Pre-requisite course(s): F78PA Probability and Statistics A & F78PB Probability and Statistics B .

Location: Edinburgh, Malaysia.

Semester: 1.

Syllabus:

• Review of independence
• Sequences of random variables and the Markov property
• Review of matrix algebra
• Review of summation notation and other useful concepts.
• Using the Markov property
• Absorbing Markov chains with finite state space:
• Computing probability of absorption
• Computing expected time to absorption
• First-step (backwards) equations
• Basic examples: Mouse and cheese, drunkard’s walk, Ehrenfest chain, genetic models, gambling chains, etc.
• Stationarity problem for finite space chains
• Tricks on the computation of the stationary distribution (fluxes, reversibility, symmetry)
• State classification
• Periodicity
• Convergence to stationarity
• The strong theorem (law) of large numbers for sequences of i.i.d. random variables
• Frequencies
• Simple random walk (SRW) in the integers
• Markov chains with infinite but countable state space
• Simulation of Markov Chains and Markov Processes
• Recurrence and transience
• Examples: Discrete option pricing, branching processes, insurance loss, queueing for service, waiting for a bus, etc.–as time permits
• Basic relations between exponential, gamma, Poisson and uniform distributions
• Simple point processes, Poisson and compound Poisson processes
• Continuous time Markov processes
• Numerical solution of the Kolmogorov Forward Equations for a time-inhomogeneous Markov process
• Examples

Reading list:

Useful reference books are:

Recommended textbooks on background matters:

• D. Stirzaker (1999). Probability and Random Variables: a beginner’s guide, Cambridge University Press.
• K.L. Chung and F. Aitsahlia (2003). Elementary Probability Theory, Springer-Verlag.
• G. Grimmett & D. Welsh (1990). Probability: an Introduction, Oxford University Press.
• S. M. Ross (1988). A First Course in Probability, 3rd edition, Macmillan.
• D. Williams (2001). Weighing the Odds: A Course in Probability and Statistics, Cambridge University Press.

Assessment Methods:

This course will be assessed by a 2-hour examination at the end of the first semester (worth 80%) and 2 pieces of course work through the semester (worth 20%). It is synoptically linked with F79SU Survival Models.

SCQF Level: 9.

## Other Information

Help: If you have any problems or questions regarding the course, you are encouraged to contact the lecturer

VISION: further information and course materials are available on VISION