Course co-ordinator(s): Prof Sergey Foss (Edinburgh).
Aims:
After studying this course, students should be able to:
• Use large deviation theory to estimate the probability of rare events
• Understand and use the Markov property
• Write down equations for the stationary distribution of a Markov chain and use, wherever possible, additional
structure to solve them
• Write down first step equations and use them to compute the time to death, probability of absorption etc.
• Apply Markov chain modelling in several problems
• Understand long term behaviour and stationarity of a Markov chain
• Use renewal process to model various situations
• Calculate statistical properties for various renewal processes
• Define martingales
• Use main properties of martingales
Summary:
To introduce fundamental stochastic processes which are useful in stochastic modelling and data science
Detailed Information
Pre-requisites: none.
Location: Edinburgh.
Semester: 1.
Syllabus:
• Random walks and Large Deviations
- definition of a random walk
- introduction to large deviations theory
- introduction to rare event simulation
• Conditional expectation
• Markov chain
- Sequences of random variables and the Markov property
- Using the Markov property
- Absorbing Markov chains with finite state space
- First step (backwards) equations
- Basic examples
- Stationarity problem for finite state space chains
- Convergence to stationarity
- Markov chains with infinite but countable state space
• Simple point processes, Poisson and compound Poisson processes
• Continuous-time Markov processes
• Renewal theory
- elementary renewal theory
- properties of the renewal function
- discrete renewal theory
• Martingales
Learning Outcomes: Subject Mastery
After studying this course, students should be able to:
• Use large deviation theory to estimate the probability of rare events
• Understand and use the Markov property
• Write down equations for the stationary distribution of a Markov chain and use, wherever possible, additional structure to solve them
• Write down first step equations and use them to compute the time to death, probability of absorption etc.
• Apply Markov chain modelling in several problems
• Understand long term behaviour and stationarity of a Markov chain
• Use renewal process to model various situations
• Calculate statistical properties for various renewal processes
• Define martingales
• Use main properties of martingales
Learning Outcomes: Personal Abilities
At the end of the course, students should be able to:
• Demonstrate the ability to learn independently
• Manage time work to deadlines and prioritise workloads
• Present results in a way which demonstrates that they have understood the technical and broader issues of stochastic processes
Assessment Methods: Due to covid, assessment methods for Academic Year 2021-22 may vary from those noted on the official course descriptor. Please see the Computer Science Course Weightings and the Maths Course Weightings for 2020-21 Semester 1 assessment methods.
SCQF Level: 11.
Credits: 15.
Other Information
Help: If you have any problems or questions regarding the course, you are encouraged to contact the lecturer
Canvas: further information and course materials are available on Canvas