Dynamic optimization and reinforcement learning
This course introduces deterministic and stochastic dynamic optimization and reinforcement learning.
The aims are (i) to motivate the use of dynamic optimization techniques (including reinforcement learning) in theoretical and applied micro- and macroeconomic research, (ii) to learn the analytical and numerical methods required to solve the optimization problems, (iii) to apply the techniques to selected empirical applications. As the methods are computationally intensive, the exercises and applications will be done in R.
After the course, participants will: have a solid understanding of the theoretical background of dynamic optimization; be able to program the methods covered in the course to investigate empirical dynamic optimization problems; understand the related techniques they encounter in the literature; appreciate the advantages of reinforcement learning; be able to use the R software.
Content
- Deterministic dynamic optimization: problem setup and motivation in discrete time; Lagrange approach; dynamic programming; Bellman equations; numeric solution strategies; numerical tools for function representations and uni- and multivariate optimization; infinite horizon case.
- Stochastic dynamic optimization: introduction to stochastic processes; Markov processes; numerical tools for integration; problem setup and motivation in discrete time; dynamic programming; Bellman equations; infinite horizon case.
- Reinforcement learning: introduction to the terminology and its relation to classical dynamic optimization; basics of neural networks for representing value functions in high-dimensional spaces.
Schedule
Course format
Prerequisites
Administration
- Maximum number of participants: 15
- The course belongs to the "Methods" module of the PhD program but is also open to Master students.
- The course (including the exam) is eligible for 6 credit points.
- The exam consists of a one-week take-home exercise and a 30-minute presentation of the results.
- Lehrende/r: Simon Haastert
- Lehrende/r: Jörg Lingens
- Lehrende/r: Andrea Rüschenschmidt
- Lehrende/r: Mark Trede