Theoretical in-depth course
In order to convey the advanced theoretical concepts, the mathematical background that forms the common basis of all neural networks, independent of their concrete structure and application, will be discussed in depth. Connections to the mathematics of inverse problems, eigenvalue and sensitivity analysis are derived and their significance in the context of neural networks is discussed. This serves as a basis for more advanced analyses including alternative and experimental training strategies as well as explainable AI methods that aim to systematically analyse complex nonlinear models.
Course
In the module "Theoretical AI Deepening" two course blocks are offered, which are designed to assist participants in developing a deeper understanding of the approximation capabilities of neural networks as well as solution strategies. All parts of the course are provided with examples and interactive Jupyter notebooks. This allows all participants to directly apply what they have learned.
The course is intended for all scientists who want to develop a deeper understanding of the fundamentals of modern AI methods.
Block A: Approximations
- Structure of ReLU networks
- Analysis as high dimensional lattice
- DNNs for the solution of partial differential equations
- PiNN
- DeepRitz
- Comparison DNN & Finite Element Methods
- Loss-energy relationship
- Weak formulation, strong formulation, approximation
- Discrete quadrature or discrete solution set
- Networks for non-smooth problems
- Error estimator for neural networks
Block B: Solution methods
- Solution method for overdetermined systems
- Least-squares
- Pseudoinverse
- Influence of norms on solutions
- Non-linear solution methods
- Gradient-descent
- Newton
- Convergence of methods
- Stochastic solution methods
- SGD
- Stochastic SVD
- Convergence estimates