This is the page for the course Optimization Techniques (ENGG-6140), Section-2, offered in Winter 2023 at the University of Guelph.
Videos of Lectures:
See videos on Youtube
Lectures
- Course introduction:
- Topic 1 (slides, annotated slides):
- Preliminaries (sets, norms, functions, gradient, Jacobian, Hessian, etc)
- Convexity of sets and functions
- Introduction to standard optimization problems
- Topic 2:
- Topic 3: (slides, annotated slides)
- Karush-Kuhn-Tucker (KKT) conditions
- The Lagrangian function
- Dual variables, primal and dual feasibility, an the dual problem
- The method of Lagrange multipliers
- Topic 4: (slides, annotated slides)
- Unconstrained an constrained first-order optimization
- Gradient descent, line-search, steepest descent, stochastic gradient descent
- Backpropagation and neural networks
- Proximal mapping, proximal gradient method
- Projected gradient method
- Topic 5: (slides, annotated slides)
- Unconstrained and constrained second-order optimization
- Unconstrained Newton’s method
- Equality constrained Newton’s method
- Inequality constrained Newton’s method (Interior-point and barrier methods)
- Quasi-Newton’s methods
- Topic 6:
- Topic 7:
- Topic 8:
- Topic 9:
- Topic 10:
- Topic 11: