Introduction to Iterative Optimization
[Academic Review] An in-depth introduction to first- and second-order optimization methods, complete with Python implementations.
This is the first part of a comprehensive presentation on mathematical optimization. Key topics include determining convexity, formulating likelihood and log-likelihood functions, and computing gradients and Hessians. The review also covers fundamental and advanced algorithms such as Gradient Descent (GD), Accelerated and Stochastic GD, Newton’s Method, Alternating Direction Method of Multipliers (ADMM), Coordinate Descent, and Proximal Gradient Descent. Sample cases demonstrate the practical implementation of these methods.
Click here to view the embedded file.