Skip to main content
Engineering Online

CSC 592 611 Optimizations and Algorithms

3 Credit Hours

(also offered as ECE 592)

This course introduces advances in optimization theory and algorithms with rapidly growing applications in machine learning, systems, and control. Methods to obtain the extremum (minimum or maximum) of a non-dynamic system and the use of these methods in various engineering applications are given.

Prerequisite

Introductory courses in probability and linear algebra.

Course Objectives

  • The goal of this course is to prepare graduate students with a solid theoretical and mathematical foundation and applied techniques at the intersection of optimization, algorithms, and machine learning to conduct advanced research in the related fields.
  • Students will gain expertise in designing algorithms based on conventional techniques and be able to deal with intractable problems and implement algorithms given the description.
  • Students will need to undertake a half-semester long project that practices the optimization theory and algorithms in their areas of interest. It is allowed to be a replication or an improvement of a known solving strategy for a given optimization problem to assess/compare performance characteristics.

Topics covered

This course aims to cover the following topics:

  • Nonlinear unconstrained optimization, linear programming, nonlinear constrained optimization, computational and search methods for optimization; convex optimization and integer programming.
  • Greedy, divide and conquer, dynamic programming; approximation algorithms.
  • Stochastic optimization, sparsity, regularized optimization, interior-point methods, proximal methods, robust optimization.
  • Convergence rate analysis, momentum-based acceleration, distributed and asynchronous algorithm design, saddle point escaping.

Grading

Homework20%
Midterm Exam25%
Final Exam30%
Project25%

Textbooks

E.K.P. Chong and S.H. Zak, “An Introduction to Optimization,” John Wiley & Sons, 2008.

A list of essential and trending papers will be provided on the course website. Useful reference books on optimization theory and mathematical backgrounds include:

S. Boyd and L. Vandenberghe, “Convex Optimization,” Cambridge University Press, 2004.

Y. Nesterov, “Introductory Lectures on Convex Optimization: A Basic Course,” Springer, 2004.

M. Bazarra, H.D. Sherali, and C.M. Shetty, “Nonlinear Programming: Theory and Algorithms,” John Wiley & Sons, 2006.

Updated 7/8/2020