EE4389: Course Information

Professor Cherkassky, University of Minnesota, Fall Semester 2015


pdf file


  • Time: Tuesdays and Thursdays, 1:00–2:15pm

  • Place: Keller Hall 3-230


  • EE3025 for ECE students, or Stat3022 for non-ECE students.

  • Working knowledge of probability theory.

  • Familiarity with computer programming (Matlab or similar environment).

Course Material

  • Textbook Predictive Learning by V. Cherkassky is available at or University bookstore.
    Book description can be found here or

  • Textbook The Problems of Philosophy by Bertrand Russell is available online.
    link 1, link 2.

  • Lecture notes, handouts and selected papers are available on the Lecture notes page.

Course Description

Advances in computer and database technology motivate the need for estimating dependencies (models) from available data. Often the main goal (of modeling) is to estimate a model providing generalization, i.e. good prediction for future data.

Such methods have been developed in various fields including statistics, machine learning, neural networks, data mining, signal processing, etc. This course presents description of predictive data-analytic modeling methods on several levels: conceptual/mathematical, technical and philosophical. The course consists of three related parts:

  • Conceptual/Theoretical Part deals with fundamental concepts and principles important for estimating predictive data-analytic models. These issues are addressed by the mathematical theory called Statistical Learning Theory, which is discussed in this course.

  • Technical/Practical Part focuses on constructive learning methods and applications. Representative learning methods include methods developed in machine learning, statistics and neural networks. These methods include Decision Trees, Multi-Layer Perceptrons, Self-Organizing Maps, Support Vector Machines and Boosting. These methods are illustrated via practical applications, such as object recognition, financial engineering, signal denoising, etc.

  • Philosophical Part explores the connection between mathematical principles presented in Part 1, and the Philosophy of Science, which is concerned with general conditions for judging the validity (truthfulness) of scientific theories. Similarly, the fundamental principles of inference from data (discussed in Part 1) underlying machine learning algorithms (in Part 2) will be also related to mechanisms of human learning and intelligence.


  • Five homework assignments 45%

  • Three writing assignments 30%

  • One written exam 25%