EE5581: Information Theory and Coding

University of Minnesota, Fall 2005

Professor Nihar Jindal

This course covers the basics of information theory and coding. The two fundamental questions that information theory considers are:
  1. What is the ultimate limit to data compression?
  2. What is the ultimate limit of reliable communication over a noisy channel?
By the end of this course we will have answered both questions, as well as provided some basic guidelines for designing practical systems that come close to the fundamental limits.
The course begins by defining the fundamental quantities in information theory: entropy and mutual information. Then we consider data compression (source coding), followed by reliable communication over noisy channels (channel coding). The final topic of the course will be rate distortion theory (lossy source coding).

Course Information



Useful Links

  • Shannon's original paper: A Mathematical Theory of Communication
  • Information Theory Society