EE5581: Information Theory and Coding
This course covers the basics of information theory and coding.
The two fundamental questions that information theory considers
are:
- What is the ultimate limit to data compression?
- What is the ultimate limit of reliable communication over a noisy
channel?
By the end of this course we will have answered both
questions, as well as provided some basic guidelines
for designing practical systems that come close to the
fundamental limits.
The course begins by defining the fundamental quantities
in information theory: entropy and mutual information.
Then we consider data compression (source coding),
followed by reliable communication over noisy channels (channel coding).
The final topic of the course will be rate distortion theory (lossy source coding).
Course Information
- Instructor: Nihar Jindal, 6-119 EE/CS, nihar@umn.edu, 625-6306
- Lecture: Mon/Wed/Fri, 12:20 PM - 1:10 PM, MechE 102
- Office Hours: Tu/Th, 10:30 AM - 12:00 PM, 6-119 EE/CS
- TA: Hakim Alhussien, 6-158 EE/CS, hakimh@umn.edu, 626-7178
- TA Office Hours: MW 4-5 PM, 6-158 EE/CS
- Required Textbook: Elements of Information Theory, Thomas Cover & Joy Thomas, John Wiley & Sons
- Prerequisite: EE 5531, Probability and Stochastic Processes (or equivalent).
- Homework: There will be weekly homework assignments.
- Exams: There will be two midterm exams
(Oct. 14 and Nov. 18) and a final exam (Wed, Dec. 21, 4-6 PM).
- Project:
A research project is a required portion of this course. The project
can either be a literature survey of a few information theory
papers, or an original research idea. The project will be due
towards the end of the semester.
- Grading: Final grade will be 15% project, 15% homework,
20% each midterm, and 30% final.
Homework
- Homework 1: C&T 2.16, 2.8, 2.2, 2.5, 2.1(a), 2.10
- Homework 2: C&T 2.6, 2.12, 2.18, 2.20, 2.28, 2.29, 2.14
- Homework 3: C&T 6.3, 3.2, 3.5, 3.6, 4.4, 4.8, 4.9
- Homework 4
- Homework 5: C&T 8.1, 8.2, 8.3, 8.5, 8.6, 8.7
- Homework 6: C&T 8.4, 8.9, 8.10, 8.11, 8.12
- Homework 7
- Homework 8: C&T 9.1, 9.2, 9.3, 9.4 (a,b), 10.2
- Homework 9: C&T 10.1, 10.3, 10.4, 13.1, 13.3
- Homework 10: C&T 13.2, 13.5, 13.6, 13.7, 13.8
Handouts
Useful Links
Shannon's original paper: A Mathematical Theory of Communication
Information
Theory Society