Homepage

Basic information theory course that follows Cover+Thomas closely. Slides for each chapter highlighting some main results and definitions; write proofs on the slides or board yourself.

The information theory course at the Japan Advanced Institute of Science and Technology (JAIST) uses Cover and Thomas as a starting point. While rigorous, content is not pursued at the same depth as C&T. I took the following directions in developing this course: A review of basic probability theory, Chebyshev inequality and law of large numbers is presented early. Short finite length codes are introduced as motivating examples before the main theorems: Huffman codes, repeat codes, rate-distortion codes, Slepian-Wolf, MAC channel Many numerical examples are intended to make students comfortable with abstract concepts. Self-study questions (SSQs) are an online-only course component (we use Moodle, contact me if interested) The entropy lower bound on source coding is proved using single..