ECE Header Logo

EEC266 – Information Theory And Coding

3 units – Winter Quarter; alternate years

Lecture: 3 hours

Prerequisite: STA 120

Grading: Letter; one midterm exam, on final exam and homework and/or a project.

Catalog Description:

Information theory and coding. Measure of information. Redundancy reduction encoding of an information source. Capacity of a communication channel, error-free communications.

Expanded Course Description:

Topics are from an introduction to error-correction codes, channel capacity for continuous channels and source coding with a fidelity criterion.

  1. Information and Sources – The definition of information, the zero memory information source, the Markov information source.
  2. Properties of Codes – Uniquely decodable codes, prefix codes, conditions for existence, Huffman codes.
  3. The Coding of Information Sources – The average length of a code, the information rate of a source, compact codes, the redundancy of a code.
  4. Noisy Channels and Mutual Information – Probability relations in a channel, channel capacity for discrete channels, Channel Coding Theorem.


  1. N. Abramson, Information Theory and Coding, McGraw-Hill, 1963.
  2. R. G. Gallager, Information Theory and Reliable Communication, Wiley, 1968.

Instructor: Abdel-Ghaffar


Last revised: November 1991