ENS Research Course  INFO5147  Academic Year 202021INFO5147: Selected Topics in Information TheoryÉcole Normale Supérieure de Lyon (ENS de Lyon) Course DescriptionThe course ‘‘INFO5147 – Selected Topics in Information Theory’’ is divided into two parts: Theoretical Foundations and Applications. The objective of the first part is to level the ground to study information theory outside the classical framework of communications theory. The motivation for studying information theory outside its most prominent application domain is to widen and strengthen its connections with other disciplines and mathematical theories, in particular, real analysis, measure theory, probability theory, optimization, game theory, and statistics. This choice provides a more general look to information theory and might inspire new applications in different fields. Certainly, by adopting this choice, information theory can be truly appreciated as a developing mathematical theory whose impact on pure and applied sciences is yet to be discovered. The second part focuses on the applications of information theory in statistics, in particular, stochastic approximations and expectation maximization algorithms; and communications theory, more specifically, storage and data transmission. These problems are studied from a modern perspective in which asymptotic assumptions are avoided. That is, these problems are formulated taking into account that data storage takes place with finite storage capacity; and data transmission takes place within a finite period. This rises the consideration of distorsion and decodingerror probabilities that are certainly bounded away from zero. Within this context, the fundamental limits of data storage and data transmission are studied in scenarios that are close to realsystem implementations. Open problems in multiuser information theory in the finite blocklength regime are briefly presented. The end of this part is dedicated to a brief introduction to compressive sensing and its applications in networking. Evaluation
Part I: Theoretical FoundationsLecture NotesThe lecture notes are available here.
Homework 1: Deadline by Sep. 27, 2020. 23h59
Homework 2: Deadline by Oct. 4, 2020. 23h59
Homework 3: Deadline by Oct. 11, 2020. 23h59
Homework 4: Deadline by Oct. 25, 2020. 23h59 Part II  A: Applications to Maximum Likelihood Estimation and Model SelectionLecture NotesThe lecture notes are available here.
Homework 5: Deadline by Oct. 31, 2020. 23h59 Part II  B: Applications to Communication TheoryLecture NotesThe lecture notes are available here for Lectures 11  12 and here for Lectures 13  14.
Homework 6: Download it here. Deadline by Nov. 15, 2020. 23h59
Homework 7: Download it here. Deadline by Nov. 24, 2020. 23h59 Student Contest (Final Exams)
