SUMS 2020: Bridging the Gap Between Computers and Semantics Through Learned Information Representations

Max Daniels
Northeastern University
[email protected]

Thanks for listening to the talk!

Here is the associated work: Statistical Distances and Their Implications to GAN Training

You can download the slides here.

Edit, Mar. 8: For those who are interested, here are some especially interesting resources related to the talk:

1. Unsupervised Representation Learning:
   1. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric argues that extraction of semantic features is a general emergent phenomena of deep visual representations. (+1 for Magic of Deep Learning)
   2. Image Classfiers are biased towards textures over shapes argues that deep image classifiers are biased towards texture extraction. (-1 for Magic of Deep Learning)
2. Word2Vec:
   • Noise Contrastive Estimation provides some really good insight on why this works at all.
   • The actual Word2Vec paper is a list of interesting experimental results, with two (somewhat hard to understand) novel simplifications to adapt NCE to their setting.
   • Yoav Goldberg's note on Word2Vec explained. This is a better resource than the Word2Vec paper for understanding Word2Vec.
3. GAN Training:
   • Original GAN Paper
   • Generalization to f-Divergences - this is my favorite theoretical explanation of GAN at a high level.
   • Wasserstein GAN + Vincent Hermann's Expository article - WGAN is hot news in GAN training right now. This paper points out a huge theoretical flaw in the original GAN formulation, derives a principled alternative, and uses it to achieve SOTA experimental results. See also the second resource which is a fantastic expository article about WGAN.
4. Compressive Sensing: this is what I spend most of my actual time on. Learn more about image compression, JPEG, MRI machines, and signal processing!
   • the paper that started the field. Candés and Tao are wonderful authors whose works are always fun to read.
   • A Compressed Sensing view of Unsupervised Representation Learning - Arora, another wonderful author whose work is fun to read, connects compressive sensing back to unsupervised representation learning, both having the shared goal of extracting minimal representations of meaningful information.