CS 675 Deep Learning

That's cute. But I don't care why it works—if it works, it's useful. If it doesn't, it's just cute.

Resources §

• GitHub - hindupuravinash/the-gan-zoo: A list of all named GANs!
• Machine Learning Mastery

Homeworks §

• CS 675 HW1 ← problem 3 only
• CS 675 HW2

Notes to Improve §

• Finish up MLE derivation & understanding for Probabilistic Generative Models
• Finish bias derivation in NN
• Finish optimizing the logistic regression ^02k7nz
• R-CNN, Faster R-CNN etc.
• deep learning is just a catalogue of tools, and finding which tool to use is the key intuition

Notes §

Background §

• Mathematics
  • Norm (Math)
  • Information Theory
• Training Basics
  • List of Machine Learning Datasets
  • Neural Networks, Backpropagation and Gradient Descent
  • Better Gradient Descent Methods
  • Training Tricks

Supervised Models §

Supervised models minimize the difference (often cross-entropy) between data distribution and ideal target distribution

• Vision
  • Convolutional Neural Networks
  • Object Detection

Unsupervised Models §

Unsupervised models all aim to cleverly extract abstract features from data without labels.

• Linear Factor Models
  • Probabilistic Principle Component Analysis
  • Independent Component Analysis
  • Slow Feature Analysis
  • Sparse Coding and Dictionary Learning
• Autoencoders
  • Variational Autoencoders
• Graphical Models
  • Restricted Boltzmann Machines
• ⭐ Generative Adversarial Network (GANs)

Time Series Modeling §

Cheat Sheet:

1. KL divergence & between two normals equation
2. Complete the square
3. Standard normal multivariate
4. VAE loss