ECE 47300, Intro. to Artificial Intelligence (Spring 2023)

Important Links

• Syllabus (pdf), tentative course schedule (pdf), and prerequisite quiz (ungraded)
• Brightspace (Grades) - For the first time logging into Piazza and Gradescope, please click the links in Brightspace under “Content” and then under the module “Piazza, Gradescope, and Circuit Links”. This will help link your Purdue account with these external learning tools. After the first time, you can just use the links below.
• Piazza (Announcements and discussion)
• Gradescope (Online quizzes, exams, and assignment submission)
• Google Colab (Online computing environment including GPUs)

Optional textbooks

The bracketed acronym is used for referencing these books.

1. [DL] Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, 2016. http://www.deeplearningbook.org
2. [ML] Machine Learning: A Probabilistic Perspective by Kevin P. Murphy, 2012. https://ebookcentral.proquest.com/lib/purdue/detail.action?docID=3339490
3. [PY] Python Data Science Handbook by Jake VanderPlas, 2016. https://jakevdp.github.io/PythonDataScienceHandbook/

Lecture content by week

1. Week 1 - Introduction to artificial intelligence

   • Tuesday: Introduction to AI. See syllabus, course schedule, and course project links above.
   • Thursday: Overview of AI; Started PCA and linear algebra (see notes below)

2. Week 2 - PCA and linear algebra

   • Tuesday: Principal Components Analysis (PCA), Review of linear algebra ( notebook, pdf); Related reading: DL, Ch.2
   • Thursday: See PCA notes; PCA demo ( notebook, pdf )

3. Week 3 - Introduction to machine learning

   • Tuesday: Intro. to ML; PCA generalization demo ( notebook, pdf); Optional related reading: DL, Ch. 5.1
   • Thursday: K-nearest neighbors (KNN) and evaluating ML methods, (pdf); KNN Demo ( notebook, pdf); Optional related reading: KNN Classifier Notes, DL, Ch. 5.2-5.3

4. Week 4 - Linear models and gradient descent

   • Tuesday: Linear and Logistic Regression; Optional related reading: PY, Linear regression, DL, Ch. 5.1.4 (short), ML, Ch. 7 and Ch. 8 (in-depth)
   • Thursday: Gradient Descent; Gradient descent demo ( notebook, pdf); Loss functions and regularization; Optional related reading: ML, Ch. 8 section 8.3

5. Week 5 - Basics of deep learning

   • Tuesday: Broadcasting rules in NumPy (and PyTorch) ( notebook, pdf); Basics of deep learning; PyTorch and automatic differentiation ( notebook, pdf)
   • Thursday: Basics of convolutional neural networks (CNN); Convolutions demo ( notebook, pdf); Optional related reading: DL, Ch. 9

6. Week 6 - Convolutional Neural Networks (CNNs)

   • Tuesday: CNNs continued, CIFAR-10 demo with BatchNorm and residual networks ( notebook, pdf)
   • Thursday: Midterm 1

7. Week 7 - Review of probability

   • Tuesday: Review of probability; Optional related reading: DL, Ch. 3, ML, Ch. 2
   • Thursday: Density estimation

8. Week 8 - Autoencoders

   • Tuesday: Autoencoders and VAEs; Optional related reading: Recent introduction to VAEs by original authors (2019), Original VAE paper (2013), From Variational to Deterministic Autoencoders
   • Thursday: Autoencoders (continued)

9. Week 9 - Generative Adversarial Networks (GAN)

   • Tuesday: Generative Adversarial Networks (GAN); Optional related reading: Original GAN paper, Deep Convolutional GANs
   • Thursday: Deep Convolutional GAN (DCGAN); DCGAN MNIST tutorial ( notebook, pdf); Optional: Transposed convolution visualization ( notebook, pdf)

10. Week 10 - Natural Language Processing

    • Tuesday: DCGAN (continued); Topic Models (LDA); LDA topic model via Gibbs sampling ( classic3 dataset, notebook, pdf); Optional: Original collapsed Gibbs sampling paper for LDA
    • Thursday: Topic models (continued)

11. Week 11 - Natural Language Processing

    • Tuesday: Word embeddings (Word2Vec)
    • Thursday: Midterm 2

12. Week 12 - Natural Language Processing

    • Tuesday: Recurrent Neural Networks (RNN); Demo of character-level RNN classification ( tutorial data, notebook, pdf)
    • Thursday: RNNs (continued); Demo of character-level RNN generation ( tutorial data, notebook, pdf); Demo of vanishing and exploding gradients ( notebook, pdf);

13. Week 13 - Natural Language Processing

    • Tuesday: Attention and Transformers; Demo of seq-2-seq language translation ( notebook, pdf)
    • Thursday: Attention and Transformers (continued)

14. Week 14 - Diffusion Models and Reinforcement Learning

    • Tuesday: Diffusion Models
    • Thursday: Diffusion Models (continued); Introduction to Reinforcement Learning; Optional reading: Chapter 1 of Reinforcement Learing: An Introduction

15. Week 15 - Reinforcement Learning

    • Tuesday: Multi-armed Bandits; Demo of bandit algorithms ( notebook, pdf)
    • Thursday: Markov Decision Processes