ECE 57000: Artificial Intelligence (Fall 2023)

Important Links

• Syllabus (pdf), tentative course schedule (pdf), and prerequisite quiz (ungraded)
• Brightspace (Grades) - For the first time logging into Piazza, Gradescope, or Circuit, please click the links in Brightspace under “Content” and then under the module “Piazza, Gradescope, and Circuit Links”. This will help link your Purdue account with these external learning tools. After the first time, you can just use the links below.
• Piazza (Announcements and discussion)
• Gradescope (Quizzes and assignment submission)
• Circuit (Project checkpoint submission and peer reviews)
• Google Colab (Free computing environment including GPUs)

Project Information

• Course project instructions
  • Tutorials on specific research topics: TBD
  • A few ideas for selecting papers: Lecture snippet from Fall 2022, Corresponding slides
  • Checkpoint 1 - Preliminary implementation and results
  • Checkpoint 2 - Review of 3 papers and implementation update
  • Logistics for final project deliverables

Optional textbooks

The bracketed acronym is used for referencing these books.

1. [DL] Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville, 2016. http://www.deeplearningbook.org
2. [ML] Machine Learning: A Probabilistic Perspective by Kevin P. Murphy, 2012. https://ebookcentral.proquest.com/lib/purdue/detail.action?docID=3339490
3. [PY] Python Data Science Handbook by Jake VanderPlas, 2016. https://jakevdp.github.io/PythonDataScienceHandbook/
4. [DD] Dive into Deep Learning by Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola, 2023. https://d2l.ai/

Lecture content by week

1. Week 1 - Introduction to artificial intelligence
   • Monday: Introduction to AI. See syllabus, course schedule, and course project links above.
   • Wednesday: (continued)
   • Friday: Principal Components Analysis (PCA), Review of linear algebra (notebook, pdf); Related reading: DL, Ch.2
2. Week 2 - PCA and linear algebra
   • Monday: (continued)
   • Wednesday: (continued), PCA demo (notebook, pdf )
   • Friday: Intro. to ML; PCA generalization demo (notebook, pdf); Optional related reading: DL, Ch. 5.1
3. Week 3 - Machine learning
   • Monday: Labor day (no class)
   • Wednesday: K-nearest neighbors (KNN) and evaluating ML methods, (pdf); KNN Demo (notebook, pdf); Optional related reading: KNN Classifier Notes, DL, Ch. 5.2-5.3
   • Friday: Linear and Logistic Regression; Optional related reading: PY, Linear regression, DL, Ch. 5.1.4 (short), ML, Ch. 7 and Ch. 8 (in-depth)
4. Week 4 - Linear models and gradient descent
   • Monday: (continued)
   • Wednesday: Gradient Descent; Gradient descent demo (notebook, pdf); Loss functions and regularization; Optional related reading: ML, Ch. 8 section 8.3
   • (Optional) Thursday: Distribution Alignment Tutorial
   • Friday: Broadcasting rules in NumPy (and PyTorch) (notebook, pdf)
5. Week 5 - Basics of deep learning
   • Monday: Basics of deep learning; PyTorch and automatic differentiation (notebook, pdf)
   • Wednesday: Basics of convolutional neural networks (CNN); Convolutions demo (notebook, pdf); Optional related reading: DL, Ch. 9
   • Friday: CNNs continued, CIFAR-10 demo with BatchNorm and residual networks (notebook, pdf)
6. Week 6 - Natural Langauge Processing
   • Monday: (continued)
   • Wednesday: Recurrent Neural Networks (RNN); Demo of character-level RNN classification (tutorial data, notebook, pdf)
   • Friday: (continued); Demo of character-level RNN generation (tutorial data, notebook, pdf); Demo of vanishing and exploding gradients (notebook, pdf);
7. Week 7 - Natural Langauge Processing
   • Monday: Attention and Transformers; Demo of seq-2-seq language translation (notebook, pdf)
   • Wednesday: Attention and Transformers (continued)
   • Friday: Attention and Transformers (continued)
8. Week 8 - Review of probability
   • Monday: (fall break)
   • Wednesday: Review of probability; Optional related reading: DL, Ch. 3, ML, Ch. 2
   • Friday: (continued)
9. Week 9 - Density estimation and autoencoders
   • Monday: Density Estimation
   • Wednesdasy: (continued)
   • Friday: Autoencoders and VAEs; Optional related reading: Introduction to VAEs by original authors (2019), Original VAE paper (2013), From Variational to Deterministic Autoencoders
10. Week 10 - Clustering and GANs
    • Monday: Clustering; K-means clustering demo (notebook, pdf)
    • Wednesday: Spectral clustering (see clustering slides); Optional related reading: ML, Ch 25.4.3
    • Friday: Generative Adversarial Networks (GAN); Optional related reading: Original GAN paper, Deep Convolutional GANs
11. Week 11 - GANs and Intel guest lectures
    • Monday: (continued)
    • Wednesday: Intel guest lecture by Bob Chesebrough - AI use-cases in the real world
    • Friday: Intel guest lecture by Eduardo Alvarez - Introduction to MLOps and LLM Pipelines
12. Week 12 - Reinforcement Learning
    • Monday: Deep Convolutional GAN (DCGAN); DCGAN MNIST tutorial (notebook, pdf); Optional: Transposed convolution visualization (notebook, pdf)
    • Wednesday: Introduction to Reinforcement Learning; Optional reading: Chapter 1 of Reinforcement Learing: An Introduction
    • Friday: Multi-armed Bandits; Demo of bandit algorithms (notebook, pdf)
13. Week 13 - Reinforcement Learning
    • Monday: (Bandits continued), Markov Decision Processes
    • Wednesday: (MDPs continued)
    • Friday: Diffusion Models