AE04 — Deep Learning Foundations

Understand neural networks, build training loops, debug training, and adapt pretrained models. Practice focuses on text and representation learning. CNNs provide a contrasting architecture to broaden intuition.

Tư vấn học phần này Lịch khai giảng

Prerequisites

AE03 competency and an understanding of gradients, overfitting, and offline evaluation.

Outcomes

A mini language-model checkpoint trained from scratch, a supervised fine-tuned model or adapter, training logs, a model card, and an offline benchmark. Learners complete the S12 and S13 labs during self-study in weeks 6 to 8 using allocated GPUs. Instructors validate the model size in advance to fit time and budget constraints.

Audience

Learners with ML knowledge who want to study DL, NLP, and foundation models.

Core capability

Train and adapt neural networks.

Syllabus

16 sessions in 8 weeks.

Week	First session	Second session
1	S1: PyTorch tensor operations, broadcasting, devices, and autograd	S2: MLP layers, activations, forward passes, and loss
2	S3: Backpropagation and the chain rule in neural networks	S4: Training loops, data loaders, batching, and checkpoints
3	S5: Momentum, Adam, and learning-rate schedules	S6: Initialization, normalization, dropout, and regularization
4	S7: Debugging gradients and learning curves, then overfitting one batch	S8: CNNs and transfer learning through a small image task
5	S9: Tokenization, embeddings, and sequence representation	S10: Attention, including queries, keys, values, and masking
6	S11: Transformer encoders, decoders, and token positions	S12: Language modeling, pretraining objectives, token generation, and launch of a mini language-model training lab
7	S13: Supervised fine-tuning and PEFT or LoRA practice with a small model	S14: Contrastive learning and embedding training
8	S15: Model ablation, memory and compute profiling, and baseline comparison	S16: Demo, error analysis, and model card defense

Lab

Tools & access

Individual GPU quota, checkpoint storage, training recipes, and debugging support.

Evidence you keep

Checkpoint, adapter, training logs, and model card.