Prerequisites
AE03 and AE04 competency, plus API, HTTP, SQL, and Git knowledge. A pre-module assignment checks familiarity with containers, queues, and basic distributed systems.
AE05Months 9–1016 sessions · 8 weeksMentor Big Tech
Design for production. Defend every trade-off.
AE05 — ML and GenAI System Design
Learners design, present, and validate an ML or GenAI architecture under product constraints. The module supports both production work and ML or AI system design interviews.
Outcomes
Time-limited lab demo, design document, and operations report.
Audience
Software, AI, and ML engineers preparing for system design interviews or production AI work.
Core capability
Design ML and GenAI systems under product constraints.
Syllabus
16 sessions in 8 weeks.
| Session | Focus | Exercise / deliverable |
|---|---|---|
| S1 | Step 1: Define the problem and product constraints | Define users, workflows, scale, and budget. Record assumptions that need validation and assess whether ML is necessary |
| S2 | Step 2: Connect model quality to product outcomes | Build an objective tree with business goals, offline metrics, and operational limits. Explain conflicts between metrics |
| S3 | Step 3: Draft the minimum architecture | Draw request and training flows. Define interfaces and independently replaceable components |
| S4 | Step 4: Design data sources and flows | Create data contracts, labeling plans, storage, and lineage. Handle late data and schema changes |
| S5 | Step 5: Organize features in the system | Specify where and when features are computed and versioned. Check point-in-time correctness and serving consistency |
| S6 | Step 6: Make model development and acceptance decisions | Use AE03 and AE04 benchmarks to select a model. Design training jobs, a registry, reproducibility controls, and release criteria |
| S7 | Step 7: Design the prediction service | Define an API contract and per-component latency budget. Choose synchronous or asynchronous serving for each use case |
| S8 | Step 8: Validate online and release a version | Design experiment groups, guardrail metrics, stopping criteria, and rollback paths |
| S9 | Step 9: Scale and maintain the system | Create a capacity plan, alerts, incident responsibilities, and data or model update rules. Validate them against compute limits |
| S10 | GenAI: Select a model and knowledge strategy | Write a decision record comparing retrieval, long context, model adaptation, and tool-provided data. Choose an API or self-hosting under stated constraints |
| S11 | GenAI: Design verifiable RAG | Design index updates, hybrid retrieval, reranking, and citations. Test expired documents and insufficient user permissions |
| S12 | GenAI: Budget for inference | Compare caching, batching, routing, and precision on a given workload. Separate time to first token from total completion time |
| S13 | GenAI: Design quality evaluation | Create cases with reference answers or criteria, calibrate judges with human labels, and block regressions before release |
| S14 | GenAI: Design operations and execution boundaries | Review failure budgets, sensitive data, and access requirements. Define retries, idempotency, traces, and tool contracts for an agent treated as an external component |
| S15 | ML case study: Recommendation or ranking | Apply all nine steps, defend diagrams and estimates, and load-test a deployed slice to validate assumptions |
| S16 | GenAI case study: Mock interview and design defense | Present an LLM or RAG system under changing requirements. Explain the effects on quality, latency, cost, and operations |
Lab
Tools & access
Deployment sandbox, API budget, load-testing tools, and monitoring tools.
CV
Evidence you keep
Time-limited lab demo, design document, and operations report.