Week 1 — Introduction to AI Engineering
Day 1 is the map. No laptops, no setup — just the story of how we got from "machines that talk" in science fiction to the AI agents you'll build this summer, and the vocabulary to understand every piece of one.
How today works: ideas, not configuration. Every topic opens with a question to the room before any answers — wrong answers are fuel. All the live demos are run from the front; your hands-on starts next week.
Agenda
Morning · 9:00 AM – 1:00 PM · The road to the transformer
- What is AI Engineering? — one discipline with three faces: building with AI, building software that contains AI, and deciding what to hand a model vs. what to write as code.
- The original dream — machines you simply talk to (a little Star Trek), imagined decades before they existed.
- The detour — the mechanical world built calculators instead: Babbage, Turing's universal machine, von Neumann's stored program.
- Programming, 100% deterministic — same input, same output, every time. The abstraction ladder from punch cards → assembly → modern languages → and, eventually, plain English.
- We tried to program the dream — and failed — hand-written rules, ELIZA, and the AI winters. Maybe intelligence has to be learned, not written down.
- Machine learning: the opposite of programming — instead of rules + data → answers, feed it data + answers → rules. Spam filters, price prediction, recommendations.
- Neural networks & deep learning — from neurons to handwriting recognition to the AlexNet moment that reorganized the whole field.
- But language stayed hard — why reading one word at a time (RNNs) wasn't enough.
- Transformers: "Attention Is All You Need" — every word can look at every other word at once. The one thing the whole machine does: predict the next token — the fact that explains everything else today.
- Scaling laws — why bigger + more data + more compute reliably wins, and why that means datacenters.
Working lunch · 1:00 – 2:00 PM
Afternoon · 2:00 – 6:00 PM · The LLM era and the agentic turn
- The ChatGPT moment — the interface revolution, and the honest weaknesses of an LLM used as a pure chatbot: frozen knowledge, hallucinations, non-determinism, and no real memory.
- How models are trained — pretraining gives knowledge; post-training (RLHF) gives usefulness and personality. Why Claude, ChatGPT, and Gemini feel different.
- Prompting is programming in natural language — zero-/single-/multi-shot, in-context learning, and why "think step by step" (chain-of-thought) measurably helps.
- Reasoning models, MoE & the DeepSeek moment — training the thinking in, making it cheap, and why one Chinese lab shook the market.
- How to choose a model — the real axes (size, reasoning, open vs. closed, multimodal, cloud vs. local) so you can place whatever launches next month — plus a tour of the non-text generative world (image, video, speech).
- RAG: thawing frozen knowledge — bring the knowledge to the model; meaning as coordinates in space (the RGB/HSL analogy for embeddings).
- Structured outputs — stop treating the model as a chat partner; treat it as a function. Messy input in, schema-shaped data out.
- Tool calling — a model that only emits text gets hands: it requests an action, your app carries it out, the result comes back.
- Agents: the loop that thinks and acts — reason → act → observe → repeat. Where "agency" comes from, how memory is engineered, and how MCP wires tools in. The keystone everything this summer builds on.
- Wrap-up & what's next — loop back to "what is AI engineering" with every word now earned, the capstone templates to read tonight, and a look ahead: next week we stop talking and start building.
Reference material — Day 1 lecture
Background reading and watching from the Day 1 lecture. Work through these to build intuition for how modern AI models work.
- Neural networks — visual primer (3Blue1Brown video series) — the best intuition-builder for what a neural network actually does.
- The Illustrated Word2Vec — a visual primer on word vector embeddings.
- AlexNet, visualized (video) — why AlexNet was the inflection point for deep neural networks.
- AI Capabilities & Limitations — a quick primer from Anthropic.
- Sparks of AGI (Microsoft, PDF) — the paper documenting surprising observations about GPT-4.
- Prompt Engineering Guide — Techniques — authoritative reference including Chain of Thought.
- 70 Years of AI — From Perceptrons to ChatGPT (Visual Breakdown) — the whole arc of how we got here, from the first perceptrons to modern chatbots.
- The Algorithm That Teaches AI (Visual Breakdown) — how models actually learn: gradient descent and backpropagation, made visual.
- The Math That Makes AI Feel Smart (Visual Breakdown) — the linear algebra behind why models seem intelligent.
- Attention Is All You Need — Finally Explained Visually (Visual Breakdown) — a visual walkthrough of the transformer architecture that powers every modern LLM.
- What is a Vector Database? (Visual Breakdown) — how embeddings get stored and searched — the foundation for retrieval (previews Weeks 3–5).
- Inside a RAG System (Visual Breakdown) — how retrieval-augmented generation wires real sources into an LLM (previews Weeks 3–5).