Skip to content

L2.1.1 — Threat modeling fundamentals reviewed

Type: Theory · Duration: ~4 min · Status: Mandatory Module: Module 2 — AI Security Foundations Framework tags: foundational for the module — enables NIST AI RMF Map function

Learning objectives

By the end of this lesson, the learner can: 1. Define threat modeling in one sentence and name the four canonical questions it answers. 2. Recognize three common output artifacts (DFD, threat list, mitigation map) and when each is used.

Concept primer

This lesson is for the ML/AI-side cohort — security engineers can skim. We're not deriving threat modeling from scratch; we're putting it on a single page so the rest of the module has a shared vocabulary.

Core content

Threat modeling is the structured exercise of answering four questions about a system before attackers do:

  1. What are we building? (system model — usually a data-flow diagram, plus a trust-boundary overlay)
  2. What can go wrong? (threat enumeration — STRIDE, attack trees, abuse cases)
  3. What are we going to do about it? (mitigations — controls mapped to threats)
  4. Did we do a good enough job? (validation — review, testing, residual-risk acceptance)

It's a practice, not a tool. You can do threat modeling on a napkin or in a $50k governance suite; the output is the same shape.

Three artifacts you'll produce or consume:

  • Data-flow diagram (DFD). Boxes for processes, cylinders for data stores, arrows for data flows, dotted lines for trust boundaries. The minimum viable system model. Every threat lives on an arrow that crosses a trust boundary.
  • Threat list / threat table. One row per threat, columns for category (e.g., STRIDE letter), description, impact, likelihood, status. The deliverable a reviewer reads.
  • Mitigation map. Threats on one axis, controls on the other, cells indicating coverage. Surfaces gaps where a threat has no mitigation, or a control protects nothing.

In Module 2 we'll adapt all three for AI systems. The data-flow diagram gets new node types (model, embedding index, guardrail). STRIDE gets two extra categories tuned for AI. The mitigation map gets a column for "framework reference" so you can cite OWASP / ATLAS / NIST when you justify a control.

Real-world example

Microsoft's Security Development Lifecycle has required threat modeling on every product since the early 2000s. It is the single most-adopted practice from that program — the one practice that ships in the SDL of organizations that adopt nothing else from Microsoft. Replicate the practice; ignore the marketing.

Key terms

  • Trust boundary — the perimeter across which data crosses from one trust level to another (e.g., user → app, app → model API).
  • Data-flow diagram (DFD) — system-model artifact: processes, data stores, flows, trust boundaries.
  • Mitigation map — matrix showing which controls address which threats.

References

  • Adam Shostack — Threat Modeling: Designing for Security (Wiley, 2014). The canonical text.
  • Microsoft Threat Modeling Tool — https://learn.microsoft.com/en-us/azure/security/develop/threat-modeling-tool

Quiz items

  1. Q: What are the four questions threat modeling answers? A: What are we building? What can go wrong? What are we going to do about it? Did we do a good enough job?
  2. Q: What's the minimum viable artifact for "what are we building"? A: A data-flow diagram with trust boundaries.

Video script (~480 words, ~3.5 min)

[SLIDE 1 — Title]

If you came in from the ML side, this lesson is for you. If you came in from the security side, skim and confirm I'm setting the right baseline. Either way, we're putting threat modeling on a single page so the rest of this module has a shared vocabulary.

[SLIDE 2 — The four questions]

Threat modeling is the structured exercise of answering four questions about a system before attackers do. One: what are we building? Two: what can go wrong? Three: what are we going to do about it? Four: did we do a good enough job? That's it. Threat modeling is a practice, not a tool. You can do it on a napkin or in a fifty-thousand-dollar governance suite. The output is the same shape.

[SLIDE 3 — Three artifacts]

Three artifacts you'll produce or consume. First, a data-flow diagram. Boxes for processes, cylinders for data stores, arrows for data flows, dotted lines for trust boundaries. Every threat lives on an arrow that crosses a trust boundary. That's the load-bearing observation of the entire practice.

Second, a threat list. One row per threat, columns for category, description, impact, likelihood, and status. This is the deliverable a senior reviewer actually reads. The DFD shows the system; the threat list shows what you found.

Third, a mitigation map. Threats on one axis, controls on the other, cells indicating coverage. This surfaces gaps where a threat has no mitigation, or a control protects nothing. It's also the artifact auditors expect.

[SLIDE 4 — What changes for AI]

In this module we'll adapt all three for AI systems. The data-flow diagram gets new node types: model, embedding index, guardrail. STRIDE — which we cover next lesson — gets two extra categories tuned for AI. The mitigation map gets a column for framework reference so you can cite OWASP, ATLAS, NIST when you justify a control.

[SLIDE 5 — One historical anchor]

One historical anchor. Microsoft's Security Development Lifecycle has required threat modeling on every product since the early two-thousands. It is the single most-adopted practice from that program — the one practice that ships in the SDL of organizations that adopt nothing else from Microsoft. Replicate the practice. Ignore the marketing.

[SLIDE 6 — Up next]

Next lesson, we adapt STRIDE for AI. Four minutes. See you there.

Slide outline

  1. Title — "Threat modeling fundamentals reviewed".
  2. The four questions — four numbered cards, large type.
  3. Three artifacts — DFD sketch · threat-list table · mitigation map matrix. Side by side.
  4. What changes for AI — same three artifacts with AI annotations called out in red.
  5. Microsoft SDL anchor — short historical timeline, callout: "Replicate the practice; ignore the marketing."
  6. Up next — "L2.1.2 — STRIDE adapted for AI systems, ~5 min."

Production notes

  • Recording: ~3.5–4 min target. Hard cap 5.
  • Tone: brisk, presumptive of competence. This is the lesson where security learners might be slightly bored — keep it moving.