55AA-019

System Security Research Methods

Level
Graduate
Status
Under development

Note: this course is still being designed and calibrated; the page content is for reference and does not represent the final teaching version.

Syllabus

Course Overview

System Security Research Methods trains graduate students to conduct security research through problem formulation, experiment design, reproduction, measurement, writing, and ethics. The course emphasizes verifiable questions over tool accumulation, and stresses counterexamples, threat validity, and reproducibility.

This is the graduate methodological entry course for security and should be taken before or alongside topic courses.

Prerequisites

  • At least one systems or security course.
  • Ability to read English systems-security papers.
  • Programming, lab-record, and technical-writing fundamentals.

Learning Outcomes

  • Extract problems, assumptions, methods, and evidence chains from papers.
  • Design reproducible and ethically compliant experiments.
  • Evaluate measurement error, selection bias, and external validity.
  • Write clear research questions, threat models, and limitations.
  • Complete a small paper reproduction or pilot study.

Course Format

  • Two meetings per week: one for core concepts and one for labs, paper discussion, or project review.
  • The course proceeds over 16 weeks, each with a checkable assignment, lab, or project milestone.
  • Reproducibility is required: code, configuration, data, lab logs, and reports must be reviewable by staff or peers.

Weekly Plan

Week 2

Security research questions and paper structure: lab and review

Decompose a top-tier paper's question and evidence chain. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Week 4

Threat models, ethics, and IRB intuition: lab and review

Write ethics and risk notes for an experiment. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Week 6

Reproduction, baselines, and artifacts: lab and review

Reproduce one small result from a paper. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Week 8

Measurement, statistics, and visualization: lab and review

Clean experimental data and report uncertainty. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Week 10

Systems experiment design: lab and review

Design ablation, stress, and robustness experiments. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Week 12

Writing, reviewing, and rebuttal: lab and review

Complete an anonymous peer review. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Week 14

Research prototypes and engineering debt: lab and review

Clean code, scripts, and reproduction instructions. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Week 16

Pilot-study report: lab and review

Submit a paper-style pilot report and artifact. Complete the paired lab, record issues, and explain design tradeoffs in class review.

Assessment

Individual assignments

Concept questions, reading responses, design tasks, and small programming or lab exercises.

25%
Labs and project

Paper reproduction, experiment design, technical writing, and artifacts.

40%
Participation and review

Participation in discussions, demos, code or paper reviews, and peer feedback.

10%
Final report and defense

Submit reproducible artifacts, a technical report, and a demo explaining methods, results, limitations, and future work.

25%

Course Project

Projects may be paper reproduction, public-data reanalysis, tool prototypes, or pilot studies. They must include a clear question, baseline, reproduction steps, limitations, and ethics notes.

Policies

  • AI tools are allowed, but generated code, lab notes, and design suggestions must be reviewed by the student and disclosed in the report.
  • Students may not submit code, proofs, configuration, or experimental results they cannot explain; each member must defend their own design, tests, and tradeoffs.
  • Late work affects iteration grades, but the course values reproducible, auditable, and maintainable results over last-minute accumulation.

Reference Courses