System Security Research Methods
- Level
- Graduate
- Status
- Under development
Note: this course is still being designed and calibrated; the page content is for reference and does not represent the final teaching version.
Syllabus
Course Overview
System Security Research Methods trains graduate students to conduct security research through problem formulation, experiment design, reproduction, measurement, writing, and ethics. The course emphasizes verifiable questions over tool accumulation, and stresses counterexamples, threat validity, and reproducibility.
This is the graduate methodological entry course for security and should be taken before or alongside topic courses.
Prerequisites
- At least one systems or security course.
- Ability to read English systems-security papers.
- Programming, lab-record, and technical-writing fundamentals.
Learning Outcomes
- Extract problems, assumptions, methods, and evidence chains from papers.
- Design reproducible and ethically compliant experiments.
- Evaluate measurement error, selection bias, and external validity.
- Write clear research questions, threat models, and limitations.
- Complete a small paper reproduction or pilot study.
Course Format
- Two meetings per week: one for core concepts and one for labs, paper discussion, or project review.
- The course proceeds over 16 weeks, each with a checkable assignment, lab, or project milestone.
- Reproducibility is required: code, configuration, data, lab logs, and reports must be reviewable by staff or peers.
Weekly Plan
Security research questions and paper structure
Decompose a top-tier paper's question and evidence chain.
Security research questions and paper structure: lab and review
Decompose a top-tier paper's question and evidence chain. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Threat models, ethics, and IRB intuition
Write ethics and risk notes for an experiment.
Threat models, ethics, and IRB intuition: lab and review
Write ethics and risk notes for an experiment. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Reproduction, baselines, and artifacts
Reproduce one small result from a paper.
Reproduction, baselines, and artifacts: lab and review
Reproduce one small result from a paper. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Measurement, statistics, and visualization
Clean experimental data and report uncertainty.
Measurement, statistics, and visualization: lab and review
Clean experimental data and report uncertainty. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Systems experiment design
Design ablation, stress, and robustness experiments.
Systems experiment design: lab and review
Design ablation, stress, and robustness experiments. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Writing, reviewing, and rebuttal
Complete an anonymous peer review.
Writing, reviewing, and rebuttal: lab and review
Complete an anonymous peer review. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Research prototypes and engineering debt
Clean code, scripts, and reproduction instructions.
Research prototypes and engineering debt: lab and review
Clean code, scripts, and reproduction instructions. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Pilot-study report
Submit a paper-style pilot report and artifact.
Pilot-study report: lab and review
Submit a paper-style pilot report and artifact. Complete the paired lab, record issues, and explain design tradeoffs in class review.
Assessment
Concept questions, reading responses, design tasks, and small programming or lab exercises.
Paper reproduction, experiment design, technical writing, and artifacts.
Participation in discussions, demos, code or paper reviews, and peer feedback.
Submit reproducible artifacts, a technical report, and a demo explaining methods, results, limitations, and future work.
Course Project
Projects may be paper reproduction, public-data reanalysis, tool prototypes, or pilot studies. They must include a clear question, baseline, reproduction steps, limitations, and ethics notes.
Policies
- AI tools are allowed, but generated code, lab notes, and design suggestions must be reviewed by the student and disclosed in the report.
- Students may not submit code, proofs, configuration, or experimental results they cannot explain; each member must defend their own design, tests, and tradeoffs.
- Late work affects iteration grades, but the course values reproducible, auditable, and maintainable results over last-minute accumulation.
Reference Courses
International
- CMU15-799: Security for Software and Hardware Systems
- Georgia TechCS 6265: Information Security Lab
- MIT6.5660 Computer Systems Security
- UC BerkeleyCyber 215: Usable Privacy and Security Research
- University of WashingtonCSE 564: Computer Security
China 985 Universities
- 中国科学技术大学网络空间安全学院
- 南京大学COSEC 系统安全科研组
- 南京大学Dragon Star: System Security and Binary Code Analysis
- 复旦大学系统软件与安全实验室
- 浙江大学网络空间安全研究中心