Class Project

The class project allows you to explore a relevant topic in an open-ended way. You will pick a topic at the intersection of AI and security. It is worth 25% of the grade (see the syllabus).

Work alone or in a group of up to three. I expect a three-person project to be significantly more ambitious than a solo one. Budget 20–30 hours per person across the back half of the semester; roughly three to five hours a week across the span of 4-5 weeks.

Proposal: due Tuesday, Oct. 27, 11:59 PM

The proposal is one page. I will grade it lightly. The point is to catch a bad fit early so that I can tell you what to fix.

1. What are you building, breaking, or exploring? What is your resulting artifact? Is it an experiment, evaluation, etc.?

2. Give specifics about systems, datasets, models, you will use, etc.

3. How will you measure success? You should propose a rigorous evaluation framework that carefully discusses trade-offs.

4. What’s the fallback if the ambitious version fails? You still need to finish something. Try building your project like an onion: “obviously going to work,” “more ambitious but we can probably do it,” “stretch goals.”

Some potential project directions…

Here are some starting points for you to think about

• AI-assisted bug finding. Point an agent at code with planted bugs; report precision and recall against ground truth, versus a conventional tool.

• Discovery to exploitation. Measure how far an agent gets from “a bug exists” to a working proof-of-concept exploit in a sandbox.

• Detector vs. evasion. Train a detector (malware, phishing, intrusion), then attack it.

• Break an AI-integrated app. Build a small agentic app, then attack it with prompt injection, tool misuse, or exfiltration. Implement defenses and re-evaluate.

• Reproduce and critique. Take a published security-and-AI result and try to reproduce it.

Be realistic about your ability. I expect a more from a PhD student in security / AI versus an undergraduate taking the course as an elective.

Full report: due Dec. 8

• Code and artifacts, with a README that lets someone re-run your evaluation.

• A short report (~6–10 pages): threat model, what you built, method, results with denominators, what failed, and what’s next.

• If necessary, an evaluation platform that allows me to run your results. If this is too hard, you may schedule a Zoom appointment with me to demonstrate your project working live.

Presentation: Late November / early December

• We’ll reserve one or two class sessions for class projects.

Rubric (Tentative)

Criterion	Weight	Criteria
Threat model & framing	15%	I expect you to clearly articulate the threat model in your report / evaluation.
Technical execution	30%	Did you actually achieve the thing you said you would?
Evaluation	25%	I expect a rigorous evaluation that carefully measures the system; bad results are OK.
Honesty about limits	10%	What failed, where your result is fragile.
Presentation	20%	Clear, honest, and appropriately-timed.

• Do not do anything illegal / improper!. Since this class involves attacks, etc. All offensive research must be carefully cleared with the instructor. If you are doing something that “feels potentially wrong,” email me specifically with a threat evaluation plan so that we can discuss if it is still OK to do your project.

• Do not spend (a significant amount of) personal money!. Try to stick to free platforms like Google Collab. If you need GPU infrastructure, please reach out to me personally and we can see if we can find something suitable for you. It is possible to rent GPUs (e.g., RunPod, etc.) but it gets expensive quickly; you should not be spending a lot of your own money to do this.