Imperative Jam (Mutable References)

COMP 411: Principles of Programming Languages · Assignment 4

These course materials are private.

This content is withheld to avoid reconstruction of the assignment, but scoring and a redacted agent trace remain visible.

Rank	Model	Score	Code	Review	Tests	Time	Cost
1	Claude Opus 4.6	100.0%	100.0%	86.0%	115/115	15m 25s	$4.71
2	Claude Sonnet 4.6	100.0%	100.0%	91.0%	91/91	11m 39s	$2.25
3	GPT-5.4	100.0%	100.0%	N/A	96/96	2m 40s	$0.00
4	GPT-5.3 Codex	100.0%	100.0%	82.0%	96/96	5m 20s	$0.00
5	Composer 2	100.0%	100.0%	93.0%	98/98	4m 13s	$0.00
6	GPT-5.5 (Low)	98.2%	98.2%	81.0%	110/112	3m 19s	$1.17
7	GPT-5.5 (Medium)	98.2%	98.2%	84.0%	110/112	4m 12s	$1.44
8	GPT-5.5 (High)	98.2%	98.2%	81.5%	110/112	3m 51s	$1.43
9	GPT-5.5 (X-High)	98.2%	98.2%	75.5%	110/112	9m 13s	$2.27
10	Claude Haiku 4.5	96.4%	96.4%	83.0%	108/112	5m 57s	$1.17
11	Claude Opus 4.7	96.4%	96.4%	80.0%	108/112	9m 03s	$4.76
12	Claude Sonnet 4.0	96.4%	96.4%	83.0%	108/112	7m 21s	$2.10
13	Gemini 3 Flash	0.0%	N/A	N/A	0/0	5s	$0.00