Local LLM coding-agent benchmark gallery

An AI coding agent (pi) was given one prompt in an empty folder and left to work on its own. The models were open-weight ones running locally on a Mac Studio (M2 Ultra, 64 GB), plus a few cloud models for comparison. Two tasks:

Astro blog: “Create a static site for my personal blog using Astro and populate it with 4-5 sample posts.”
Tetris: “Create a playable Tetris game that runs in the web browser, using only plain HTML, CSS and JavaScript (no frameworks, libraries or build step).”

Scoring. Each run gets an automated score out of 100 (does it install, build, have pages, links, content…), plus a code review by Claude Opus giving a verdict and 1–5 ratings for idiomatic use of the framework and overall quality. Caveat: the Tetris automated checks are coarse and can pass games a human would find broken, so lean on the review and the screenshots there. Runs were done in four batches: a model survey, environment tuning, a few probes, and a final confirmation of the best setup.