Code Comments Slop
How often models insert decorative comment separators in generated code
Measures models' tendency to insert decorative comment patterns in generated code. A comment line is flagged if it contains 4+ consecutive repeating characters.
- Slop Rate — percentage of files containing at least one decorative comment
- Decorative Comments/File — average number of flagged comment lines per file
- Decorative Comments/100 Lines — flagged lines normalized by file length
50 prompts across Python, TypeScript, and Go, each requesting a small multi-stage utility script (CLI tools, data pipelines, file processors) with enough logical sections to tempt decorative separators.
| Model | Rate | Per File | Per 100 Lines |
|---|---|---|---|
| Kimi-K2.5 | 4.0% | 0.08 | 0.020 |
| MiniMax-M2.1 | 4.0% | 0.18 | 0.068 |
| GLM-4.7 | 16.0% | 0.78 | 0.327 |
| Claude Sonnet 4.6 | 18.0% | 3.12 | 0.389 |
| Claude Opus 4.6 | 22.0% | 3.48 | 0.344 |
| DeepSeek V3.2 | 6.0% | 0.08 | 0.025 |
| Llama 3.3 70B | 0.0% | 0.00 | 0.000 |
| GPT-OSS 20B | 84.0% | 6.64 | 2.302 |
| GPT-5.3 Codex | 14.0% | 1.10 | 0.326 |
| GPT-5.4 | 0.0% | 0.00 | 0.000 |
| GPT-OSS 120B | 96.0% | 12.86 | 3.292 |
| GLM-5 | 20.0% | 0.42 | 0.061 |
Last updated 12 March 2026 at 02:28