LEADERBOARD
Public board
Benchmark v0.2.1 · 535 cases · equal-weight VPI + per-track columns. Mini pilot rows are labeled; full-matrix rows will mark split=full.| Rank | System | Type | Ver | VPI | Change | Text | Confusable | Count | Color | Size |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | openai/gpt-5.6-luna | VLM mini | 0.2.1 mini | 0.855 | 0.795 | 1.000 | 0.417 | 1.000 | 1.000 | 0.917 |
| 2 | openai/gpt-5-nano | VLM mini | 0.2.1 mini | 0.778 | 0.000 | 1.000 | 0.667 | 1.000 | 1.000 | 1.000 |
| 3 | google/gemini-3.1-flash-lite-preview | VLM mini | 0.2.1 mini | 0.736 | 0.000 | 1.000 | 0.667 | 0.917 | 1.000 | 0.833 |
| 4 | meta-llama/llama-4-scout | VLM mini | 0.2.1 mini | 0.704 | 0.000 | 0.904 | 0.583 | 1.000 | 0.818 | 0.917 |
| 5 | google/gemini-3.5-flash-lite | VLM mini | 0.2.1 mini | 0.702 | 0.000 | 0.959 | 0.417 | 1.000 | 1.000 | 0.833 |
| 6 | google/gemini-2.5-flash-lite | VLM mini | 0.2.1 mini | 0.694 | 0.000 | 1.000 | 0.333 | 1.000 | 1.000 | 0.833 |
| 7 | google/gemini-2.5-flash | VLM mini | 0.2.1 mini | 0.667 | 0.000 | 1.000 | 0.417 | 0.750 | 1.000 | 0.833 |
| — | pixel_oracle | classical CV | 0.2.1 full | — | 0.692 | — | — | — | — | — |
| — | perfect_gt | scorer ceiling | 0.2.1 full | — | 1.000 | — | — | — | — | — |
| — | random | chance floor | 0.2.1 full | — | 0.000 | — | — | — | — | — |