Generally, whenever I do see frontier AI model's evaluation metrics on say Artificial Analysis. It gives out a benchmark on a set of questions (or tasks). But it never quite actually shows what tasks it fails at. I'm more curious on benchmarks that focus on SWE. Comments URL: https://news.ycombinator.com/item?id=50150861 Points: 1 # Comments: 0