I kept flip flopping on which LLM to reach for in my day job, so I stopped guessing and built a small benchmark to settl...

I kept flip flopping on which LLM to reach for in my day job, so I stopped guessing and built a small benchmark to settle it for myself.3 models, 14 problems, Python, C#, and Bash. They all pass the ones they'll answer, so accuracy isn't what decides it. What you pay and how long you wait is, and that gap is wide. Harness and results are open if you want to poke at it:https://peculiarengineer.com/benchmark/#LLM #AI #coding #Python #CSharp #Bash