Current status: attempting to run my scraping/reverse-engineering benchmark prompt against DeepSeek 4 Pro via Ollama, bu...

Current status: attempting to run my scraping/reverse-engineering benchmark prompt against DeepSeek 4 Pro via Ollama, but their servers are melting, as one might expect. So I'm having to nudge it along.So far no open-weights model (including Kimi K2.6) has completed the benchmark (basically figuring out how to build a tool to build Apollo GraphQL hashes when pointed at a site's minified JS that builds those hashes.Sonnet 4.6 gets to a solution after multile hours of churning, but the run time of the resulting script is a couple minutes. Opus 4.6 was the first one that solved the problem (a few hours and two prompts), and run time of that script was a couple seconds. Opus 4.7 at Medium effort took an hour or so and one-shot this at the lower runtime. GPT 5.5 took maybe 12 minutes and one-shot it at the lower runtime. Other proprietary models that I tried (GPT-5.4, Gemini Pro 3.1) failed.Testing on closed models was via GitHub Copilot. Open-weights testing was either via OpenCode Go or O...