How good is Ox Alpha, the new 'stealth' model?

By David Greenwald |

TL;DR:

  • Mystery LLM Ox Alpha was released for free on Aug. 20 and is currently topping the OpenRouter charts
  • The private Greenwald Benchmark puts it at near Fable levels, at 4x the token burn
  • It struggles with hallucinations and extra instructions
  • Try it while you can (if you don’t mind sharing your data)

Unless you’ve been watching the OpenRouter leaderboards like they’re the MLB standings, you probably missed the release of the most intriguing LLM of the summer: Ox Alpha, the alias of a secret new model released by a “stealth” company on Aug. 20.

In just a few days, it ripped to the top of OpenRouter: at the time of this writing, it is up to 17.5 trillion tokens used today.

It helps that is also free this week, with use unlimited outside of intermittent outages—today’s jaw-dropping token burn happened despite hours of downtime. The catch: OpenRouter claims the model won’t train on your data, but it will be retained by the provider. For… mystery reasons.

Theories of its provenance abound here—there’s evidence it could be one of Zhipu AI’s GLM models, and Google’s Deepmind team have also been hinting at ownership—but this is a post about something more important. Is it any good?

It is, and here’s how I know.

Ox Alpha’s Greenwald Eval

Over the last few weeks, I’ve put together a set of evals modeling my real-world use-cases: web research, code exploration, fact-checking, adversarial review, and code writing a number of cases from SQL to Swift. For each case, I tested multiple scenarios: prompt-only, the base rules I use in CLAUDE.md, a custom skill (if applicable), and base plus skill. The goal here was not just to grade models, but to create a proving ground for my harness, and see if and how rules impact performance.

I’ll be sharing more results from the “Greenwald Eval” here in this brand-new blog in the coming weeks, but I can say that the results track the big picture of my expectations (smarter models have done better) with surprises and lessons in the details (effort sometimes results in worse performance; Opus 4.8 and 5 fail different evals; Fable at low effort is just as fast and efficient as Sonnet low effort and beats both Opus versions; the list goes on).

I have tested a range of models—notably, most of the Claude range, Meta’s Muse Spark 1.2, both DeepSeek V4 Flash versions, and Google’s Gemma 4 31B and 26B. Until last week, only the mighty Fable was able to pass every eval. Incredibly, it did so at low effort, with only one case requiring medium.

Ox Alpha passed ‘em all, too.

There were some key differences. Fable at low effort used an average of 3,900 tokens to pass all but one eval; Ox Alpha at medium effort was close on tokens but failed 6 of the tests. At high effort, it passed everything, needing a 16,000 token average to do so.

That kind of usage makes it less efficient than Opus 4.8 at xhigh, but less hungry than DeepSeek V4 Flash 0731 or Muse Spark 1.2. That may help us make some guesses about model size. (Unfortunately, I haven’t run ChatGPT through any of these yet, so that’s a missing comparison.)

More thinking time for more intelligence isn’t a bad thing, depending what this model ends up costing. Which right now is nothing.

Strengths and weaknesses

What is Ox Alpha good at?

  • Code: It passed all my basic code-writing tests with thinking off, including a tricky CSS test that broke many other models (although lower efforts struggled, with high passing it again)
  • Security: It found 8 bugs in my “plan mode” pi extension and closed them up. Given the same prompt, Fable downgraded to Opus 4.8, which found one bug left and called the project “heavily hardened.”
  • Adversarial review: In my eval, it found the planted bugs but didn’t over-reach on the clean case, which other models such as DeepSeek V4 Flash 0731 failed at.

Where does it struggle?

  • Skills made it worse—much like 5th generation Claude models, which prefer fewer instructions.
  • Fabrication—at lower effort levels, it struggled on extensive web research tests, a challenge for most models but an especially weak point here (solved with effort level, however).

Overall: it’s impressive, and I’m doing as much work with it as humanly possible before the metering starts.

You can try Ox Alpha on OpenRouter or OpenCode.