If AI Writes All the Code Then Why Did Anthropic Invest in Goodfire

Last week, it was all about OpenAI. The company’s new code interpreter, Windsor, had developers uploading tax returns, entire data pipelines, even medical workflows and watching ChatGPT write code like it was filling in a spreadsheet. Analysts dubbed it “the new Excel.” But barely a week later, the conversation has veered again.

Now it’s Anthropic’s turn. The Claude-maker just invested in a public benefit startup called Goodfire. And while Anthropic’s check was small, just $1 million, the implications are enormous. Goodfire isn’t offering a shinier chatbot or a faster copilot. It’s building Ember, a platform that lets developers reach inside a model and manually rewire its neurons.

Today, we're announcing our $50M Series A and sharing a preview of Ember – a universal neural programming platform that gives direct, programmable access to any AI model's internal thoughts. pic.twitter.com/asKBLKyZpA

— Goodfire (@GoodfireAI) April 17, 2025

Forget code. Forget prompts. Goodfire is going after the black box itself.

Neuron is the New API

Founded in 2024 by Tom McGrath, Eric Ho, and Daniel Balsam, Goodfire has raised a $50 million Series A led by Menlo Ventures, with participation from Lightspeed, B Capital, Wing, South Park Commons, and Work-Bench. The round valued the company at $250 million less than 12 months after it was founded.

But Ember, not the valuation, is what makes this startup different. The platform is model-agnostic and exposes the internal states of large neural networks, letting developers map specific neurons to behaviors like “lion” in an image classifier or “Santa hat” in a diffusion model. In language models, Goodfire even controversially refers to some patterns as “consciousness neurons.”

This isn’t just diagnostic tooling. Developers can edit those neurons, changing the model’s output by altering its internal logic. “We are not building compliance explainability tools,” said CEO Eric Ho. “We’re building instruments to redesign neural networks from the inside out.”

Interpretability, Not Interface

That approach flies in the face of today’s prevailing wisdom, where models are treated like untouchable endpoints. Input goes in, output comes out—and how it works internally is considered either a mystery or someone else’s problem. Goodfire is rejecting that passivity.

It’s also backed by serious technical credibility. McGrath previously led interpretability efforts at DeepMind. Ho built and scaled a prior startup to $10M ARR. Researcher Lee Sharkey is known for applying sparse autoencoders to decode the inner structures of language models—a technique closely aligned with Ember’s goals.

Goodfire’s tech is already in use. At the Arc Institute, researchers used Ember to identify previously unseen biological mechanisms in Evo 2, a DNA foundation model. “Their interpretability tools have been instrumental in unlocking deeper insights,” said Arc co-founder Patrick Hsu.

Anthropic’s First Startup Investment

This marks Anthropic’s first-ever startup investment, a quiet but deliberate departure from its no-ecosystem stance. Until now, the company had stayed laser-focused on its own Claude family of models, backed by more than $8 billion in funding from Amazon, Google, and others. It avoided the kind of venture building OpenAI is now famous for.

But the $1 million check to Goodfire came via the Anthology Fund, a $100 million initiative co-launched by Anthropic and Menlo Ventures. The fund targets startups building either atop Claude or attacking fundamental AI challenges. Goodfire is the first startup to “graduate” from Anthology’s early-stage bets into a full Series A.

CEO Dario Amodei didn’t frame the move in commercial terms. Instead, he called mechanistic interpretability “a critical foundation for the responsible development of powerful AI.” That’s not just rhetoric. It signals a shift in Anthropic’s strategic priorities.  “Our investment in Goodfire reflects our belief that mechanistic interpretability is among the best bets to help us transform black-box neural networks into understandable, steerable systems, a critical foundation for the responsible development of powerful AI.” he said. 

The Irony? Code Is Dying, and Anthropic Knows It

And there’s a deeper irony. This is the same Dario Amodei who recently predicted that 90% of code will be written by AI. Windsor seemed to validate that vision—code as a fading craft, replaced by natural language. But while the industry celebrates the death of coding, Anthropic is doubling down not on writing code, but on understanding what happens after the code has been written. On making AI models fixable, not just usable.

As Deedy Das of Menlo Ventures put it, “AI models are notoriously nondeterministic black boxes.” In Ho’s words: “Nobody understands the mechanisms by which AI models fail. So no one knows how to fix them.”

That’s the deeper bet Anthropic is making. That the next phase of AI won’t be won through scale, or interface, or prompts. It’ll be won through control at the lowest possible level. And that means mechanistic interpretability may become not just a research obsession, but a commercial necessity.

The stakes are enormous. Goodfire’s pitch to enterprises is simple: if your model gives bad advice or makes a biased decision, dashboards won’t save you. Intervention will. And that means giving developers not just insights into what went wrong—but tools to fix it before it happens again.Last week, the buzz was all about coding’s disappearance. This week, Anthropic is saying: maybe it’s not about code at all. Maybe the future of AI isn’t about how we write software. Or, as Eric Ho put it:
“Nobody understands the mechanisms by which AI models fail, so no one knows how to fix them.”