What if Anthropic is Right?
If you live online, you'll have noticed the vibe has shifted pretty substantially against Anthropic in recent weeks. Whereas even just a few months ago, the company was seen as the more scrappy, idealistic underdog – one which happened to be formed by founders who left the seemingly unstoppable foe in OpenAI – that tide has now turned.
Certainly part of it is just the shift from underdog to presumed leader in AI. But a lot of it has to do with the way Anthropic carries itself as a company. Whereas the strong stances and rhetoric were once seen as almost endearing, now they're seen more as a problem. From the US Government to Big Tech and many in between.
The mood now feels so negative against Anthropic that it seems worth asking the obvious question that no one wants to bother with any longer: what if they're right?
That is to say, many now seem certain that most, if not all, of Anthropic's stances on AI are purely self-serving. When there are statements that aren't necessarily against, but certainly not full-throated endorsements of "open" models, many hear a company protecting their moat. And when there's a request for a controlled slowdown of frontier AI development – which is hardly just Anthropic employees, but is signed by their CEO – many assume that's a pulling up of the drawbridge.
Look, I've been cynical of Anthropic's potential motivations in the past – before it was cool. But I also see all of the vitriol about the above and don't find that exactly fair either. Does Anthropic do things in their own self-interest? Sure. All companies naturally do. But it also seems to me that this is a company which remains more ideologically driven than most. Which is to say, they not only think they're doing the above for the right reasons, but they also largely actually believe in what they're saying.
And that just pisses people off even more. But that's just because they think Anthropic is wrong in many, if not most, of their stances. Again, closed models. Decelerating development. Putting up more guardrails. Holding back some models. It's driving some opponents truly insane.
But I ask again, what if Anthropic is right to do or at least to push for such things?
I'm not saying I agree with that notion. Frankly, a lot of their rhetoric rubs me the wrong way too. But I'll also acknowledge that a lot of it is also likely above my full understanding. But that's also a problem because I'm not sure it's not above anyone's understanding. Because I'm not sure anyone truly understands where this is all heading...
And with that in mind, doesn't it at least make sense to acknowledge that Anthropic could be right? Or at the very least to think through that possibility on the off chance that they are? To borrow the now somehow bro-y phrase: to "steelman" the argument.
What if we're not on the cusp of AGI, but rather RSI – recursive self-improvement – and what if that's the actual key unlock to get us to AGI? What if we almost accidentally trip into RSI and advancements go not just exponential but hyperbolic? What if this, in turn, pushes us far beyond a point of no return? The genie out of the proverbial bottle, never to be put back?
That is, an AI that can no longer be turned off. Not necessarily Skynet from Terminator, but also not completely innocuous? Mainly because it starts doing things that we don't understand. Because we don't understand a lot of what AI is doing even right now in what may end up seeming like the relatively rudimentary LLM days of the technology!
This sounds almost silly. Just break the glass in case of emergency, right? Turn off the damn data centers? But what if that requires, say, turning off the entire internet? Can we even do that without crippling much of the world's actual infrastructure?
While the recent security incidents disclosed by OpenAI and Anthropic seem more related to human error or carelessness, there also certainly seems to be some level of the raptors testing the fences, to borrow another movie analogy. "They were testing the fences for weaknesses, systematically. They remember."
Which also sounds insane! I know! Especially after we've all been subject to Chicken Littles in AI for years, from Blake Lemoine to "Sydney". AI was always just on the verge of waking up. Or it already was awake. But what if those Cassandras were less wrong than just early?
Maybe the government stepping in to try to exert some control over these models will help in that it will naturally slow things down? They're clearly less worried about the RSI fears versus simply the security ones, but what if it's all sort of the same thing? Again, an almost casualness in which we cross into catastrophe. Just as Mythos seemed less about pure capabilities rather than near-infinite capabilities, what if that's also the path to RSI? And that, in turn, makes AGI inevitable through pure brute force?
What if it's true that LLMs aren't the path to AGI, but RSI allows LLMs to create "World Models" or other types of AI that are the key to unlocking AGI?
In that case, the fail safe may not be the US government – or even if we convince the Chinese government to go along with some sort of safety plan, but rather the build-out bottlenecks for AI. But there too, what if RSI unlocks the ability for AI to almost infinitely optimize models to run on current infrastructure?
All of this sounds crazy – and also increasingly not entirely crazy.
And all I'm saying is that it's certainly worth thinking through. What if Anthropic isn't fear-mongering out of an attempt to protect their moat? Or being burdensome in their stances simply because of ideology? What if they not only fully believe in this future, but what if they're right about it?
M.G. Siegler
M.G. Siegler
M.G. Siegler
Member discussion