The Casual Catastrophe of AI
It's a question of commitment. And incentives. And scale.
To me, that's how I'd boil down the current state of AI relative to humans. It's extremely oversimplified, but I'm not sure it's wrong.
I started thinking about these notions when writing about the value of writing in the age of AI. This naturally led to thinking through the value of thinking in the age of AI. But what really drove home the concept was reading all the coverage around Anthropic's latest model, Mythos. You know, the one too dangerous to be released.
You can't help but read all of these stories about all the bugs, vulnerabilities, and exploits that Anthropic's model is finding across basically all computing systems out there in the real world and think "holy shit, we're cooked." While 'Project Glasswing' seems like a valiant effort to get ahead of the issues, come on, we know how this movie ends...
But my main takeaway is that it has less to do with the genius of these AI models – I mean, that's part of it, and clearly Mythos seems to be the smartest yet – but it's more about the breadth. Both of knowledge and time.
Said another way, reading all these security experts and researchers talk about Mythos, it's pretty clear that the model isn't so much finding issues that human beings cannot, but it's finding issues that human beings have not, and most depressingly, finding things that human beings will not.
Why? Again, time and incentives.
If you tasked a capable human with finding every single bug in a certain system, they presumably could do it – if given enough time and resources. These issues don't require super human knowledge, in fact, they require human knowledge. But often times to find them all, it requires human knowledge scaled in super human ways. Again, spending more time on it than any human reasonably would. Because again, the incentives are simply not there for a human to spend their entire life looking for bugs. Perhaps if the vulnerability was great enough, sure. But that's sort of an unknown until such things are found. No one creates systems to have obvious vulnerabilities for others to fix. They're the byproduct of a million little variables – a scale a human isn't suited to deal with.
But AI is. Issues that might take a human years to find and fix can be found and solved almost instantly by such systems. We know this to be true because Mythos is finding issues in systems that are a couple decades old! Despite some level of usage the entire time, humans simply never found the issues.
Luckily, it seems, never did attackers. That's the thing, the flip side is the real problem here. Historically, many vulnerabilities have been fixed only after someone exploited them in some way. Again, that's because the incentives are in favor of the attacker versus the defender. If and when Mythos-caliber tools are put in the hands of hackers... yeah.
That's obviously exactly why Anthropic isn't releasing Mythos to the public and also why they've set up Glasswing. While the company may be first to such capabilities, they won't be the last. They probably don't even have long to try to get ahead of the situation. While I generally dislike the nuclear weapons analogy for AI, I must admit, this all does feel a bit Manhattan Project-y. The good guys are racing against the clock to implement a new technology before the bad guys catch up. But they will. They always do.
And sadly, there's no real hope of deterrence here as with nukes. Again, incentives. Is the Glasswing gang going to unleash Mythos to take out the would-be hackers? I mean, maybe they could for a big enough evil organization. But most such bad actors will either be lone wolves or operate in tiny teams. Even if you could preemptively attack, you simply won't be able to know where to focus such concerns at all times. I mean, maybe AI would? Maybe? But that's probably overly optimistic.1
Anyway, point is that Mythos is clearly great at finding exploits and while the powers-that-be are trying to use it fast to fix such issues, the bad guys will eventually get their hands on it as well. So it will be a cat-and-mouse game both in tracking down those would-be bad guys, but more importantly, tracking down the vulnerabilities and hoping the good guys can stay one-step-ahead technologically.
The Scale of the Problem
But I go back to the notion of scale. Given the issues Mythos has already found – across every operating system and seemingly every piece of software they've looked into – it's hard to feel anything other than overwhelmed here. And again, that to me is sort of the story of AI right now. It's less about "superintelligence", and more about intelligence scaled in a way that humanity cannot.
There are incredible potential upsides to this idea, such as in drug discovery and disease eradication. Again, these systems can run basically infinite scenarios – possibilities so large that a human simply cannot even fathom, let alone execute. The only limiting factor is resources – as in compute, not time. Incentives are no longer needed to lead down one path because AI can go down every path (though incentives remain on the human side of the equation, tasking such systems, of course).
This will apply to other scientific discoveries, obviously. In space, in the deep sea, etc. Humans may technically have the capabilities, but not the time.
This same general idea is what is taking coding out of our hands. And that too is being applied to other "white collar" areas of work. Reviewing legal documents is tedious and time consuming. But not for AI. Etc.
Creative endeavors feel more protected. And that's because while AI technically could write the works of Shakespeare – again, time is not an issue, endless possibilities are literal – the system wouldn't necessarily know when it had. It would only know which version to pick if compared against the existing works of Shakespeare. But what about future Shakespeares?
Creativity comes from constraints, not the lack thereof.
This is taste. Which has sadly become a buzzword amongst tech bros. But it does matter in the future of our interaction with AI. It's a part of what's going to raise the relative value of human-made work. But the bigger part is the other constraint, the larger one: time. People are going to learn that they're not paying for output, they're paying for input. How much time was spent on something – the most precious resource that a human being has. The variable that doesn't limit AI.
A Quantum Leap
To bring it back to the moment at hand, reading about Mythos paints a clear picture of a future in which problems are both solved and created by the human-centric notions of time and incentives being thrown out the window with AI.
And it seemingly points directly to the next big technological quandary if and when the comparatively unlimited resources of quantum computers can both make new discoveries by doing computation at a scale that's impossible right now while at the same time likely cracking traditional cryptography. It's the same general high-level notion. And it's likely to define the next decades of both computing and the world.
In a way, it's the same idea that has defined computers from the get go. But at a scale that can now both break and fix the real world. Perhaps in real time. With an almost casualness that's impossible for the human mind to comprehend. It's both absolutely exhilarating and completely terrifying.
M.G. Siegler
M.G. Siegler
M.G. Siegler
M.G. Siegler
M.G. Siegler
1 And let's not even delve into the Minority Report element of "pre-crime" here – attacking a target before a crime has been committed. What are the lines there? ↩