Artificial Iconoclasts
“Here’s to the crazy ones…” so kicks off Apple’s iconic "Think Different" 1997 advertising campaign.1 It has been in my head for the past few weeks, ever since reading the thoughts from Hugging Face co-founder Thomas Wolf fretting that AI may be driving us towards “a country of yes-men on servers.” Those thoughts were in response to Dario Amodei's essay “Machines of Loving Grace” and in particular, his assertion that AI will deliver us a “country of Einsteins sitting in a data center”. That is, the notion that AI will allow the breakthroughs of the 21st century to be achieved in just 5 to 10 years. Wolf is essentially saying that we’ll get the opposite of what what Amodei is saying we’ll get.
There’s a lot in there to unpack, and I’d highly suggest reading both essays. But at a high level, Wolf’s comments are the ones that have stuck in my head over these past few weeks, reading about new AI breakthroughs on a daily basis. The technology we’re building and dealing with is clearly incredible. At the same time, I also worry it runs some real risks that Wolf speaks to...
Because of the way that LLMs have been trained — with more or less all of the data on the internet — in a way, they’re almost the ultimate example of “group think”. It’s such an extreme version of group think that it probably negates a lot of you’d be concerned about with such a connotation. But also, no one knows for sure. Are the data inputs influencing the outputs just as a human brain would? Probably not, but it’s also probably not as random as what our brains seemingly do. (And that itself is probably not as random as we’d like to think as beautiful, individual snowflakes, but I digress…)
The point — really Wolf’s point — is that AI is perhaps not going to lead to major breakthroughs in the same way that human beings have done so historically. And that’s because a lot of those breakthroughs come from people who are considered to be crazy, or at least having a crazy spell, because they’re thinking so far outside the box that they seem, well, insane. In hindsight, their genius is clear, but in real time, they’re often not taken seriously — or worse, are deemed heretics! Something compels them to persist, but it’s not any natural incentive structure.
Said another way: it’s unlikely to be anything an LLM would understand and thus, use to influence its own output.
Again, this is the point of “Here’s to the crazy ones”. It’s celebrating the outside-the-box thinkers who have changed the world. But it’s not clear that AI is set up to do the same just by the nature of how it has been built and trained to date. Maybe we figure out a way to change that — do we start to weigh “contrarian” opinions more heavily to counter prevailing “wisdom”? Do the systems themselves start doing that because of some unforeseen incentive? Wolf lays out some other potential options and I like the way he frames the problem:
The main mistake people usually make is thinking Newton or Einstein were just scaled-up good students, that a genius comes to life when you linearly extrapolate a top-10% student.
This perspective misses the most crucial aspect of science: the skill to ask the right questions and to challenge even what one has learned. A real science breakthrough is Copernicus proposing, against all the knowledge of his days -in ML terms we would say “despite all his training dataset”-, that the earth may orbit the sun rather than the other way around.
To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask. One that writes 'What if everyone is wrong about this?' when all textbooks, experts, and common knowledge suggest otherwise.
Since the early days of ChatGPT, we’ve heard non-stop about “hallucinations” in these systems. That has become synonymous with spitting out nonsense or unwanted tangents — oh hi there, "Sydney"... But what if that was a feature, not a bug of such systems? What if we need such hallucinatory capabilities in these systems in order to have them actually, well, think?
This is all above my pay grade, but I’m just throwing out thoughts — thoughts that these AI systems will likely one day ingest. But actually, I’m pay-walling this one, so perhaps not. So it will be up to you, dear humans, to tell our future AI overlords about such heretical ideas.
Back to the topic at hand, none of this means AI is worthless, of course. In fact, it might be the most worthwhile technology ever created even if the only thing it does is help unlock new thoughts in the human beings thinking them. We all know the concept of getting “stuck” on a problem. Writing has perhaps the most famous version of this in the form of “writer’s block”. What breaks such barriers is often as simple as taking a step back and doing something else and then revisiting the issue.2 Or talking to someone else about whatever problem you’re trying to conquer. “Fresh eyes” and all that. If AI can act as a sort of infinitely scalable set of “fresh eyes”, I suspect that it’s impossible to overstate the value there.
All of this is another way of saying that AI’s actual ultimate super power may be a way to get human beings to look at problems in a new way. To “think different” as it were.3
One more thing: going back to the concept of “yes men” I do think there is an obvious issue already forming around chatbot responses to queries. As people ramp up asking these services their opinions on something, they’re almost always positive on such matters. Actually, strike that, I think they may always be positive. That’s undoubtedly on purpose, to give positive interactions and thus, leaving users with “good vibes” after talking to the bots. But it’s, um, really fucking weird.
These bots really are the ultimate “yes men”! I would love to ask a bot about an idea I have and it to tell me that it’s stupid. I truly would. I recognize not everyone values such feedback, but the best people do (I don’t make the rules). I worry that the AI simply can’t do that — at least not yet — because it requires something tangential to knowledge, but very different: an actual opinion.
Sure, AI can the trained to feign opinions. But until it can actually form them, we’re obviously not at AGI yet. But then it also would require the capability to change those opinions over time with new inputs. All of this is undoubtedly technically possible, but it’s nuanced. You have to be a little bit crazy to give thoughts on something without knowing literally all the facts. AI can literally know all the facts, but does that stop it from forming an opinion?
1 Undoubtedly their second most famous ad after the original "1984" Macintosh spot.
2 My own personal "hack" here is to listen to music. Something about music seems to almost reset my brain. Is it um, crazy to think that we should have AI systems listening to music?
3 Naturally, I have to use this opportunity to link to the version narrated by Steve Jobs as well -- which is almost hauntingly poetic now, as he was, of course, one of those crazy ones.