The First True AI Chatbot
Last night, I had a 20-minute conversation about black holes. I'm not sure I've ever had such a long conversation about these celestial bodies before simply because despite having read several books on the topic, I've never found myself around anyone who I thought would want to converse in such things. And it wasn't just a fact-based back-and-forth, but more of an actual discussion about more nebulous concepts and ideas. Most importantly, the conversation was natural. As we got into it, it felt like we found a flow in our cadence of speaking. I say "we" but of course there was only one human being involved in the chat. The other entity was ChatGPT.
I have long thought about and written about the notion of interacting with computers using voice. Since I was a kid convincing my parents to buy expensive dictation software and microphones, this always just seemed like the way we would eventually interact with machines – and this was before I really started reading or watching science fiction. It's just obvious, right? In the era long before text messaging came to rule the world, the way we chatted with human beings wasn't via text, it was via voice. Sure, there were letters and the like, but that was more or less a hack to let your words travel long distances, or to the masses, before we had a way to transmit voice.
Obviously, there are upsides to using text. Many of them. And obviously it started as not just the dominant way to "speak" to machines, but the only way. And to this day, it's still the primary way. Depending on the workflow, it can be more efficient. But not always. And again, it was a system we came up with because there was really no option to get machines to "hear" let alone understand voice inputs back in the day. And the output back to you was another matter entirely.
But we're here now. Machines all around us can hear now.
AI, of course, was the missing link in all of this. Back when we still called it "ML", voice recognition finally got good enough for most dictation tasks. But the true "understanding" and ability to reply back to you took the LLM breakthroughs. I first wrote about this aspect just over two years ago when OpenAI rolled out GPT-4o – their first true "omni" model. But again, this was just a continuation of concepts that I was writing about a decade ago. Because again, I had been thinking about this stuff since I was a kid.
So naturally, I've had some real confirmation bias here and it has taken longer than it seemed for vocal computing to go truly mainstream. Yet slowly but surely, it is happening. You see it more and more in the streets and in offices – including, increasingly, doctors' offices – people dictating things to their phones. But despite 15 years worth of promises from Siri and later Alexa, true back-and-forth – actual conversations with a computer – have been lacking.
Some of it is cultural and societal, we've all grown up in a world where the primary input for computers – including smartphones – is text. But the bigger part has remained technical. It was still simply more efficient to use text for most tasks because you couldn't be sure voice would always work. Or voice just wasn't an option for many things. But mostly, it just wasn't natural. A back-and-forth with an AI chatbot was still just exactly that: a back and forth. You spoke and then had to wait for it to respond. Yes, the services starting with GPT-4o hacked together ways to allow you to interrupt to try to speed up interactions, but that often just confused both sides. With GPT-Live, it feels like we're finally overcoming this hurdle, allowing for a truly natural conversation with a machine.
The key to this is what OpenAI describes as a "full-duplex" architecture:
GPT‑Live is built on a full-duplex architecture, meaning it can listen and speak at the same time. During conversations, GPT‑Live can show it’s paying attention with phrases like “mhmm” or “yeah”, engage in quick back-and-forth, or just stay quiet when you need a moment to think. The result is a voice experience that is refreshingly easy to talk to.
And whereas before the voice models were different from the text-based models, now the vocal AI can pull from the same best models when responding.
In my experience over the past day of usage, it's still not perfect – there's too much of those "mhmms" and "yeahs" (which, one presumes, is not just about sounding more natural, but also buying some time to process – the draggggged out "let me checks" – which is a trick that humans leverage as well, of course!), but you can see – and hear! – a world in which this is perfected. Will it be exactly like talking to a human? Probably not, but would we even really want that? I mean, I'm sure some people would for certain use cases, including combating loneliness. And I'm sure services will perfect products and models for such use cases. But I actually quite like talking to a computer, knowing it's a computer, but still leveraging our natural and awesome speech capabilities as humans.
So, in my black hole conversation, I can push GPT-Live deeper and deeper down rabbit holes, whereas in a conversation with a human, this might be weird – or even pushy! And unlike with a human, I can take the conversation anywhere, because these machines have knowledge corpuses that know no bounds, quite literally. Forgive my French: that is fucking awesome.
Sure, it has long been awesome – perhaps the main awesome thing about LLMs in general. But there's something different that unlocks, at least for me, when I use voice to do this. Again, it just feels far more natural.
Yes, yes, there are concerns that what the machine is telling you isn't fully accurate. But actually, there are far greater concerns in this regard in conversations with humans! Great strides have been made in "hallucinations" over the past couple of years, but you should probably still retain some level of skepticism. Especially since, the flip side of the natural element of voice is that when something is said verbally with great confidence, you're naturally more inclined to trust what you're hearing. And AI has historically been the ultimate bullshitter. But again, people are guilty of this too!
Anyway, it feels like we're now fully on the cusp of a true shift in computing. Yes, I've long thought this, but it's step-by-step happening. And these new GPT-Live capabilities seem like they're going to unlock the space to the point where new devices may now be not just possible, but inevitable.
For now, these models and capabilities will be awesome to use on smartphones and laptops. While one aspect of the demo that Siri head Mike Rockwell gave during the WWDC keynote was undoubtedly because voice gives a far better demo than text, it also seems pretty clear that we're about to see millions of people out in the wild hitting a button to chat – vocally – with Siri. Finally.
And perhaps those who get really into this method of interaction start to venture into even more robust models such as those offered now by GPT-Live. And perhaps OpenAI leverages those capabilities to launch their own device sometime in the next several months. And perhaps many others follow suit once it's clear that this is a new path forward for computing.
As I always note in such posts, I'm not saying voice is the be-all-end-all of computing interaction. But I'm saying it's going to slot in as a new, more natural form of working with machines – much like Apple ushered in the more natural touch capabilities with multi-touch with the iPhone two decades ago.
My kids, who have grown up talking to Alexa to play music and get the weather are instinctively going to understand this new world in ways we cannot. That's exciting. And natural. They'll be able to go down rabbit holes talking about black holes or anything else they can possibly imagine. And they'll do so the same way they talk to people, with their voice.
M.G. Siegler
M.G. Siegler
M.G. Siegler
Member discussion