AI's Mixed Messages • Siri AI's Potential Developer Problem

Do you feel like every other headline and talking point coming out of the companies at the AI frontier contradict each other? You're not alone there. The future is so bright... because of the nuclear blast that the rogue AI agents will trigger. By accident or on purpose. But not to worry! It will be great, as long as we all agree to get ahead of the problem. But if not, we have a great opportunity to fix the problem with the fantastic new technology we're creating.

Meanwhile, Siri AI nears. You know, the version that actually works. For real this time. Well, unless developers decide not to work with it, which very well could happen given Apple's many contentious relationships. That will make creating the mythical "personal assistant" tech awfully hard...

Read on for thoughts on...

Please Get Excited for the End of the World

Another day, another dichotomy. The various teams building our AI future clearly know at this point that they have a major narrative challenge. How do I know that? Because they keep talking about it. To be fair, they're also asked about it in every interview, so it's sort of a vicious cycle: it's the main talking point because it remains the main talking point. Of course, all narratives die in the hands of saturation eventually. People will get bored and move on – I'm not saying that's right or wrong, that's just the way it works. Unless... you don't allow them to get bored.

Like, say, if you state that a report suggesting that chain-of-thought monitoring is at risk is "confused", then go out and write a 3,000-word blog post discussing your own fears that chain-of-thought monitoring is at risk. Here's OpenAI Chief Scientist Jakub Pachocki:

We understood the potential significance of chain-of-thought monitoring at the same time we developed reasoning models. When we shipped o1‑preview, we deliberately designed the product to hide the chain of thought⁠, to protect it from supervision pressure in the long term. In development since, we have strived to maintain the rule of not supervising the reasoning process. CoT monitoring became an extremely important tool for us in studying how our models generalize from their training distribution, allowing us to observe and analyze not only their actions but also their internal process.

This tool continues to be critical as we study the Astra class of models. However, unfortunately our evaluations indicate our ability to rely on CoT monitoring is progressively diminishing.

I mean, kudos for the honesty and transparency, but it was also necessary after the original misdirection. But the bigger issue remains the overall narrative, which this post further fuels with Pachocki's proposed solution to the "new kinds of danger" made possible by this more powerful models and agentic use cases:

At the same time, even with the uncertainty that comes from anticipated broad AI progress and the need to build defensive systems, we must not let that become an excuse for recklessness. The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes.

In other words, his view – and the view shared by many building out the AI frontier – is that we need to slow down AI development. There is simply no way to reconcile this notion with the fact that the work continues. While the justification ends up being some flavor of "if we don't build this, someone else – someone less responsible – will" what people actually taste is more in line with the famous Gavin Belson quote, "I don't want to live in a world where someone else makes the world a better place better than we do."

But it's actually far worse than that because again, they keep talking up the nightmare scenarios here. It's more like "I don't want to live in a world where someone else has the opportunity to end the world before we can showcase that we can."

And increasingly, these messages clash head-on with the counter-narratives that the more rational – not to be confused with the rationalists, from which many of these narrative issues may stem! – leaders in the space are trying to propogate. (Aside from Bill Gates!) Anyone seeing all these headlines will have whiplash. And so is it any wonder that people increasingly don't trust these companies or this technology? How could you when the companies are constantly contradicting themselves?

Again, they'd frame this as honesty and transparency, which sounds great on paper, but it really just looks like they're not on the same page internally, at best, and at odds, at worst. And various reports going on a few years back up such consistent tensions.

So what's the solution? It's obviously not simple – it might be the most complicated situation in the history of technology. But clearly step one is getting everyone internally on the same page. That doesn't mean cracking down on dissent, but it means finding an actually consistent narrative for what the game plan is going forward. And sticking to that externally. It cannot be: "this technology is awesome and the future is going to be great as a result... unless the technology ends the world first."

I mean, no shit! But this is the state of the narrative!

Apple Built Siri AI, Will Developers Come?

With the roll-out of Apple's new OSes imminent, Siri AI – aka "New Siri" aka "Siri That Works" – seems ready to roll. I've been beta testing the new version for weeks at this point and the results have remained solid. Honestly, the biggest problem I have is remembering that I can now use Siri because she actually works – after years of that not being the case, old habits and all that...

But this new Siri comes with other risks, and one in particular that could be the key to getting people to adopt it as a daily habit. As Kalley Huang and Brian X. Chen note:

Apple’s success with Siri AI hinges in large part on third-party developers, who have a lot at stake. If developers allow their apps to work with Siri AI, the apps could become more useful on Apple’s devices by making their content easier to find. But big brands such as Amazon, Google and Meta could also risk sidelining their own assistants, or making it more difficult to add A.I. to their apps in the future.

So far, whether developers will plug into Siri AI remains an open question. As of last week, some staple apps, including Gmail and WhatsApp, were not working with an early version of Siri AI that is available to app developers, and it’s unclear whether they will.

This new Siri is finally able to pull context and content out of your messages and email – but so far, only if you use Apple's Messages and Mail products. If you use, say, the Gmail app or WhatsApp, you're currently out of luck. Obviously, Apple hopes the developers of those apps will use their new OS hooks to make such content readily available to be served up by Siri but, well, those developers in this case are Google and Meta, respectively. Both, of course, have competitive AI offerings that they'd prefer you use for such things.

Perhaps because Apple and Google partnered on the distillation of Gemini here for Siri AI, they also will have an agreement about such integration. But will the Pixel team, which has spent years trying to entice the much-coveted iPhone users to switch, be thrilled with that?

And Meta, well, they're unlikely to do Apple any favors here. Especially if it makes the iPhone more enticing to use versus, say, any AI hardware they may produce. Could they trade it in exchange for better access to the APIs that they want for the Ray-Ban Meta smart glasses? I mean, maybe? But probably not!

My guess is that some developers will play ball with the new Siri here, hoping it leads to more engagement/usage of their services. But some obviously will not. The question is what that split looks like. Clearly with the Vision Pro, Apple thought developers would be quick to jump on board. And well, they weren't. The iPhone is a different beast at a different scale, but there are obviously risks alongside those opportunities.

And how much do we want to bet that some developers will throw Apple's concerns about data security in the hands of a third party back at them?

Further, even amongst some smaller developers, Apple has done themselves no favors when it comes to goodwill. They continue to nickel and dime on the App Store even as countries and legal proceedings around the world chip away at their arcane rules. And that fleecing may be about to get worse?

So yeah, Apple better hope that Siri AI's agentic capabilities are so good that consumers demand integration lest they start to use other services that are integrated with the system. Otherwise, Siri, even in her fixed state, could be in for some growing pains. People may continue to not use Siri, not because she doesn't work, but because she doesn't work with their preferred apps and so why bother?