It’s the End of the Web as We Know It (And I Feel Fine)
A flurry of recent news portends a blizzard. The link-based model which has driven the web for the past few decades may be on the brink of collapse. At the very least, it's seemingly about to be drastically altered in a way that's going to fundamentally change the fabric of the internet. That perhaps sounds too grandiose, but I'm really not sure that it is. Still, it's probably too big to write about in a single column, so for now I'll just mainly stick to the business of publishing content on the web. Oh, and web search.
I speak, of course, mainly about AI. And specifically how the rise of LLM-based queries is already upending traditional web search. But there's also the ongoing situation with social media, where the continued fragmentation of audiences has hastened the decimation of link sharing that was already occurring. This has given rise to a new hope in the form of the "Fediverse" – a concept which at a high level seems to hearken back to the early days of the web. But there's also perhaps a silver-lining – and potentially much more than that – in the impending disruption of web search.
The Search for the New Search
To that end, while Google keeps saying they're not seeing any signs of web search slowing, might I suggest that you follow what they're doing not what they're saying. Google is a company which is famous for being overly cautious with big changes to their services – and rightfully so in many ways, because they serve billions of users of many of those services, daily. And yet the speed at which they're merging AI into the crown jewel of their business, web search, is pretty incredible. In just a few months, we've gone from "AI Overviews" being tested to being fairly ubiquitous. And now we have "AI Mode" rolling out as the next public "experiment". It's hard to see it as anything other than an answer to the rise of the LLM-based products. And in particular, ChatGPT.
To be clear, this is exactly what Google should be doing. It's just honestly impressive that they're actually doing it. We're witnessing them disrupt themselves in real time. At arguably the peak of their power, at least when it comes to making money. They'll never admit to any of this, of course. Again, Search is fine! And it is fine, for now. But it's also likely to be one of those Ernest Hemingway bankruptcy situations. Web search is going to collapse gradually then suddenly. Call our current time a state of anagnorisis. Call it whatever you want. It's happening.
Just as it has happened with cable television. But in that case, the powers that be refused to admit it until it was obvious years later. And now Netflix is bigger than all of them. Their market cap is greater than that of Comcast and Charter and Disney – arguably the three most important players in the old cable system – combined.
Google, of course, is a far larger company and far better business than any of those ever were, but that just makes the stakes that much greater. If someone were to actually disrupt web search, we're talking about the rise of a new multi-trillion dollar company. So yeah, Google would prefer that they be the ones doing their own disrupting here. And they have a real shot at it thanks to some combination of those aforementioned billions of eyeballs, a trusted brand, technical prowess, and ingrained habits.
And yet, what I increasingly see and hear when out and about is people talking about ChatGPT. In particular, young people. And just as with the cable situation, that's the real concern for any incumbent. It's less about losing the users you have, far more worrying is the users who are growing up not using your wares. The "cord-nevers" as it were. Might we be entering a world of "search-nevers"?
That sounds silly because for anyone reading this, web search is absolutely vital to your daily life. But that doesn't mean it's always going to be the case. In fact, at some point, it will not be the case. Everything gets disrupted eventually, it's just a matter of how and when. And again, I think many of us can now see that in the form of AI. And I think that includes Google.
I'm not saying this will happen overnight. Again, it's much more likely to be a gradually-then-suddenly-type thing. The generation of kids who will be brought up using AI – the "AI natives" – growing up without being as reliant on web search as you or I. So I repeat: just look to Google's own actions here. They're rolling out these massive changes now. At speed and at scale. That speaks query volumes.
The Collapse of the Click
And so that leads to the second-order effects of such a shift. And we're already starting to see that play out. Some data pointing to a near total collapse of traffic from these new AI tools doing web queries on a user's behalf. Yes, Google, ChatGPT, and others are trying to add in links back to reference sites as prominently as possible, but they can't force a user to click. If they're getting what they need from the AI, why would they?
You might think that they should because that's the way the web has largely worked to date. But there's nothing mandating that it's the way it has to work. It's more of a implied contract from the web search days. But again, those days are ending. Or at the very least morphing. Web search was built on the back of links. AI search is built on the back of information.
And that notion points to the obvious way forward here. It will also sound crazy to some because it's decidedly not how the world has worked to date, but it's also the most obvious way forward: the AI services are going to have to pay for such information.
Many companies and even countries have been trying to get Google (and Meta, and others) to do this forever in the web search days. But it never made much sense in a link-based (and related: advertising-based) economy. Google surfaces your site or content in their results and your payment is the click. As results got shoved further down the page with ads and Google's own content getting more prominent placement, the dynamic was obviously shifting. And once Google started injecting answers rather than just the "10 blue links" it was a clear sign of what has now come to pass with AI.
And obviously most of the AI players are now paying for such content. While initially they may have scraped much of it for free using the web, if these tools truly are to replace web search, they're going to need access to continually fresh information. And so all of the major publishers are now striking content deals with these services, including, of course, with Google itself for Gemini.
Many old school web folks view this as the ultimate Faustian bargain, but it doesn't have to be – it could be the model that actually works in the age of AI. Because the flip side of this is that without all the traffic flowing in from Google Search (and social, which I'll get to in a minute), the publishing model on the web itself collapses. Because it, like Google, has been dominated by ads. Fewer eyeballs = fewer dollars = eventual collapse. Without a new model, like the one above, it's game over.
A New Hybrid Monetization Model
Of course, the publishers cannot rely solely on such payments to survive – hopefully they're no longer that dumb. And they're not, which is why basically all sites large and small are now moving to paid subscriptions. This has also worried many – in particular those "open web" advocates – that information is going behind paywalls and creating sort of "have" and "have-nots" dual internets. But actually, I think it will play out in a far more nuanced manner (shocking, I know). In this new world, a lot of "free" content will be surfaced via the AI channels (which again, are paying for it directly from the publishers) while many people also have their own set of publications that they pay directly for.
The problem here is the same with streaming. Just as you're ultimately not going to pay for half dozen different streaming services, you're also not going to pay to subscribe to a dozen – or two dozen – different publications. I mean, I'd be surprised if people ultimately pay to subscribe to more than a few. It's perhaps not great news, but it is what it is. It's reality.
And that points to a world where the massive publishers do well, and perhaps even thrive, and on the other end of the spectrum, many smaller publishers – maybe even one-person shops, like this one! – do well because costs are so much lower. But the middle is squeezed, badly. It sucks, but it's also the market at work. If the model doesn't work for you, you have to make it work – or you don't and it won't.1
I don't mean to be insensitive here – it's just the new reality we're likely to see. And, in fact, have been seeing. Seemingly not a day goes by without news of layoffs at various publications. Disruption disrupts, and that includes real people with real jobs and real lives, unfortunately.
But again, there's hope in the form of a new model rising – as it must and will – that is potentially more sustainable than the one we're going to exit. Whereas the advertising-based model worked well in the delivery mechanism of yesteryear, print, it never fully fit on the web outside of search and the social feeds.2
People got all excited because of the power of the click, but at the same time, the infinite canvas of the web just led advertisers and publishers to get high on their own supply. The web – at least with regard to publishing – has increasingly looked like a nightmare of pop-ups, pop-overs, and just generally garish ads that completely – sometimes literally – overshadow the actual content. It has sucked for everyone. For the readers for the writers and even for the advertisers.
All of this isn't to say that advertising on the web is dead, but it too will have to morph. I suspect on the publishing side, we'll see it come back in more tasteful and respectful ways because it's augmented by these other models: content licensing and paid subscriptions. The real question is what advertising looks like in a future where traditional web search is vastly diminished? As long as user intent is still there in queries, all sides will find a way to leverage that, obviously. But that's an entirely different post.
The Remaining Questions
That matters because for the above to work – and certainly the content licensing element, the AI providers are going to have to get paid at scale to be able to pay at scale. Subscriptions to AI services will be a big part of that, and perhaps even a massive part, as OpenAI is proving. But again, for true scale, they undoubtedly need to augment their own business models with advertising. And I have to believe that they'll have the scale and intent to make it happen and work.
In terms of what the content licensing agreements look like in this future, that's above my current pay grade. I mean that literally – no one is currently paying me anything for this content, and it's hard for me to think about what might even make sense. It will obviously be derived from what the large players get, but all of those are individually bargained right now, it seems (with some collectives popping up to get better leverage). This will undoubtedly be a problem that technology itself solves and there are a number of startups already working on this type of solution. A market solution is needed and will come to scale this model.
At first, it undoubtedly will be a crude, blunt model. But eventually, one can imagine a system which ultimately pays variably based on the information surfaced. (And perhaps even based on data training – which will undoubtedly make some writers want to vomit and should clearly be opt-in, but could still be some sort of model!) And while you'd worry there will be a race to control the new "SEO" ("AEO" – AI Engine Optimization?) in this world, authority will likely play an even greater role than it has in web search. Again, links originally underpinned that as a clever way to figure out authority at scale (which was then always under assault by those seeking to gain and literally profit from tricking such systems), AI will need a new solution. Users can provide some of it, but I'm guessing AI itself will be even better at determining authentic authority! Can we trust it to do that? Some will be terrified by this notion, especially about concerns of varying degrees of bias. Then again, this could also be far more personalized, one imagines...
But now we're getting too far in the weeds here...
Backing up, in such a world, in-depth content should actually be rewarded more than click-bait stuff – perhaps the one absolute upside of a world of fewer clicks! And experts writing about niche topics may benefit the most in terms of payments from higher-cost models and/or AI tools like deep research.
How sustainable would such payments be? It's impossible to say in my theoretical model, but you could imagine that money will ebb and flow for certain types of sites while perhaps being more steady for news publishers (you could easily see something like "minimum guarantees" to ensure the content keeps flowing ahead of the actual AI usage). Again, in the world I'm envisioning, licensing fees, subscription revenue, and some level of scaled-back advertising/sponsorship mix and match to make it work.
Of course for advertising to work at all, you still need eyeballs. And to get those, you need clicks. If I just spent the past 2,500 words describing a post-click world for search, I still believe there will be new mechanisms (and maybe some old ones!) that pop up to help with discovery and distribution on the web beyond simply our newfangled AI tools. And I go back to the notion of the Fediverse there.
I'm still not 100% sold that it will work, namely because there have been many similar concepts in the past – remember OpenSocial? I do! – that have failed to take off despite people generally liking the ideas at a high level. There need to be killer products coming about at the right time.
Perhaps the timing is right this time with the aforementioned splintering of yesterday's social web. And I think it potentially slots in nicely with all of the above with regard to the disruption of search. And in particular it creates an avenue for some level of advertising to still work on the web (alongside however it will work with these various AI products). And that still matters for scale and accessibility – we just need to ensure the advertisers and publishers and services don't overdo it this time.3
Know Thy Enemy
For now, I feel as if we're standing at the edge of a new era. And a lot of seemingly unrelated stories in the news every day are actually all directly related to this new world we're entering. It's terrifying for basically all sides – from Google down to the publishers – because it will just look and work a lot different than it has to date. But that's ultimately probably a good thing.
I'm reminded of the music industry when piracy had crept in. Or the movie industry after DVD sales started to collapse. They eventually found paths to leverage new technology to create better models and just as importantly, better products for consumers. After years of going after would-be saviors in the form of micro-payments and the like, there may actually be a new path forward here for publishing too. It may look like an enemy to many in that world right now. But it doesn't have to be.
M.G. Siegler
M.G. Siegler
M.G. Siegler
1 This will give rise to... what else? Bundles. Publishers will bundle together their content to make it easier and more enticing for customers to pay.
2 Where it has fit insanely well!
3 Until they inevitably do and the whole system is blown up again...