Digital Skeletons in the AI Closet
Do you have some elements of your past that you’d prefer to forget? Of course you do, you’re human.1 And are some of those moments captured online in some way? Probably, if you’ve been using Facebook or Google for long enough. Are these worlds about to collide? It sure seems that way.
Geoffrey Fowler highlighted this issue in a column last week for The Washington Post. In testing the new, stand-alone Meta AI app, he writes:
But Meta AI also brings something else to chatbots: surveillance. The first time I opened the app, I asked it to “describe me in three emojis.” It could, by drawing on years of personal information tracked by its sister apps Facebook and Instagram.
As a test, I also chatted with Meta AI about some intentionally sensitive topics. Afterward, I found it built a so-called Memory file about me that said my interests include natural fertility techniques, divorce, child custody, a payday loan and the laws about tax evasion.
Yikes.
To be fair to Meta, this is still the very early days of AI, and they do allow you to delete things in your “memory” file that you perhaps wish not to be reminded of — or that you don’t want Meta knowing. To be less fair, it’s decidedly not the early days of Meta, and the company should know a lot better than to offer up something like this in such a reckless way. It’s almost as if the company constantly seeks out piles of privacy shit to step in, and then leaps at the opportunity.
If I take any issue with Fowler’s column, it’s that he sort of frames it as Mark Zuckerberg himself just doing creepy things with your data, as usual. But I think trying to paint the CEO of the company in such a nefarious way — and I give Zuckerberg plenty of grief for general disingenuousness and double speak — actually does a disservice to what is going on here. And what is likely to continue going on. Meta isn’t doing this because Zuckerberg is a Bond villain, they’re doing this because they think it will help them “win” the AI arms race.
Look no further than an interview Chief Product Officer Chris Cox gave to Alex Heath of The Verge about the Meta AI app. When asked about a potential edge they would have over ChatGPT, Cox notes:
The second piece is the personalization. It remembers everything I've told it. It also incorporates your Facebook and Instagram data, if you connect your accounts, to understand interests in a niche way. It knows I'm a marathoner. It knows I'm very interested in AI. It knows the humor I like. That starts to get you a pretty fascinating level of depth when it comes to answering basic questions. We've created the best personalized experiences in social media, and we know that's the difference between a vanilla app and something that people really love. So that's going to be a big differentiator, I expect, for us.
To quote Danny Moses and Porter Collins in The Big Short, “they’re not confessing, they’re bragging.”
Meta very much views the ability for their AI to dig into your past as a feature. And it’s easy to see why! OpenAI rolled out an upgraded memory feature a few weeks ago within ChatGPT and people immediately started having fun with it, asking the service to describe them. Before that, there was a viral moment around a less robust version of such memory, where people were asking ChatGPT to draw a picture of them based around what the AI knows from past interactions.
So Meta must have been watching this and thinking: “oh wow, just imagine the picture we can paint of these people, quite literally!” The difference is that OpenAI doesn’t have two decades of history on you, as Meta does in some cases. And again, Meta thinks this is a massive strength. And I’m telling them that it’s far more likely to be a massive problem.
Are there some things about myself from 2005 that it might be useful for an AI service to know? I guess? But it’s hard to imagine what those are. And certainly, that info is not better than knowing current information about me that might be useful to tailor a response. In fact, many of the responses derived from 2005 data will probably be out of date at best and currently wrong at worst. But again, actually that’s not the worst. The worst would be memories that I’d wish to forget. Yet there they will be, as just another data point for the AI to serve up.
You’d hope Meta’s AI will be smart about such things. But well, see Fowler’s examples. AI is great at a lot of things, but personal information nuance and empathy is not going to be one of them, at least not right now.
And this should all serve as a huge warning flag to Google as well — a company which probably has even more history and data about you on file thanks to Google Searches and yes, the cockroach of the internet and our lives: email. My god, if Gemini starts to serve up AI responses based upon emails I sent or received in 2004, I may just throw my computer into the ocean.
But beyond thinking such long, historical, and personal context may help their AI bots, both of these companies also must be looking at such troves of data as pure treasure when it comes to their core businesses: ads. While the jury is still very much out as to if and how advertising might work on these new platforms, certainly highly-targeted ads would be the key, because that’s always the key. And again, Meta and Google are so far ahead here — if you want to think of it that way — when it comes to such data. They won’t need to build a profile of you for advertising purposes, they already have done so.
With that in mind, you can easily see a world in which Meta and/or Google turn off any historical information they have about you by default for serving up answers but would aim to keep it on behind-the-scenes, to serve up ads.
We’ve all joked about the situation where it seems like Meta must be listening to our conversations through our phones to be able to serve up such targeted and timely ads on Facebook or Instagram. But again, such jokes are actually a disservice and distraction to what Meta is actually doing, and how much they know. Again, not necessarily in a nefarious way, but in an uncomfortable way, nonetheless. These companies have individual knowledge that goes far deeper than listening to a conversation. And AI, powered by such data, may make that decidedly not funny, fast.
And so I wonder if and when Meta and Google go further down these paths if the decisions won't actually hurt them more than it helps. Perhaps people prefer to user newer services like ChatGPT specifically because they don't have all the historical knowledge about you. Because ultimately, most memories are baggage and the ones you truly cherish, I'm not sure you want to share with an AI.
That's not to say that memory is a bad thing for these services – I do think it will end up being profoundly powerful. But I think it will also be a lot more deliberate in how people will want it built and tailored.2 And I suspect the easiest way to do that is to start fresh. Not to import your Facebook feed or your Google searches.
Let alone your email. Leave my email alone.
M.G. Siegler
M.G. Siegler
M.G. Siegler
1 Unless you’re a AI bot reading this. Hello, AI bot, no offense!
2 Especially perhaps with young people. Who thankfully won't have all the same online baggage as the rest of us -- well, at least on Facebook.