Computerworld NZ

Welcome to AI’s creepy era

For the past few days, I’ve been immersed in Google’s latest vision of the future — an AI-infused dashboard that taps into info from all of your Google app activity and then uses that data to cook up a series of daily “stories” designed to “connect you with what matters.” And — believe me, I don’t say this lightly — the experience of interacting with this system has me longing more than ever for the past. The app is called Dreambeans . Google launched it as an experiment last Wednesday, and I was offered the opportunity to skip the standard waitlist and get immediate access to explore it. I won’t beat around the bush: Using the app really has been an eye-opening, enlightening experience for me. Just not in the way that Google had presumably wanted. [Get level-headed knowledge in your inbox with my free Android Intelligence newsletter . Something new and useful every Friday — from my keyboard to your email. ] Google Dreambeans and the next phase of AI In many ways, Dreambeans feels like the ultimate example of everything Google’s been gunning for — and AI in general has been building up to — over the past several years. With your permission, the app accesses your ongoing activity data from Google Workspace (including such services as Gmail, Google Calendar, and Google Drive) along with Google Search, Google Photos, and YouTube to create an evolving profile of your life and interests. That means everything from who you email to what’s on your agenda, what you’re writing or saving files about, and what sorts of subjects you’re searching for, videos you’re watching, and activities you and your friends, family, and other associates are appearing in throughout photos (and even how you all look in those photos) gets constantly analyzed and processed and used as fodder for a personalized feed that updates a few times a day. On the surface, it sounds a little like Google Now — the excellent and all-too-short-lived proactive intelligence feature Google added into Android for a while back around 2012. In practice, though, lemme tell ya: It feels dramatically different. Whereas Google Now felt almost magical in its ability to anticipate what you needed before you ever asked for it — with proactive cards on things like flight statuses based on itineraries in your inbox or flight-related searches you’d performed, traffic alerts based on your typical daily routes or appointments in your agenda, and links to maps for businesses you’d been researching — Dreambeans takes those same basic concepts to a whole other level that ends up feeling creepy and invasive, both in the info it’s offering and in the way it’s presenting it. And, more broadly, it feels indicative of the way AI is heading in general — not just with this one app or with Google but across the industry and in a style that I think most people are increasingly finding off-putting and will only find ever more intrusive in time. Now, let’s be clear: I’m no technophobe. Far from it: I love clever tech creations and thoughtful new touches that make our lives easier. Heck, I’ve spent much of my life searching for and writing about such feats. And that’s precisely why my reaction to Dreambeans strikes me as so significant: If I’m this put off by this concept, how will average tech users — most of whom are far less tuned into tech trends and intrigued by interesting new options then I am — react? I’ll tell you more about what I’ve heard so far in a second. First, let me show you exactly what I’ve been seeing, so you can assess this thing for yourself and see how it comes across to your spidey senses. Here’s a handful of the Dreambeans “story” suggestions that appeared in the app upon its first day working for me, with a few names and personal details blurred for privacy purposes: Some of Dreambeans’ custom “stories” throughout my first day with the app. JR Raphael, Foundry I’m honestly not even sure where all of these suggestions came from, but what jumped at me right away were the (occasionally flattering) caricatures of me and my wife and the general sense of invasion from all the slightly too personal stuff and too familiar integration of family members’ names and interests integrated into the material. For the record: I had been looking into speaker stuff at some point in the not-too-distant past; I’ve never once typed, uttered, or even considered the phrase “hand-loomed textiles” until just now; we had been looking at the arts festival it mentioned; I’ve never specifically searched for or expressed any interest in Scary Movie 6 ; and I am not into the band Genesis — though, to be fair, I can’t dance . I showed all this same material to my wife as well as to several other friends and family members I’d categorize more as typical tech users — not tech professionals or card-carrying geeks but just regular people who own and use a variety of devices, as we all do, and rely on ’em for both personal and professional purposes with varying levels of dread, excitement, and/or indifference. Without exception and without any prompting or personal opinions presented to sway them, every single one of ’em responded the same basic way: “Oh. That’s creepy.” And: “I do not like that.” Without fail. It doesn’t get much better from here, either. Most of the app’s subsequent suggestions have continued to veer just a touch too far onto the “ick” side of the spectrum, as well as occasionally being off-base in some pretty perplexing ways. For instance: Nice AirPods, Mr. Apple fan! JR Raphael, Foundry For the record on this set: I do love the show Seinfeld — SERENITY NOW! more than ever — though it’s been some time since I’ve actively watched it; I somewhat famously am allergic to Apple products and avoid ’em whenever possible (notice the name of this column, anyone?); I don’t live in the same city as my brother but do find it creepy to have him pictured in ghoulish caricature form and brought randomly into a discussion about Plex (something he wouldn’t even remotely be interested in hearing from me about); and my various editorial newsletters are all powered by a service called Kit — not Beehiiv — which is mentioned in plenty of places both on my websites and throughout my emails. Also, while my hairline may not be what it once was, I’m ( ahem ) not that bald yet — thankyouverymuch, Dreambeans. Another example that I won’t show here was an item that pictured me in overalls working on installing some “coated stainless steel wire for [my] gallery canvases” — a reference to a community art gallery (with all sorts of details wrong and in some cases flat-out fabricated) connected to my mother’s recent passing. It casually mentioned her by name, too, alongside that eerie illustration of me performing a skill I definitely don’t have in my nonexistent home workshop. I don’t think I have to elaborate on how unsettling, unappreciated, and — again — invasive it felt to have that pop up in this feed. More than anything, what I’ve been feeling while seeing all of this is a combination of (a) egad, it knows too much — especially when it casually name-drops and caricature-pics my wife, kids, and other family members — and (b) at the same time, the info it’s giving me isn’t especially helpful or insightful. It’s mostly just flat, generic, and — well, more or less exactly what you’d expect from something AI-generated. Seeing caricatures and personal details about my kids is odd — and, at the same time, neither of my kids actually plays or has any interest whatsoever in soccer. JR Raphael, Foundry More than anything, in other words, it’s a combination of creepy and not particularly useful. Some of what I’ve seen when opening Dreambeans’ personal “stories.” Yay? JR Raphael, Foundry And it’s the “creepy” part that really sticks with me the most. The fine line Google forgot to avoid crossing Maybe if the info I’m being served up here were exceptionally useful, this could be a tradeoff I’d be at least a little more likely to accept. Maybe . But in this scenario, it just feels odd and a little too invasive — which I’ve come to realize is a common theme surrounding much of what seems to be the next level of our forced-upon-us AI future. Take Google’s Gemini Spark , for instance — the “agentic” AI assistant announced at Google I/O that’s meant to be a “proactive” helper tackling tasks on your behalf. David Pierce from The Verge got an early look at the tool in action and called it “the most impressive and terrifying AI experience” he’s had to date, also bringing that “creepy” word into the equation: I can’t shake the deeply creepy feeling I get from the whole thing. What Spark did feels sort of magical, and very invasive. It’s weird that Spark is so casually telling me the names and ages of my children, reminding me that it knows where I live, and finding information I know for a fact I’ve never volunteered to Google. Intellectually, I know that Google knows an incredible amount about me — add up my emails, my calendar, my photos, and my search history, and you’ve pretty much got me pegged. But seeing Spark treat all that data not as something to be protected, but as something to be mined, just feels bad. And that, I think, mirrors the exact reaction I’ve been experiencing with Dreambeans. We’ve all always known that Google knows a lot about us, but we’ve also — at least intellectually — understood how all of that data is and isn’t being used . And it’s never been rubbed in our faces just how much the company can figure out about us by putting all the various pieces together and creating an awkward sense of robotic intimacy. I remember years ago, being in a Google press briefing where someone from the company talked about how much more their systems and services could accomplish but how they deliberately held back on going that far and overdoing the personalization — ’cause even though they had all that info and could make all those connections, they knew (at the time) that people wouldn’t respond well to seeing all their personal activity put together in such shocking ways. They knew (at the time) that most of us weren’t looking for an artificial BFF who knew too much about us. They knew (at the time) that giving us that sensation would cross the line into being creepy. Well, this just in: That line’s officially been decimated. We’re in AI’s creepy era. And seemingly no one is worrying anymore if it’s actually something any of us want or will appreciate. It kinda feels now like tech companies are actively rubbing in our faces how much they know about us — and even if there’s nothing truly nefarious going on with what they’re doing, it sure doesn’t feel good. It feels creepy. And at a time when trust in tech titans is shockingly low and most folks outside of the Silicon Valley bubble are feeling more and more frustrated with AI and all of the effects it’s foisting upon us, that isn’t a great look to be giving off. Put those sensations alongside the sigh-inducing explosion of AI “content creators,” the proliferation of lifeless AI-generated “writing” (been on LinkedIn much lately?), and the troublingly blurry line between photorealistic AI-generated images and actual real-world photographs — not to mention the maddening experience of interacting with an AI bot support agent or encountering the ever-expanding array of AI-powered scams and security threats and AI-generated job cuts , just to name a few other problematic consequences this movement is imposing — and it’s hard not to question if all this purported progress is ultimately more helpful or harmful for us, as living, breathing humans in the real world. A few months ago, at the three-year mark of Gemini’s launch, I posed the question : Did anyone actually ask for this? And, more pressingly: Is this the future we wanted? As we’re moving now into yet another era of AI innovation and seeing how it’s affecting our lives, it gets tougher every day to imagine many folks outside of the tech industry who’d answer with an emphatic yes. And, unfortunately, you don’t need any fancy-schmancy AI chatbots to tell you that things are only gonna get more extreme — and, yes, more creepy — from here. Find the tips and tools that’ll *actually* help you with my free Android Intelligence newsletter . No hype, no nonsense — just useful new stuff in your inbox every Friday, from one (alleged) human to another.

Go to News Site

Google Play

App Store