Think our tech is getting a little too smart for its own good? You’re not alone.
In the past few weeks, TechRadar brought you three key updates on the invasion of your privacy without consent, owing to the AI learning models nestled in our gadgets and appliances.
In case you missed the latest missives, new LG TVs can record you in the background and will ask you to notify anyone who comes into your home, to comply with “wiretapping” laws. Next up, Nothing’s new earbuds offer a ‘squeeze the stems to record calls’ feature being called Audio Snapshot, which is tantamount to phone-hacking in cute packaging. And finally, there’s the continued backlash over Meta’s AI glasses. Despite the social media giant implementing a feature that stops recordings if a wearer tampers with the front-firing light, hatred of these and other smart shades (oft-renamed ‘pervert glasses’ by the general public) has led to people calling for camera-less glasses.
Latest Videos FromTechRadar
So, in a world where our glasses can watch, our earbuds can hear, our speakers can speak and our TVs are a little too closely invested in our lives for comfort, how much more will we allow our tech to intrude before enough is enough — and what can we do to regain some privacy?
‘I see this as part of a bigger pattern that is a push by technology and other forces to normalize consumer surveillance’
Lauren Hendry Parsons, Mozilla Foundation (Director, Communications)
Your tech will see you now
Lauren Hendry Parsons is a seasoned advocate for digital rights and a gifted speaker in making these complex issues accessible. She is a director within the non-profit Mozilla Foundation, helping to make the organization’s push for a “technology future that serves and is shaped by the people who use it” a message that resonates on a global scale.
Luckily, she has agreed to talk to me on the growing concern over surveillance tech’s prevalence in our lives.
Parsons states that in order to zoom in on these issues, we need first to zoom out. “I see this as part of a bigger pattern,” she begins, “that is a push by technology and other forces to normalize consumer surveillance — surveillance of people by other people — and that’s something that we’re seeing cropping up a lot.”
So what’s changed — after all, recording devices have existed for some time? The answer is the growing number of places where such surveillance is becoming normal.
“It’s something that started in the Internet of Things home devices,” she says. “You might talk to a smart speaker, or talk actively to another product that is doing your bidding — but it was usually tied to a place.
“Now, we’re seeing the normalization of surveillance-chic behaviors, where fashion influencers are sharing their own Ring doorbells to show off their outfit of the day.
“That moves surveillance from something fixed inside a home, where people can at least see the object and talk about it, to something that’s constantly moving through every space and every conversation.”
And it’s being packaged in pink earbuds, chic sunglasses or fun doorbells? Parsons agrees: “The chicification of surveillance to me is deeply concerning.”
‘It moves surveillance from something fixed inside a home, to something that’s constantly moving through every space and every conversation’
Lauren Hendry Parsons
Who counts as the user — and should they really have control?
I want to talk about Meta’s AI glasses and the involvement of Kylie Jenner in the ad campaign, especially when women’s safety (regarding non-consensual surveillance and distribution of such footage) has been listed as a key concern.
I mention the fact that when speaking to several male colleagues about the benefits of such glasses (rather than the concerns), not one of them mentioned what I suspect all women thought: that wearing such glasses might mean filming your potential attacker, and that would be a very good thing.
“There are absolutely situations where people might want the convenience these products provide,” Parsons agree. “Being able to translate something on the go or take a quick snapshot without getting your phone out. For people with disabilities or limited capacity, these tools could be genuinely useful.
“But I don’t see the trade-offs being clearly explained. I don’t see meaningful ways to opt out. Even if these companies have perfect data handling processes and perfect privacy records, the risk is being transferred to the consumer instead of being held by the tech company.”
Parsons mentions one of the core issues at Mozilla Foundation: privacy by design. “Privacy by design, which is what we recommend as best practice, means products should be private by default,” she says. “And that has to include bystanders.
“Technology companies talk about giving the user control, but who’s counting as the user?
“With smart glasses, televisions, and recording earbuds, the purchaser isn’t the only person affected. Just as children are affected by smart speakers or passengers by dashcams in cars, the person across the table or on the other end of the call is also a data subject, even though they have no access to the controls and no real way to opt out.
“True privacy by design must protect both the person operating the device and the people involuntarily brought into its data collection environment.
“When something (with surveillance tech built in) is fixed in your home or office, you can at least choose not to go there. You can suggest meeting somewhere else. That’s difficult and has social consequences, but it’s possible.
“If it’s something someone is unobtrusively wearing in their ear or on their glasses, or something you don’t even know they have, opting out becomes almost impossible.”
‘No surprises, genuine user control, limited data collection, sensible settings, and multiple layers of protection. These examples show what happens when products fall short of those principles’
Lauren Hendry Parsons
I ask how we might regain some agency here, or at least get a handle on our privacy regarding AI glasses, call-recording earbuds and all-seeing TVs. Parsons points to the work she does.
“Mozilla’s privacy principles are useful here,” she says. “They’re about no surprises, genuine user control, limited data collection, sensible settings, and multiple layers of protection. These examples show what happens when products fall short of those principles.”
Another thing Parsons is keen to impress upon TechRadar is central to her work: understanding the trade-offs. “I don’t see how everyone on the phone can understand those trade-offs and give informed consent,” she says.
“The only way that could happen is if this (the recording of everything we do) became such a standard operating procedure that everyone expects it, because everyone has it. At that point, we would have completely accepted constant surveillance as the norm.
“I don’t think the answer is necessarily to prohibit recording technology. It’s to stop presenting recording as an ordinary, consequence-free convenience.”
‘I don’t think the answer is necessarily to prohibit recording technology. It’s to stop presenting recording as an ordinary, consequence-free convenience’
Lauren Hendry Parsons
She mentions translation in AI glasses as an example: “Companies could show translations without collecting or storing the underlying data, but we know that’s not what’s happening.”
Another watch-word in Parsons’ work is ‘friction’, and how disproportionately easy it has become to monitor, film and record secretly and without consent.
“Features with serious privacy implications should involve equal and proportionate friction,” she says. “The more sensitive the activity, the stronger the consent mechanism should be.”
I mention that when testing a set of earbuds recently, a simple beep to signify that recording has begun is the only notification the caller on the other end of the line receives. Parsons believes concern there is valid.
“Notification is not consent,” she says. “We’ve become used to surveillance in places like train stations and public spaces because it’s framed as being for safety. But I’d hate to see that extended to person-to-person interactions.”
‘Notification is not consent. We’ve become used to surveillance in places like train stations — but I’d hate to see that extended to person-to-person interactions’
Lauren Hendry Parsons
I ask if perhaps people should simply get OK with omnipresent filming or data collection by individuals and wider companies, since it seems so hard to avoid? Parsons’ answer is emphatic: “That’s deeply problematic.”
“We’re seeing the expectation that people should be comfortable being filmed all the time. Even terms and conditions often state that by attending, you agree to being photographed or filmed for marketing purposes. The default is that your data is collected unless you make the effort to opt out or avoid the event entirely. Again, that’s deeply problematic.
“The diminishment of privacy is cumulative. It’s one thing to have a fixed camera in a train station. It’s another to be filmed by multiple people in different places without your consent, or to have your audio collected.”
‘It’s one thing to have a fixed camera in a train station. It’s another to be filmed by multiple people in different places without your consent’
Lauren Hendry Parsons
‘The burden of this issue is not distributed equally’
Parsons reads me a quote from one of her own features, and it cuts to the core of the agency, consent and privacy debate better than anything I could ever try to surmise.
“The burden of this issue is not distributed equally, and the risk of ambient recording will fall disproportionately on people who already have less control in a situation,” she says. “That includes workers, children, parents, patients, tenants, domestic abuse survivors, and protesters. This makes vulnerable populations more vulnerable.”
Turning the cameras around
Parsons mentions a novel but initially counter-intuitive solution suggested by some prominent female content creators. Instead of shunning Meta’s glasses (or playing Disney songs if you realize you’re being filmed, because Disney’s music is better protected than women’s lives), the idea is that women do wear them, specifically to film men when they start to behave in an inappropriate or predatory manner, then upload the footage everywhere. The thinking is that these products might then get regulated very quickly.
“If women collectively used them in that way — in a kind of, let’s call it a neo-Me Too movement — I’d put money on consent issues being addressed much more quickly,” Parsons says.
“The issue isn’t simply that these products can record. It’s that the examples you gave (the earbuds, the glasses, the TV that records in the background) all prioritize effortless capture over meaningful consent, when there are other ways to do it.
“The person operating the product gets all the convenience and control, while everyone else gets a light, a beep, or a legal disclaimer. That’s an unacceptable imbalance.
“Recording has many legitimate and valuable purposes, but companies should be asking how to build consent into the infrastructure, minimize what’s captured, and protect bystanders as rigorously as they protect the user.
“The false dilemma is, ‘Do you just not want this technology?’ But we shouldn’t have to choose between useful technology and privacy.
“Preserving both means companies need to treat consent as a product requirement, not something buried in their terms and conditions.”
‘Companies need to treat consent as a product requirement, not something buried in their terms and conditions’
Lauren Hendry Parsons
Facial recognition (and how to make it disappear)
I ask Parsons about Mozilla Foundation’s stance on digital facial recognition, a highly controversial feature that our phones and smart glasses are technically capable of. Although the big brands have so far disabled or restricted the software on glasses, there are strong suggestions that hackers have managed to scoot around it.
She mentions the rise of anti-surveillance fashion; garments that make their wearer invisible to the algorithm and thus can thwart the surveillance tech stuffed into our handsets and wearables.
“Looking ahead, we have two things happening simultaneously,” Parsons says. “We have vastly increased computing power and data analytics capabilities that allow us to store and analyze more information than ever before, and we have recording devices embedded everywhere in increasingly unobtrusive ways. That combination is worrying.
“One of the brands we spoke to, Capable Design, explained it well: ‘The problem with surveillance technology is citizens don’t have a possibility in the physical world to opt out. In the digital space, we can say yes or no to cookies. In the physical world, we don’t have this option. We wanted to create physical protection that would allow citizens to give their consent or not.’
“It feels as though we’re being forced to pick sides: either you’re comfortable with surveillance or you’re actively trying to avoid it. Again, it comes back to the false choice between privacy and convenience because technology developers aren’t prepared to give us both.
“I think these efforts are people trying to take back some agency. There’s a feeling that all of this is inevitable.
“Particularly in the UK, where social norms and politeness are strong (this interview is taking place in the UK), I worry we’re going to politely accept our way into a surveillance state because people feel uncomfortable challenging others or assume this is an inevitable progression.”
‘Particularly in the UK, where social norms and politeness are strong, I worry we’re going to politely accept our way into a surveillance state’
Lauren Hendry Parsons
Our time together is nearly up, but Parsons wants to talk just a little more about the future. “I want to say this loudly because it’s a core Mozilla Foundation belief: the future of technology is not inevitable,” she says.
“These are choices being made by people within companies. They could choose a different path.
“That’s why Mozilla Foundation funds projects and people who challenge the status quo. We’re investing in positive alternatives because opting out as a consumer can feel almost impossible.
“We need companies making better choices, regulators setting stronger requirements, and a greater appreciation that friction isn’t always a bad thing. A little friction can protect people, encourage creativity, and ensure meaningful consent.
“That’s what I hope for the future: that we don’t accept inevitability, that we invest in better technology choices, and that responsibility isn’t pushed entirely onto the individual consumer at the point of use.”
I want to say this loudly: the future of technology is not inevitable’
Lauren Hendry Parsons
AI another way
So, are there people within companies — even AI companies focused on “autonomous agents, edge intelligence, computer vision, and robotics” — looking to make different choices? Actually, yes.
Div Garg — CEO and Co-Founder of AGI (an AI on-device ‘personal assistant’ company) — is perhaps most famous for turning down a big gig at OpenAI in order to work on a “Siri that actually works” within his own startup. Fortunately, he’s prepared to talk to me about the concerns surrounding the scraping (and in-cloud storage) of our personal data for AI learning.
“The debate around whether AI wearables that record our everyday lives are an invasion of privacy overlooks the fact that this technology doesn’t need the cloud whatsoever,” he had said, prior to our meeting today. “It’s cheaper and easier for manufacturers, but it isn’t a necessity. Put the model on the device, and no data ever has to go beyond it.”
Now, I ask him to expand. “My company hasn’t cared about audience growth,” he says. “We believe all AI agents and assistants should be running on-device, so the user should own the model and the data.
“What’s happening now is people are typing their data into services, and companies use that data to show you ads. I think all of this can be misused to sell you more things, and we don’t want that.
“AI is great, but it’s only great if it’s neutral and genuinely beneficial. It shouldn’t become a source of revenue for corporations through people’s data. That’s what we’re trying to change. We believe this kind of edge AI is the future, and we want to build something that’s fully privacy-safe and that people can truly trust.
“We don’t want people giving away their data without knowing more about it. We need the opposite approach, and I don’t think enough people are thinking about this.”
Garg pauses. “Maybe Apple should have been the company doing this,” he adds, “but they’ve been lagging behind when it comes to AI”.
I agree, and ask him why that might be? “No one has really taken the initiative to solve these privacy concerns. We’ve been holding a lot of events recently; for example, the founder of Edward AI, who built Edward before it was acquired by Google, joined us as an advisor. He believes in building the next personal operating system.”
Did you know TechRadar now has membership?
Become a TechRadar Insider by simply clicking ‘Join Now’ at the top of this page. Have a question? Please email membership@techradar.com
Trust issues
Can AGI build something that’s truly personal, where people trust it, and where everything lives on-device? Garg believes so but, interestingly, he doesn’t see trust as the deal-breaker.
“A big thing we’re proposing is almost a zero-trust approach,” he says. “You don’t actually have to trust the provider. If the AI model itself is guaranteed to be running on the device, then no data leaves the device. You don’t have to trust that a company is telling the truth — you know because of the way it’s built. That’s where I think the world should be moving.”
‘You don’t actually have to trust the provider. If the AI model itself is guaranteed to be running on the device, then no data leaves the device. You don’t have to trust that a company is telling the truth’
Div Garg, CEO at AGI
The idea that brands are prioritizing AI in a product, because they know it’s a buzzword, but with a more watchful eye on profit margins rather than the privacy of a user’s data (and consent to capture that data) doesn’t sit well — with either of us.
So why might companies try to not keep AI on-device, aside from cost issues? Does it drain your battery? Does it make a device heat up? Does it use a lot of RAM?
“I’d say you have to think outside the box,” Garg says. “It requires a lot of innovation. You have to figure out how to make these models small enough to run locally.
“Modern phones (because this is where Garg sees AI, as the chief controller of your other devices) now have GPUs and processors called NPUs that can run AI models. I think the world is moving towards deploying more of these models on phones, but it’s still very new.
“For example, we have a collaboration with Qualcomm. We’re talking with their R&D and machine-learning teams about how our models can be highly optimized for phones. We want to make sure they don’t drain your battery, cause overheating, or create other issues.
“This is a new frontier in technology, and you have to be innovative to make it work.”
I ask Garg what he would say to anyone wary of adding a personalized AI model to their lives? “It depends a lot on who’s making the decisions,” he says.
“We want consent. We want to make sure people aren’t being recorded without permission — I’m pretty sure there will be lawsuits around some of these issues in the future.
“My belief is that AI should be consensual. Even if you download a new AI app, it should ask, ‘Are you OK with us collecting your data?’ If someone gives consent, that’s OK. But if someone says no, they shouldn’t be forced into it.”
On-device only
Garg is obviously only interested in an on-device AI future, but I worry that affordable convenience often wins out and that people may not understand the true benefits of such precautions.
I mention a set of headphones I tried, that offer real-time cognitive strain detection. My issue was that these cans (which were $200 cheaper than the inaugural 12-channel EEG sensor over-ears) use cloud processing in combination with on-device processing for some of the new metrics. The company didn’t reveal exactly what data was being sent to the cloud, but it had the potential to change my mind pretty drastically about using the product. What if the publishing house I work for was able to access and monitor how hard I’d been focusing throughout the work day?
Garg laughs, but only for a second.
“I think it’s definitely possible, but it comes down to the developer,” he concedes. “If a developer is doing something with your data or trying to collect it, that’s hard to detect unless you’re the operating system itself or the phone manufacturer.
“The phone company can probably detect it and tell you, but a lot of this comes down to how much you trust the people building the software. That’s one reason we’re so focused on the operating system.
“Right now, if you use something like Claude, you have one place where you do everything, and it connects to different services. We believe in a world where you have a single personal AI assistant connected to different devices, like the neuro-headphones you mentioned. Then you can see for yourself where the data is going because everything is connected through your own personal assistant rather than being scattered across different services.”
Are we ready for unique personal AI assistants?
I mention Pete Steinberger’s OpenClaw and the early adopters who tried to create their own personal AI assistant over a weekend. Does Garg think people are ready for that?
“We definitely want to make it easier,” he says. “Even if you’re using something like OpenClaw, you need to buy hardware, set up API keys, and do a lot of technical work. Most people don’t have the patience for that, especially if they’re not developers.
“We want this to work out of the box. You download an app, and it starts learning your routines. It can make proactive suggestions like, ‘Would you like me to order your favorite coffee?’ It’s learning your habits.
“If it’s all happening on-device, that solves the privacy problem because you can have personalization and all these useful features without your data leaving the device.
“Over time, your phone becomes more intelligent. It gets to know you better and becomes better at helping with the things you like to do.
“We also think this works well with smart glasses and other wearables. Your phone becomes the control center, just like an Apple Watch connects to an iPhone. The other devices become satellites connected to your phone, and the AI model lives on the phone itself.”
Despite my continued reservations, it’s a compelling argument. I can absolutely see a world in which a personal, on-device AI might be helpful in simplifying our lives.
I have one more question, since Garg is the visionary developer here. Is AI still essentially a loop (meaning something that is not intelligent in itself or autonomous — just very persistent at completing tasks) and does he ultimately see it moving beyond that? Or will humans always have the agency and autonomy, with AI just working in the background?
“A combination of both,” he responds in an instant. “You want it to feel like collaborating with a co-worker. Depending on how much you trust it, it could manage your calendar or your emails automatically.
“It could say: ‘I know you have a meeting tonight. Would you like me to help you find a restaurant? Would you like me to call a cab?’ You want a system that’s learning about you, but you also need trust. You need to know you’re not being exploited by giving it access to your data, and that it’s genuinely beneficial.
‘If you use ChatGPT or Claude today, they’re serving the same thing to billions of users. Everyone has the same assistant. In the future, people will want their own completely personal assistant’
Div Garg
“That’s the missing piece. If you use ChatGPT or Claude today, they’re serving the same thing to billions of users. Everyone has the same assistant.
“In the future, people will want their own completely personal assistant. Maybe you’ll even give it a name! It’ll be unique to you. You can give access to your own data because it’s running on-device. You trust it, and no one else has the same AI.”
Of course, myriad users already trust AI chatbots with their innermost thoughts, and even as romantic partners, so perhaps by continuing to give the tech a wide berth I’m in the minority.
Then again, it’s actually not the technology I fear. It’s the technology in the hands of people who seek to misuse it.
Thoughts? There’s a comment section below, dearest gentle reader — and my human eyes are ready to read your human thoughts.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
The best turntables for all budgets
https://cdn.mos.cms.futurecdn.net/kYf4AgYPKP7fQjoErXAhYa-2000-80.jpg
Source link
becky.scarrott@futurenet.com (Becky Scarrott)
