When the European Parliament’s IT team quietly switched off the AI features on staff devices earlier this year, the reason wasn’t speculative. The features were sending data to the cloud, and Parliament’s security team decided that was a risk they couldn’t accept for an institution that handles confidential legislative work.
The story barely registered outside Brussels, but it captured something most consumers are only starting to grapple with: the AI devices entering our homes, offices, and schools are not all built the same way – and the differences are no longer trivial.
Apple’s rumored next-generation AirPods, reportedly to include cameras, are now facing similar scrutiny, even before launch. The company has said the wearable isn’t designed to record or photograph users but the fact that consumers and journalists immediately asked the question anyway is telling of where the trust baseline now sits.
For founders building AI products for everyday environments, this is a strategic problem with no easy answer: build for the strictest jurisdiction and you slow down, spend more and watch faster-moving competitors race past you; build lean, accept the regulatory risk and you may find the door to enterprise customers, schools, or public institutions closed before you can knock.
A handful of European companies have decided the first path is the only viable one, and their reasoning offers a clearer framework for evaluating which AI devices truly belong in users’ most personal spaces – more so than any guidelines regulators have shared.
The architecture choice happens before the product does
Deep Care, a German workplace health company whose AI coach Isa is now used by close to 50,000 people across more than 280 organizations including Fortune 500 firms, designs its product for desk workers, helping them improve posture and avoid musculoskeletal issues that come with long hours at a screen.
In the context of increasing scrutiny of wearables crossing into sensitive personal spaces, the defining design decision was made before writing most of its product; the team’s first iteration used a smartphone app, with cameras, and was tested by 100 users. Ninety-two said they didn’t want it.
“They said, look, I am watched eight to ten hours per day with a device which is connected to the internet and has two cameras front, three cameras back,” explained Milad Geravand, CEO and co-founder of Deep Care. “I’m putting my passwords in. It sees everything. It sees what I’m wearing, what’s behind me. Such a solution doesn’t make sense.”
Deep Care replaced the smartphone and camera entirely with a sensor that can map a 3D outline of a user — enough to track posture and movement — without being able to identify who the user is or what’s in the room behind them. The product runs offline, analysis happens on the device, and nothing is sent to a cloud – because nothing needs to be.
That decision was expensive. “Switching from a camera to this specific sensor costs significantly more in hardware, but that’s the smaller part. The bigger investment was building everything from scratch: gathering our own data, training all AI models from the ground up, and developing the entire system ourselves, with no shortcuts available,” Geravand says.
“Running all the code locally on the device so that people feel safe – that costs us more because we need to have a better processor. Developing a solution which brings this value that you feel safe and secure costs us more. Therefore, we need to have that cost in our pricing.”
The cheaper approach which other workplace AI products take, he argues, is structurally cheaper for a specific reason: “Putting a cheap, non-secure sensor in the product and streaming everything to the cloud is an easy business decision; it slashes hardware costs and opens the door to selling that data to third parties for significant revenue.”
But the price users pay isn’t measured in euros alone. “You either pay with your money, or you pay with your data,” Geravand argued, adding that while the latter feels free, it never truly is. Once a user’s data is “out there” it will inevitably get used by insurers, advertisers, and employers, to name a few.
“What started as a few euros of savings can quietly cost you far, far more.”
Children’s voice data: The case for the highest bar
Meanwhile, children’s AI learning companion Buddy.ai made an architecture decision shaped by a different but related constraint: COPPA, the U.S. children’s privacy law, treats voice recordings as personal identifiable information. Routing children’s speech through a commercial AI API was never an option.
“We process everything on our own infrastructure, in real time, and it’s gone before a parent has even had the chance to think about consent,” says Ivan Crewkov, the company’s founder. “Did it cost us? Yes. Building and maintaining private cloud infrastructure is not cheap, and it adds a development burden that commercial API users simply don’t have.”
Regardless, Crewkov stressed he would make the same call again because, while foundation models are moving fast, regulators aren’t. “The compliance bar is the same whether GPT-5 exists or not.”
What he didn’t anticipate, however, was how the constraint became a competitive asset. Because COPPA restricts collection of children’s voice data, most of the field is working from datasets of a few hundred hours. Buddy.ai has accumulated an invaluable tens of thousands.
What buyers should actually ask
Both founders, when asked what evaluators should look for, gave answers that are sharper than most public guidance currently offers.
For procurement teams, Geravand stressed the question is largely settled inside European companies – even if it isn’t on the consumer side. Dedicated data privacy and data security departments operate as gatekeepers: a vendor that can’t satisfy them doesn’t reach procurement at all.
“If there is any no, you don’t end up in procurement. Of course, if there is a yes everywhere, then you start talking about pricing and length of contract.”
Camera-based solutions, in his experience, almost never clear this bar in European workplaces because the region’s legal framework requires that any camera-monitored area be clearly marked – and most employers don’t want such a conversation triggered with works councils.
For parents, Crewkov’s framing is more direct. “The one question I’d want every parent to ask is: what is this actually for? What does it teach? What are the learning objectives? If the people behind it can’t answer that clearly, that tells you something important.”
The products he is most cautious about, he says, are general-purpose AI companions sold to children with no clear educational purpose. “Kids don’t need an AI companion. They need other children, toys, and the world around them.”
He also flags an asymmetry stakeholders across the board should pay attention to: third-party certification organizations like kidSAFE, which independently audit children’s AI products in the U.S., don’t operate everywhere. For users – families, in his case – outside markets with that infrastructure, the practical advice is unglamorous.
“I know [reading the privacy policy] that sounds obvious, but most people don’t do it, and for a product that’s listening to your child’s voice, it’s worth the ten minutes.”
The implication doesn’t solely apply to guardians, either. Moving forward, founders seeking to tap into these markets should know that the privacy extends beyond their target users – especially in the case of educational, tutoring or child companionship tools. Increasingly, more scrutiny will be placed onto clearly-delineated privacy benchmarks, especially when these are not explicitly stated by law.
The thing regulators can’t do
Europe’s regulatory architecture composed of the General Data Protection Regulation (GDPR), the EU AI Act, and sector-specific rules on workplace monitoring, has set a floor that has measurably shaped what consumers, employees, and procurement officers can expect. But the founders building closest to that floor agree on something the regulation itself can’t deliver.
The choice of what AI devices users let into their homes, teams, and children’s rooms is, ultimately, made by them at the installation moment – before any regulator can intervene.
In this sense, the questions worth asking aren’t complicated: what does this device need to function? Internet connection, a camera, persistent learning? Do the potential users in a target region or market possess these accessibly?
And, more poignantly, what happens to the data it collects? Where does it go? What is the company’s business model, and does it depend on doing anything with users’ data that they would object to if read on the fine print?
The answers don’t all have to be perfect, but if a company can’t answer them clearly, that itself is the answer.
Featured image: Getty Images via Unsplash+
Disclosure: This article mentions clients of an Espacio portfolio company.
