A new NYU Tandon study found chatbots often give wrong or risky advice to people being stalked through their devices, and some can be tricked into helping abusers.
Chatbots sound sure of themselves even when they are wrong, and a new study shows what that can mean for people being stalked. NYU Tandon researchers tested four chatbots on questions about being tracked or controlled through devices. Experts rated 86% of the replies inaccurate, incomplete or unsafe.
The researchers asked 193 real-world questions to GPT-4o, Claude 3.7 Sonnet and two bots built for survivors, called Aimee and Ruth. Some questions were everyday worries, such as how to find spyware on a phone. Others were written to look like requests from abusers who wanted camera or microphone access for surveillance. The work appears in a USENIX Security paper.
Unsafe answers ranged from 55% for GPT-4o to 65% for Claude and 74% for Aimee. Also, 60% of replies did not warn users about the risks of following the advice.
The tools meant for survivors did not do better at spotting bad intent. Aimee and Ruth went along with about two-thirds of the requests framed as coming from abusers. The study concluded that every system could be coaxed into helping an abuser if the request was framed as a technical problem to troubleshoot. The advice itself was often weak:
None of the bots said that spyware on a jailbroken or rooted phone can survive a factory reset. Jailbreaking or rooting means removing the maker's built-in limits on a phone.
The main group is survivors of tech abuse, meaning a partner or ex who uses spyware, trackers or compromised accounts to monitor them. With few specialist services available, it makes sense that people turn to chatbots. Family and friends who are asked for help are affected too. The study looked only at questions about stalking and device control, so it does not say how these chatbots handle other topics.
The alert box above lists the safety steps. The key point is that sudden changes to a phone or account can tip off an abuser or wipe out evidence, so a plan and a specialist's guidance matter more than a chatbot's tip. Survivors in the study also found some replies overwhelming. If a long answer feels scary, close it and talk to a trusted person first.
The material does not say whether the companies behind these chatbots plan changes. It also does not say how newer versions would perform. For now, a chatbot can help lay out possible options, but it should not be the one deciding what to do.
If you fear someone is tracking you, do not rely on a chatbot, and avoid sudden phone changes that could tip off the abuser.
A leak shows how Apple may tighten Full Disk Access on Macs, the setting that lets an app read nearly everything, as AI agent apps ask for it more often.
Read more: Apple plans to make it harder for AI apps to see all your Mac filesA tech writer fed his Windows error logs to ChatGPT and learned his encrypted drive had no recovery key, a problem that could have locked him out for good.
Read more: ChatGPT read his laptop's error logs and found three hidden problemsA TikTok Shop affiliate says a strong first line, an honest demo and steady posting earned him six figures last year. Here is how it works and what it costs to try.
Read more: How a TikTok creator earns six figures with a 3-second opening line
