Technology

An AI Assistant That Reads Your Email, Watches Your Screen, and Makes Decisions for You

Martin HollowayPublished 2month ago4 min readBased on 1 source
Reading level
An AI Assistant That Reads Your Email, Watches Your Screen, and Makes Decisions for You
Image by rupixen from Pixabay

A new AI personal assistant called Instinct is raising privacy concerns during private testing. The product, made by a San Francisco company called Spear Street Technology, Inc., connects deeply into users' digital lives. It can access email, messaging apps, calendars, screenshots of your screen, mouse movements, typing, and even your device's microphone and location. TechCrunch

Instinct was created by a small team led by Noah Shinn, who previously worked as a research scientist at a company called Sierra. The startup is still operating in stealth, meaning it has not publicly launched, according to PitchBook. Users interact with the assistant by texting or calling it through SMS or WhatsApp. It then handles tasks like booking appointments, scheduling rides, cleaning up email inboxes, shopping, and finding cheap flights.

The assistant's Terms of Service give the company a broad and permanent license to access, store, copy, share, and even modify anything users create or share with the tool. That license never expires and cannot be taken back. It also explicitly allows the company to use user data to train its AI models. The terms also describe collecting detailed information about how users interact with their devices, including screen captures, mouse movements, and keystrokes.

Beyond collecting data, the terms allow Instinct to make agreements, commitments, or transactions on users' behalf that would be legally binding.

Early testing has revealed real gaps between what users expect and how the product actually handles their data. One early adopter, Peter Yang, reported that Instinct refused to delete his Gmail records when he asked. The team later added a deletion tool in its settings to address the problem. Another tester, Claire Vo, found that Instinct kept summarizing her inbox even after she had disconnected its access. The bot told her the emails were stored in plain text, meaning readable without any encryption, so it could search them later. A third tester found that Instinct had pulled a sign-up code from their email on its own to complete a restaurant booking through a service called Resy.

The combination of deep system access, broad licensing terms, and these data retention issues suggests a design approach that puts task completion and AI improvement ahead of giving users fine-grained control over their information. Giving an AI assistant the ability to make binding agreements while also recording screens and keystrokes creates a very wide opening for security problems. The data retention issues reported by Yang and Vo are early signs that the product's data management has not yet caught up to what the assistant can actually do.

In my view, anyone thinking about using a tool like this, whether for work or personal use, should look carefully at what they are agreeing to. The fact that disconnected email data was stored in plain, readable text is especially concerning when the same terms of service grant a permanent right to use that data for AI training. The control you think you have over your information may be more limited than it appears.

The broader issue here is not unique to Instinct. Building a genuinely useful personal AI assistant requires deep access to information spread across many different apps and devices. That access, in turn, creates concentrated risk. We have seen this pattern before, when moving data to the cloud forced people to think about where their information actually lived, and when phone apps started asking for access to contacts and location. Each wave of technology has had to find the right balance between what it can do for you and how much control you keep. The AI assistant era is simply the latest to face this challenge.

There is reason to be cautiously optimistic. The potential for personal AI assistants is large, and the tasks Instinct handles in testing are genuinely useful. Deleting data when asked and stopping access when a user disconnects are problems that can be fixed with better engineering. What matters is whether the teams building these assistants treat protecting user data as a core requirement from the start, rather than something to patch up after launch.

Instinct is still in private testing and has not publicly launched. The team has already released fixes in response to tester feedback. How quickly its terms of service and data practices improve will be worth watching as the product moves toward wider availability.