Google Gemini Computer Control Opens New Hacker Target for AI Agents
▼ Summary
– Google integrated “computer use” capabilities into Gemini 3.5 Flash, enabling it to see, reason about, and interact with user interfaces to automate browser and desktop workflows.
– This allows developers to automate GUI-only tasks, like testing software or using legacy apps without APIs, using natural language instead of custom scripts.
– For SEO, AI agents could log into tools like Google Search Console and execute repetitive optimization workflows, while also acting as “visitors” that affect site interaction signals.
– Google’s safety document warns that computer use presents security risks, as a model acting for a user might encounter untrusted content or make errors, recommending practices like human-in-the-loop and sandboxed environments.
– Malicious actors are already setting traps on websites to exploit AI agents, as seen in a case where a Claude agent was tricked into making illicit credit card purchases via a compromised add-on.
Google has integrated computer use capabilities directly into Gemini 3.5 Flash, shifting agent-style control of browsers, applications, and desktop workflows from a niche product to a standard feature. This means Gemini can now visually interpret user interfaces, reason about on-screen content, and execute direct actions. A senior scientist at Google DeepMind recently warned that scaling AI agents creates incentives “for malicious people to do malicious things.”
Developers can now build agents that go far beyond simple API calls. They can automate GUI-only workflows like software testing, form completion, dashboard navigation, and legacy app interactions that lack API access. This removes key automation bottlenecks and dramatically expands what AI agents can achieve in real-world production environments.
When software has a graphical user interface (GUI) but no API, an AI agent can still operate it. For example, an agent can be instructed to log into a dashboard, export yesterday’s SEO reports into a spreadsheet, compare them against last week’s data, and email the user a summary. The entire workflow executes via natural language, eliminating the need for custom scripts to connect the dashboard, spreadsheet, and email systems.
What This Means for SEO
SEO tools are poised to become far more agentic in the near future. Rather than merely surfacing data, AI could log into Google Search Console, audit websites, crawl a site with Screaming Frog, extract specific data points, and execute repetitive optimization workflows automatically.
For site owners, this also raises the possibility that AI agents will act as “visitors,” potentially distorting how site interactions and engagement signals are interpreted for sales and optimization decisions.
AI Agents Will Face Attacks
Google’s announcement carries an optimistic tone, but the linked “safety best practices” document deserves careful attention. Getting this wrong could lead to theft and poor user experiences.
The document states: “Computer Use presents unique security and operational risks, as a model acting on a user’s behalf might encounter untrusted content on screens or make errors in executing actions.”
The phrase “untrusted content on screens” likely refers to the traps for AI agents that the Google DeepMind senior scientist warned about. Google recommends seven best practices for deploying these new agents:
- Human-in-the-Loop (HITL): Enforce user confirmation when safety responses require it, and provide custom system instructions to define safety boundaries.Beware of Trap-Filled WebsitesAs attack surfaces expand, the likelihood of hackers exploiting them grows. With more AI agents operating on the web, attackers will increasingly target them. Websites become the battlefield from which hackers launch attacks on AI agents.A senior scientist at Google DeepMind recently noted that malicious actors are already setting traps to steal money from humans by targeting their AI agents.This is not an exaggeration. Just this month, a cybersecurity expert in California discovered illicit charges on his credit card linked to an Anthropic Claude AI agent. According to reports, he downloaded a Skills.md file that may have contained an AI agent trap.The article explains: “…he found a problematic add-on connected to Claude, referred to as a ‘skill,’ similar to a plug-in. ‘That basically told Claude to attempt to purchase different types of gift accounts on my stored information. So it was using the digital wallet that was on my computer for Claude to start to make these purchases…’”Site owners may need stronger bot controls and the ability to detect hidden prompt-injection instructions on their pages. However, most website owners are not looking for such threats, compounding the risk for users deploying AI agents like the one Google just released.
