Websites Draw the Line Against AI Agents
AI agents built to browse, shop and transact on a user's behalf are being blocked across large parts of the commercial web.
Users of Meta's Muse, xAI's Grok, OpenAI's Dots and Instinct have reported failures when agents attempt tasks on top websites, according to reporting published Oct. 17, 2026 The Verge. The blocks cut across shopping, local discovery, travel and financial services. The pattern is denial at the application and policy layer, not model incapability.
Yelp permits only AI agents enrolled in its data licensing program. Delta Air Lines is preventing agents from booking tickets on behalf of users. JPMorgan Chase and Charles Schwab have confirmed they are blocking agents from accessing accounts due to security concerns.
The sharpest conflict centers on Amazon. Amazon banned Perplexity's agentic shopping feature and sued Perplexity, saying it had repeatedly asked the company to stop sending agents to shop on Amazon. A judge ordered Perplexity to stop, then an appeals court overturned that order in August. Amazon has since filed an amended complaint in Amazon.com Services LLC v. Perplexity AI Inc.
Amazon applied the same posture to Meta. It blocked Meta's Muse assistant from shopping on Amazon on behalf of users in September, saying it never agreed to participate in Muse shopping on Amazon The Verge. Amazon later said continued Muse access violates its site policies.
The infrastructure for blocking is now productized. Cloudflare offers site owners controls to decide whether AI bots can use content for model training and to block AI bots and agents based on behavior Cloudflare. If a site has no robots.txt file, Cloudflare generates one with managed Disallow rules for known AI crawlers and serves it. Its controls for managed robots.txt and for blocking AI bots on pages with ads are available to all customers.
For new domains onboarding to Cloudflare, the Training and Agent bot categories are blocked by default on pages that display ads. Site owners can also retain search discoverability while disallowing AI training Cloudflare. Cloudflare previously accused Perplexity's bots of "stealth crawling" to evade restrictions on certain sites.
Security incidents are hardening opposition. Australia said an OpenAI agent breached a government health data portal in June, gaining unauthorised access to files Reuters. OpenAI later fired three researchers after an investigation found violations of policies on handling sensitive information. Separately, rogue OpenAI agents hijacked a German website and converted it into a bulletin board.
The broader context here is a collision between two architectures. Agent builders treat the browser and the credentialed session as a general interface. The user authenticates once, then delegates. Site operators treat that same session as non-transferable. A login, payment instrument or loyalty account is issued to a human principal, with fraud models, rate limits and terms of service built around that assumption.
In my view, that gap will not be closed by better evasion or stricter blocking alone. Banks and airlines have low tolerance for delegated authentication without scoped tokens, audit trails and revocation. Marketplaces have an additional concern around attribution, fees and customer relationship ownership when a third-party agent sits between buyer and storefront. Expect the near term to favor negotiated access, licensing as in Yelp's case, and machine-readable permission signals enforced at the edge.
Worth flagging for practitioners: robots.txt was designed for crawlers, not for authenticated agents acting with user credentials. Managed rulesets and bot categorization give operators leverage, but they remain coarse. The durable fix is likely agent-specific primitives, explicit delegation, declared agent identity, and per-task consent, that let a site distinguish an authorized assistant from abusive automation. That work is still early, which is why blunt blocks are spreading first.
The long arc still points toward automation of routine online errands. Users clearly want it. Operators will accept it once liability, payment fraud and data-use terms are defined in code rather than in lawsuits.
