Good Bots, Bad Bots: Who Owns Your Front Door?

Picture a shop on a busy street where more than half of the people walking through the door aren't shoppers. Some are personal shoppers sent by real customers, each carrying a list. The rest are there to shoplift or to copy every price tag for a competitor.

Every company with a website now runs that shop. It has to tell the personal shopper from the thief at the door, then hand the personal shopper accurate, current facts at the counter, while still talking to the humans who make the final call. Those two jobs belong together. Turn a personal shopper away and your product drops out of the running, and the purchase goes to a competitor.

Key figures (October 2026)

• Bots made up 53 percent of web traffic in 2025, and 40 percent was bad bots (Imperva 2026 Bad Bot Report).
• Traffic from AI agents and agentic browsers grew 7,851 percent year over year in 2025 (HUMAN).
• Daily AI agent requests on Cloudflare's network grew more than 1,700 percent in a year (Cloudflare, September 2026).

The customer stopped coming in person

The iPhone moved the customer to a new screen. Agents remove the screen entirely: the customer describes what they want, and software goes to the shop on their behalf.

That shift has already reached the checkout. OpenAI launched Instant Checkout in ChatGPT in September 2025, and Google followed in January 2026 with the Universal Commerce Protocol. In my piece on the carriage trade I called the survival rule the Studebaker Test: you survive a shift like this to the extent you own something the newcomer still has to pass through. (Also read: Is AI Doing to SaaS What the Car Did to Carriage Makers?)

When the shop locks its door

Amazon is the clearest test of what happens when a shop locks out the personal shoppers. Its robots.txt disallows the shopping and browsing agents of OpenAI, Anthropic, Perplexity and Google. In November 2025 it sued Perplexity over its Comet shopping agent, won a preliminary injunction in March, and then lost it on appeal in August, when the Ninth Circuit found it was the user, not Perplexity, who accessed Amazon.

The personal shopper doesn't go home, though. On October 28, James Douma posted: "Amazon locked out my Grok Bot. So it just bought the same thing from a different site (at a lower price)." The agent didn't quit. It finished the job somewhere else, and the shop never saw the sale it lost.

Amazon is big enough to steer shoppers toward its own AI tools. For most companies, blocking a good bot doesn't protect anything. It takes the product out of the consideration set, and the recommendation goes next door. When next door is a marketplace or an affiliate, the brand can end up paying that site a commission on a sale it could have made directly, and loses the customer's transaction data too.

What the personal shopper wants

Once inside, the personal shopper heads straight for the shelves. In HUMAN's April 2026 data, nearly 70 percent of agentic activity hit product and search pages, and about 3 percent reached checkout and payment (HUMAN).

Agents come for price, stock, offers and eligibility. A randomized study of AI shopping agents found they penalize "sponsored" tags, reward platform endorsements, and weigh price, ratings and reviews differently from one model to the next. A stale price or wrong stock count simply sends the agent elsewhere. Someone still approves the purchase, so the shop serves facts to the agent and messages to the human.

Do personalized offers change the agent's mind?

Here the evidence runs out. The Universal Commerce Protocol's draft loyalty extension already lets a shop show an agent the buyer's member price, tier and rewards before checkout. But no published data yet shows whether a personalized offer served to an agent changes what it buys.

The logic still points one way. An agent carries its customer's preferences and, increasingly, their identity, so a member price or a real-time offer that fits that customer is just another fact on the shelf, and facts are what agents compare. A generic promotion looks like the sponsored tag the study saw agents discount. Treat relevant, current offers to verified agents as a bet worth measuring, not a proven lever.

Sorting good bots from bad

All of that depends on the door. If the counter hands real-time prices and member offers to anyone who asks, it also hands them to scrapers, and HUMAN puts the median share of traffic attempting a scraping attack near 20 percent.

So the first job is sorting every visitor into keep or block. Verified agents acting for a customer get served with structured, current data. Anything unverified that behaves like fraud gets challenged or blocked. The industry needs a shared way to make that call, and one is forming: the IETF's Web Bot Auth working group is standardizing cryptographic signatures for bots and agents, Cloudflare already verifies signed requests from OpenAI, Google and AWS, and Visa's Trusted Agent Protocol builds on the same approach. Companies should accept signed agents now, publish product data agents can read, and count agent-driven sales separately from human ones; otherwise the shop can't trust its own numbers, the Proxy Collapse. (Also read: The Proxy Collapse: When Web Traffic Stops Being Human)

Nobody owns the front door

The hardest part of that sorting isn't technical. In most enterprises, analytics counts the footfall, marketing dresses the window, security guards the door and commerce runs the counter. I call this the Unowned Door: the decision about which software visitors to trust and what to hand them is spread across teams that are each measured on something else.

That makes it an incentive problem. A paid media leader paid on visits has every reason to read agent traffic as a win, and a security leader paid on blocked attacks has every reason to keep the door tight. Each is rational alone, but one counts every bot as a win and the other treats every bot as a threat, so nobody sorts the good from the bad. Great product data counts for nothing if the operating model can't serve it, the Zero Factor at work. (Also read: Why Owning the Moat Isn't Enough for AI-Era SaaS)

Only the C-suite can redraw those scorecards. Give one executive both jobs: deciding which bots get in, and making sure the good ones get accurate, current answers. Right now, nobody owns that call.

Key Takeaways

• Agents remove the screen: software now researches, compares and increasingly buys on a customer's behalf.
• Blocking a good bot takes your product out of the consideration set, and the recommendation goes to a competitor.
• Sort every visitor: serve verified agents with current, structured data and block what behaves like fraud.
• No published data yet shows whether personalized offers to agents change what they buy; treat it as a bet worth measuring.
• The Unowned Door: analytics, marketing, security and commerce each hold a piece, so only the C-suite can give the door one owner.

FAQ

How can a company tell good bots from bad bots?
Increasingly through verified identity rather than self-reported labels. The IETF's Web Bot Auth working group is standardizing cryptographic signatures for bots and agents, and Cloudflare already verifies signed requests from operators including OpenAI, Google and AWS.

What happens when a site blocks AI shopping agents?
The agent doesn't stop shopping. Amazon blocks major AI agents in its robots.txt, and users have reported their agents buying the same item from another retailer, which takes the blocking site out of the consideration set.

Do personalized offers work on AI agents?
No published data yet answers that. Protocols such as the Universal Commerce Protocol's draft loyalty extension let shops show agents member prices and rewards, and the logic favors relevant, current offers, but the effect is still unmeasured.

What is the Unowned Door?
The Unowned Door is the gap that opens when the decision about which software visitors to trust, and what to serve them, is split across analytics, marketing, security and commerce, with no single owner measured on the whole.

---

---

About the Author

Martin Goetzinger has spent his career in enterprise software sales, helping large organizations such as Apple, Microsoft, and Verizon connect data, insight, and action. His work focuses on transforming how businesses measure success and create customer value through technology.

Outside the enterprise world, he writes about the five forces he believes are reshaping everything: AI, blockchain, energy, personalized health, and robotics. Not from a purely technical lens, but from a human one as to how these technologies will redefine work, wealth, and well-being.

He is based in the U.S. and publishes at www.MartinGoetzinger.com.

Disclaimer

The views expressed in this article are the personal opinions of the author and are provided for informational and educational purposes only. Nothing in this article constitutes investment advice, financial advice, legal advice, or any other form of professional advice. Do not make investment or financial decisions based on the content of this article. Always consult a qualified professional before making decisions that affect your finances, business, or livelihood.