Guide

Amazonbot: What Amazon's Crawler Reads, When, and How to Block It

Amazonbot made 2,549 verified requests to our site in 30 days and crawled twice as hard every weekend. What it reads, and the robots.txt rules.

Tarang AgarwalOctober 17, 20266 min read
Cover for a 30-day study of Amazonbot requests to getintel.ai, with robots.txt rules for Amazon's crawlers

Amazonbot, Amazon's web crawler, crawled getintel.ai about twice as hard every weekend: 142 requests a day on Saturdays and Sundays against 64 on weekdays, across 2,549 IP-verified requests in the 30 days to 15 October 2026. The eight weekend days carried 44% of its month, and it re-read most pages about once a week.

Amazon says Amazonbot is used to improve its products and services and may be used to train Amazon AI models. On our site it made more requests than Perplexity, Microsoft or Meta, and it read more of our pages than any other bot.

Disclosure: GetIntel sells AI visibility tracking, including AI crawler logs. These numbers come from one site, ours, over one month, from DataFast's bot report. The daily counts are downloadable as a CSV. For every AI company's bots side by side, see our full AI crawler log.

In this article

What is Amazonbot?

Amazonbot is the Amazon crawler that may feed AI training; Amazon says it improves its products and services and may train Amazon AI models. Amazon documents two more agents, one for search and one for live fetches, each with its own robots.txt token:

AgentWhat Amazon says it doesUsed for AI training?Requests to getintel.ai, 30 days
AmazonbotImproves Amazon products and servicesMay be2,549
Amzn-SearchBotSearch in Amazon products, such as AlexaAmazon says no0
Amzn-UserFetches live pages for a user's request, such as an Alexa questionAmazon says no0

All 2,549 Amazon requests we logged were Amazonbot. That made Amazon the third-busiest AI company on our site, behind OpenAI (5,408) and Google (2,665), with 14% of the 17,599 verified bot requests in the window.

Amazonbot crawls twice as hard at weekends

Every Saturday and Sunday between 16 September and 15 October 2026 went above 100 Amazonbot requests, against a weekday average of 64.

Daily IP-verified Amazonbot requests to getintel.ai, 16 September to 15 October 2026: every Saturday and Sunday is a peak of 119 to 184 requests, while weekdays sit mostly between 49 and 86
Daily IP-verified Amazonbot requests to getintel.ai, 16 September to 15 October 2026: every Saturday and Sunday is a peak of 119 to 184 requests, while weekdays sit mostly between 49 and 86

Daily Amazonbot requests, weekends highlighted. Chart by GetIntel from DataFast bot logs for getintel.ai, October 2026.

WeekendSaturdaySunday
19 to 20 September144184
26 to 27 September157135
3 to 4 October128130
10 to 11 October137119

All eight weekend days went above 100 requests. Only one weekday did, Monday 21 September with 127, straight after the biggest weekend. The quietest day all month was 49, so unlike ClaudeBot or GPTBot, Amazonbot never went missing.

Our dashboard counts days in IST, the site's own timezone. To rule out a timezone artefact we pulled the same month in UTC: the pattern holds, and a Saturday or Sunday was the busiest day of all four weeks (140 to 188 requests).

We don't know why Amazon schedules it this way, and the logs can't say. For a site owner, a weekend jump from Amazon's IP ranges is normal.

It re-reads most pages about once a week

Amazonbot touched 722 distinct URLs at about 3.5 requests each. That average hides how regular it is. Of the pages it read, 400 were read exactly four times in the month, and 115 exactly five, so about 71% of its pages came round roughly once a week.

Amazonbot re-reads like a search indexer, although DataFast labels it training and Amazon says it may train models. The other training crawlers read each page about once: GPTBot made 504 requests across 433 URLs, ClaudeBot 607 across 597.

The practical upshot: if you change a page, Amazonbot will probably have the new version within about a week. GPTBot and ClaudeBot, which read most pages once in the month, may not return soon.

What Amazonbot reads, and the page it skips

SectionShare of Amazonbot requests
Blog30%
Docs26%
AI Software Index21%
Other pages (features, comparisons, solutions)21%
Tools2%
Homepage0.2%

The homepage line is the odd one. Amazonbot asked for it 5 times in 30 days. ChatGPT-User fetched it 838 times and OAI-SearchBot 363 times in the same window. Amazonbot spreads itself across deep pages instead: no single URL got more than 8 of its requests.

It also read 135 of the raw .md copies of our docs pages, the plain-text versions we publish for machines. GPTBot did the same (118). Both bots that read our docs heavily for training also requested the markdown copies.

Amzn-SearchBot and Amzn-User never showed up

Amazon's two newer agents didn't appear once in our verified logs. Amzn-SearchBot is the one Amazon says makes content eligible for search features such as Alexa, and Amzn-User fetches a page when an Alexa user's question needs current information.

So on our site, every Amazon request was the crawler that may feed training. None was Amazon reading a page to answer someone. Compare OpenAI, where 29% of requests were ChatGPT-User answering a person, and Anthropic, where Claude-User came every day (our ClaudeBot breakdown has that split).

That may just reflect how few Alexa users ask about AI visibility software. A consumer brand would likely see a different picture.

Amazonbot robots.txt rules and the noarchive tag

To block Amazonbot from the whole site while leaving Amazon's search and user agents alone:

User-agent: Amazonbot
Disallow: /

To block it from one section, say your docs:

User-agent: Amazonbot
Disallow: /docs/

Amazon gives you a finer tool too. It says its agents respect the noarchive robots meta tag, which it defines as "do not use the page for model training". noindex and none mean do not index the page. So a single page can opt out of training without changing robots.txt:

<meta name="robots" content="noarchive">

Things to know before you rely on any of it:

  • Crawl-delay doesn't work. Amazon says Amazonbot does not support it. If the weekend peaks strain your server, rate-limit at the firewall instead.
  • Changes are slow. Amazon says each agent setting is independent and changes can take about 24 hours, and it may use a cached copy of robots.txt up to 30 days old.
  • The other two agents differ. Amzn-SearchBot follows the rules you give other search bots if robots.txt doesn't name it, and Amazon says Amzn-User, because users start its fetches, may not follow all robots.txt directives.

The firewall is also where a default bot-blocking rule can shut Amazon out entirely, whatever robots.txt says; our access checklist test checks for that first.

Should you block Amazonbot?

No, not for most B2B software sites: blocking Amazonbot does not affect ChatGPT, Perplexity or Google answers. It only decides what Amazon can learn from your pages and use in its own products. A company that wants AI assistants to know what it does gains nothing by hiding from one of the bigger ones.

Two Amazon-specific options matter more than a full block. noarchive opts single pages out of training, which suits paid or licensed content.

And because Amazonbot ignores Crawl-delay, firewall rate-limiting is the fix if the weekend peaks hurt a small server. Blocking Amazonbot while leaving Amzn-SearchBot and Amzn-User open keeps you eligible for Alexa answers and opts out of training. For how OpenAI splits the same jobs across its own bots, see GPTBot vs OAI-SearchBot vs ChatGPT-User.

To see Amazonbot and every other AI crawler on your own site, by page and by day, GetIntel's AI traffic view logs verified requests on the Pro and Growth plans.

What Amazonbot's logs leave out

  • A B2B site with few Alexa-type questions. A consumer brand or a store may see Amzn-User and Amzn-SearchBot where we saw none.
  • Four weekends is a pattern, not a promise. Amazon can change its schedule any time, and one month can't show a seasonal cycle.
  • Reads are not use. A request tells you Amazonbot fetched the page. It doesn't tell you whether Amazon trained on it, indexed it or threw it away.
Tags:AI crawlersAmazonbotAmazonrobots.txtresearch

Written by Tarang Agarwal

Tarang Agarwal is the founder of GetIntel. He writes about AI visibility, generative engine optimization, and growth for SaaS founders, marketing teams, and the agencies who run AI-search visibility as a service line.

FAQ

Frequently asked questions

Amazonbot is Amazon's web crawler. Amazon says it is used to improve its products and services and may be used to train Amazon AI models. It is separate from Amzn-SearchBot, which supports search features such as Alexa, and Amzn-User, which fetches pages live for a user's request.

On getintel.ai Amazonbot came every day for 30 days, with between 49 and 184 requests a day. It crawled about twice as hard at weekends, averaging 142 requests on Saturdays and Sundays against 64 on weekdays, and it re-read most pages about once a week (about 71% of its pages were read four or five times). Data: 2,549 IP-verified requests, 16 September to 15 October 2026.

Add a robots.txt group with User-agent: Amazonbot and Disallow: /. That leaves Amzn-SearchBot and Amzn-User alone, since each Amazon agent has its own token. Amazon says changes can take about 24 hours and it may use a cached robots.txt up to 30 days old.

No. Amazon says Amazonbot does not support the crawl-delay directive. If its traffic strains your server, rate-limit Amazon's requests at your firewall or CDN instead of relying on robots.txt.

Yes. Amazon says its agents respect the noarchive robots meta tag, which it defines as do not use the page for model training. Add <meta name="robots" content="noarchive"> to that page and leave robots.txt unchanged.

Put this into action

Daily topic scores across ChatGPT, Perplexity, Gemini, and Google AI Overviews, plus a ranked task list your coding agent can work through. Built for founders, teams, and agencies.