Applebot is Apple's web crawler. Apple says the pages it collects power search in Siri, Spotlight and Safari, and may also be used to train Apple's foundation models for its generative AI features. Applebot-Extended is not a second crawler. It's a robots.txt token that only controls whether Apple may use what Applebot already fetched for that training. Block Applebot-Extended and Applebot keeps crawling you for search.
On getintel.ai, Applebot made 623 IP-verified requests in the 30 days to 15 October 2026. It came every day, but 61% of its month landed in one five-day sweep, 23 to 27 September. Applebot-Extended never appeared in the logs, which matches how Apple describes it.
Disclosure: GetIntel sells AI visibility tracking, including AI crawler logs. These numbers come from one site, ours, over one month, from DataFast's bot report. The daily counts are downloadable as a CSV. For every AI company's bots side by side, see our full AI crawler log.
In this article
- What is the difference between Applebot and Applebot-Extended?
- Applebot came every day, but mostly in one week
- Nearly a third of what Applebot read was markdown
- Applebot reads the homepage, unlike Amazonbot
- Applebot-Extended robots.txt rules
- The Googlebot fallback catches people out
- Should you block Applebot-Extended?
- Gaps in one month of Applebot logs
What is the difference between Applebot and Applebot-Extended?
Applebot crawls pages; Applebot-Extended is a robots.txt token that only controls whether Apple may use them for AI training.
| Token | Crawls pages? | What it controls | Seen on getintel.ai, 30 days |
|---|---|---|---|
Applebot | Yes | Fetching pages for Siri, Spotlight and Safari search, and for training unless opted out | 623 requests |
Applebot-Extended | No | Whether Apple may use Applebot's pages to train its models | 0 (it never sends requests) |
Apple's own wording is that Applebot-Extended "does not crawl webpages" and is "only used to determine how to use the data crawled by the Applebot user agent". So if you search your server logs for Applebot-Extended and find nothing, nothing is wrong. There's nothing to find.
Apple was the eighth-busiest AI company on our site, with 623 of the 17,599 verified bot requests (3.5%), just behind Anthropic's 779. Every one of those requests was Applebot.
Applebot came every day, but mostly in one week
61% of Applebot's 623 requests landed on 23 to 27 September 2026. On the other 25 days it averaged 10 a day, and it never missed one.
Daily Applebot requests, the September sweep highlighted. Chart by GetIntel from DataFast bot logs for getintel.ai, October 2026.
| Period | Applebot requests | Per day |
|---|---|---|
| 23 to 27 September (5 days) | 382 | 76 |
| Other 25 days | 241 | 10 |
The two biggest days were 27 September (144) and 24 September (112). Outside the sweep it never went above 23 a day, and twice it made a single request. But it never skipped a day, which puts it between the two patterns we saw elsewhere: GPTBot and ClaudeBot, which vanished for days between bursts, and Amazonbot, which crawled every day on a weekly rhythm.
The timing lines up with the other training crawlers. GPTBot's two bursts were 19 and 24 September, and ClaudeBot's biggest were 20 and 22 September (our ClaudeBot breakdown and our OpenAI bot breakdown have the days). Three companies' crawlers all swept the same site within nine days. That may be coincidence, or a reaction to the 20 old posts we deleted on 19 September. One site can't tell those apart.
Applebot read most pages only once: 256 of the pages it touched got a single request. Across the month it averaged about 1.7 requests per URL over 371 URLs, a light, mostly one-pass pattern. The logs can't tell us whether a given request fed search or training.
Nearly a third of what Applebot read was markdown
| Section | Share of Applebot requests |
|---|---|
| Docs | 52% |
| Blog | 20% |
| Other pages (features, glossary, comparisons) | 16% |
| Homepage | 10% |
| Tools | 2% |
| AI Software Index | 1% |
Half of Applebot's reading was our docs, and 191 of its requests (31%) were for the raw .md copies of our docs pages, the plain-text versions we publish for machines. That's the highest markdown share of any bot we logged, just ahead of GoogleOther (30%). GPTBot was at 25% and Amazonbot at 5%.
It also nearly ignored our AI Software Index, the 127 category pages that GPTBot and Amazonbot each spent about a fifth of their requests on. Applebot read the glossary instead: five of its top 12 pages were glossary definitions.
Applebot reads the homepage, unlike Amazonbot
Applebot asked for our homepage 60 times between 16 September and 15 October 2026, 10% of its month and more than any other single page. Amazonbot, with four times as many requests overall, asked for it 5 times.
| Bot | Homepage share of its requests |
|---|---|
| Applebot | 10% |
| meta-externalagent | 7% |
| GoogleOther | 2% |
| GPTBot | 1% |
| ClaudeBot | 1% |
| Amazonbot | 0.2% |
Among these crawlers, Applebot leaned on the homepage the most. Our reading: that fits its search job. Siri and Spotlight need to know what a company is, and a homepage is where that's stated. A crawler collecting only training text has less reason to keep coming back to it. For the same reason, the homepage is the page worth checking first if you want Siri to describe you correctly.
Applebot-Extended robots.txt rules
Use an Applebot-Extended group to opt out of training and an Applebot group to control crawling; the two are separate. To stay in Siri, Spotlight and Safari search but opt out of Apple's model training:
User-agent: Applebot-Extended
Disallow: /
To opt out of training for one section only:
User-agent: Applebot-Extended
Disallow: /docs/
To keep Applebot out completely, which also removes you from Apple's search features:
User-agent: Applebot
Disallow: /
Apple adds a few details worth knowing:
- Crawl-delay is ignored. Applebot does not follow it. Apple says it slows down by itself when a site gets slow or starts returning errors.
- Meta tags work. Applebot supports
noindex,nosnippet,nofollow,noneandall, plus theX-Robots-Tagheader. Withnosnippet, Apple says it won't generate a description or web answer for the page. - You can verify it. Real Applebot traffic resolves by reverse DNS to
applebot.apple.com, and Apple publishes its IP ranges.
The Googlebot fallback catches people out
Apple says that if your robots.txt doesn't mention Applebot but does mention Googlebot, Applebot follows the Googlebot rules. So a Googlebot block also blocks Applebot unless Applebot has its own group.
That matters in both directions. If you wrote a Disallow for Googlebot to keep staging pages out of Google, Applebot obeys it too. If you gave Googlebot an Allow for something your wildcard group blocks, Applebot gets that too. A robots.txt written only with Google in mind will also decide Applebot's behaviour.
The fix is a named User-agent: Applebot group. Once Applebot has its own group, the Googlebot rules no longer apply to it.
Should you block Applebot-Extended?
Usually not, for a software company that wants AI assistants to know what it does. Blocking Applebot-Extended only stops Apple training its own models on your pages; it doesn't remove you from Siri, Spotlight or Safari, and it doesn't affect ChatGPT, Perplexity or Google. Plain Applebot is the one that keeps you in Apple's search on every iPhone, and that is the one you should almost never block. The broader case for and against blocking training bots is in our full crawler log.
For how other AI companies split search and training across their bots, see our OpenAI bot breakdown. To see Applebot and every other AI crawler on your own site, by page and by day, GetIntel's AI traffic view logs verified requests on the Pro and Growth plans.
Gaps in one month of Applebot logs
- One docs-heavy B2B site, one month. A news site or a store, with more Siri and Safari searches pointing at it, may see much more Applebot traffic. One sweep in 30 days is what we saw, not a schedule Apple has published.
- No view of training use. Applebot-Extended never shows up in logs, so no log can tell you whether Apple trained on your pages. Only your robots.txt decides that.