What is Google-Extended and should I block it?
Google-Extended is a robots.txt product token that controls whether content crawled by Googlebot is used to train Google's foundation models and appear in AI Overviews. Blocking Google-Extended does not affect your standard Google Search rankings or indexing; it only opts you out of those AI uses. Whether to allow it is a training-data and AI-summary decision, separate from search visibility.
Google-Extended vs Googlebot: Crucial Differences
Google introduced the Google-Extended standalone user-agent token to give web publishers separate control over Gemini AI model training and Vertex AI grounding without forfeiting Google Search indexation.
Blocking Google-Extended does NOT drop your organic rankings on Google.com, but it will exclude your brand from being cited in Gemini conversational summaries and AI Overviews.
| User Agent | Primary Role | Google Search Ranking Effect |
|---|---|---|
| Googlebot | Core Web Search & Indexing | Blocking destroys organic Google rankings |
| Google-Extended | Gemini & Vertex AI Training | Zero effect on organic search rank |
| Googlebot-Image | Google Image Search | Affects Google Images only |
Best practices for SaaS and Technical Websites
For B2B software and documentation sites, allowing Google-Extended ensures that when users ask Gemini for software recommendations, technical tutorials, or API solutions, your domain is retrieved as an authoritative citation source.
- 1Keep Googlebot and Google-Extended allowed for all public marketing and guide routes.
- 2Disallow both bots on private authenticated application paths (/dashboard/, /auth/).
- 3Verify your robots.txt syntax periodically to ensure no unintended wildcard Disallow rules exist.
Frequently Asked Questions
Does Google Search Console report Google-Extended crawl statistics?
Google Search Console logs primary Googlebot crawl activity; Google-Extended requests appear in your raw server access logs.
Can I allow Googlebot but block Google-Extended?
Yes. In robots.txt, set 'User-agent: Google-Extended' with 'Disallow: /' while keeping 'User-agent: Googlebot' set to 'Allow: /'.
Does Google-Extended crawl images and PDFs?
Google-Extended crawls all publicly reachable document formats allowed by your robots.txt configuration.
Why should SaaS companies allow Google-Extended?
Allowing Google-Extended ensures your product features and documentation are accurately cited in Google Gemini AI answers.