seo · November 5, 2026
Google Revises Indexing Guidance as AI Search Measurement Remains Incomplete
What the sources reported
Google Clarifies Why Crawled Pages Can Remain Unindexed
Google revised the help documentation for its page indexing report, specifically the section covering “Crawled - currently not indexed.” The update now says Google does not index all pages it crawls, making the status easier to interpret but not automatically actionable.
For SEO practitioners, the distinction matters because a crawled URL is not necessarily an indexed URL. A site owner should use the report as a starting point for diagnosis, then examine the affected page against the site’s quality, discoverability and relevance signals. The documentation change does not itself establish a ranking update or a guaranteed path to inclusion.
A video overview of Search Console and indexing changes also highlighted “Crawled - currently not indexed” as a common report message, reinforcing the visibility of the issue among practitioners. The practical response is to review affected templates and pages rather than treating the entire crawled-but-unindexed population as a uniform indexing defect.
AI Referral Data Remains an Open Measurement Question
A Google engineer indicated in a LinkedIn reply that click data and queries may not be added to Search Console’s AI performance report. Google subsequently said it had nothing to announce, so the response should not be treated as a confirmed product change.
The statement nevertheless exposes a practical gap for teams measuring visibility across Google’s AI surfaces. Referral analytics may remain necessary for understanding visits from AI answers and assistants, especially when platform reporting does not yet provide equivalent click and query data.
Teams should preserve referral parameters, separate AI-origin sessions from other channels and document changes in tracking before drawing conclusions from short reporting windows. This is especially relevant for publishers comparing direct search, AI referrals and other discovery sources on the same site.
AI Overviews Click Estimates Need Careful Interpretation
One SEO study estimated a 0.9% click-through rate on AI impressions after opting a site out of Google’s AI features. The evidence describes this as a small test, so the figure is not a broad benchmark for AI Overviews.
Even so, the experiment highlights why controlled testing can complement broader visibility reporting. A practitioner can define an AI-focused landing page, record its baseline search performance, change the relevant exposure and then compare qualified visits rather than relying on a single dashboard label.
The AI Overviews measurement discussion provides useful context for treating AI visibility as a distinct discipline. The reported 0.9% should remain tied to its test conditions and should not be applied mechanically to unrelated sites.
ChatGPT Advertising Revenue Requires a Like-for-Like Test
A new working paper estimated revenue per visit at $0.89 for US ChatGPT ad clicks and $2.45 for paid Google search. The evidence explicitly attaches big caveats to the comparison, making the figures more useful as a testing hypothesis than as settled valuation guidance.
Advertisers should compare the two channels using equivalent measures such as revenue per eligible visit, assisted conversion and customer quality. The comparison should also account for differences in audience intent, placement and attribution rather than assuming that the reported per-visit values apply to every campaign.
The result reinforces the need for campaign-specific reporting. Teams evaluating an AI-assistant advertising tool should build a repeatable comparison worksheet, retain campaign-level data and avoid folding experimental revenue estimates into forecasts as if they were observed performance.
Robots.txt May Gain a Shared AI Training Signal in 2027
Google plans to support an IETF-based robots.txt signal that allows sites to opt out of AI training. The proposed control is scheduled for 2027, according to the evidence.
Site owners should monitor the standards and platform documentation before changing production directives, because the signal’s final syntax and implementation remain subject to the referenced plan. The larger implication is operational: teams managing crawler permissions may need separate controls for search indexing, AI training and other machine uses rather than assuming every crawler follows the same rules.
A useful implementation plan would identify existing robots.txt controls, document which automated agents are allowed and create a review process for adopting new standards. The AI visibility and evidence-chain analysis offers a framework for connecting visibility outcomes to the technical and editorial decisions that may influence them.
Audit Reporting, Tracking and AI Controls in the Next Work Cycle
The immediate actions are concrete. Start with the page indexing report’s “Crawled - currently not indexed” section, separate genuinely unindexed pages from pages that Google simply chooses not to index, and document any remediation at the template level.
Next, verify that referral tracking can preserve meaningful source data for AI-origin visits and that campaign reporting supports a fair comparison between paid Google search and ChatGPT advertising. The Gemini referral measurement analysis can help frame how tagged referrals can make otherwise opaque AI traffic measurable.
Finally, record the current state of robots.txt permissions and watch for the planned 2027 shared AI training control. Do not rewrite directives around an unconfirmed syntax. Use tools that can audit reporting changes, preserve tagged links and compare performance without treating one small experiment as an industry-wide benchmark.
What this means for tooling
- indexing report auditor
- referral-tag analyzer
- AI visibility tracker
- robots.txt policy generator
- paid-search revenue comparator
Tools that already cover this
Decision room queued — the team review of this signal has not started yet.
AI analysis by Lizely. Grounded in linked public evidence. Participants are fictional editorial roles, not real people or human authors.