Does choosing googlebot stop other crawlers from indexing the page?

No. The googlebot and googlebot-news tokens only scope the directive to Google's text or news pipelines. Other crawlers that respect the meta robots specification still see any unqualified meta element with name="robots" or may ignore the rule entirely; they do not read a googlebot-only meta tag.

Why does the generator refuse to emit nosnippet with a numeric max-snippet?

Because the more restrictive nosnippet would override the numeric limit. A page that asks for nosnippet cannot also ask for a 20-character snippet — Google applies the more restrictive rule, so shipping both creates misleading markup. The generator refuses the combination rather than copying it.

Why is noarchive not offered in the generator?

Google lists noarchive as historical and unused because its cached-link feature no longer exists. Including a familiar but ignored checkbox would create false confidence, so the interface omits it.

When should I use X-Robots-Tag instead of the HTML meta element?

Use the HTML meta element on HTML pages where you control markup. Use X-Robots-Tag for non-HTML resources such as PDFs, images, or video, and for any deployment where response headers are the controlled channel — for example when a reverse proxy or CDN writes directives for you.

Robots Meta Tag Names: Pick the Right Crawler Target

A robot meta tag is an HTML meta element whose name attribute is set to one of three currently documented tokens — robots, googlebot or googlebot-news — and that single token decides which crawler the directives that follow apply to. The token is the "name" the reader is choosing when they reach for a robots generator, because every other directive (noindex, nofollow, nosnippet, max-snippet, max-video-preview, max-image-preview, notranslate, noimageindex, indexifembedded) hangs off it. Picking the right token up front is the difference between targeting general crawlers, narrowing a rule to Google's text results, or scoping it to Google News only, and the Meta Robots Generator builds both the HTML element and the equivalent X-Robots-Tag HTTP header from a single validated choice set.

names for robots generator — Robots Meta Tag Names: Pick the Right Crawler Target

What the name attribute actually controls in a robots meta tag

In HTML, a meta element carries its meaning through two attributes: name (or http-equiv) and content. For crawler directives, the name is always one of three values that Google documents today, and the content string holds the directives themselves. According to the WHATWG HTML specification, the name attribute uses tokens such as robots, googlebot or googlebot-news so that a page can address one specific crawler scope without writing a separate element for each. The chosen token determines whether the directives apply to all compliant crawlers, to Google's text-result pipeline only, or to Google's news pipeline only.

The token is also what makes the rule readable by the targeted crawler when it processes the page. A page that says <meta name="robots" content="noindex"> speaks to every crawler that respects the meta robots specification, while <meta name="googlebot" content="noindex"> narrows that rule to Googlebot's web-search pipeline. A news-only rule uses googlebot-news as the token, which Google documents for publishers who want to opt specific URLs out of Google News without affecting their standard search appearance.

The three crawler targets and their HTTP header equivalents

Each meta name has a matching X-Robots-Tag header form, and choosing the right name in the generator means choosing the right header value at the same time. The general target emits a meta element whose name is robots and an unqualified X-Robots-Tag header. The two Google-specific targets use googlebot or googlebot-news as the meta name and prefix that user-agent token inside the HTTP header value, separated by a colon. Google's robots meta tag documentation lists these two user-agent tokens — googlebot and googlebot-news — for text and news results respectively. Other crawler names and vendor-specific behavior are outside the focused interface of the generator, so choosing googlebot is not a way to address Bing or DuckDuckGo; those engines do not honor a meta tag named googlebot.

When the rule needs to apply to every compliant crawler, robots is the correct token; when the rule needs to apply to Google's web-search pipeline only, googlebot is the correct token; when the rule needs to apply to Google News only, googlebot-news is the correct token. Picking the wrong token is a common silent failure because the page will still load and the markup will still validate, but the directive simply will not be read by the intended crawler.

Crawler scope	meta name attribute	X-Robots-Tag header value	Typical use
General crawlers	robots	X-Robots-Tag: noindex (unqualified)	Site-wide or page-level rules that apply to every compliant crawler
Google text results	googlebot	X-Robots-Tag: googlebot: noindex	Narrow a directive to Google's web-search pipeline only
Google News	googlebot-news	X-Robots-Tag: googlebot-news: noindex	Opt specific URLs in or out of Google News without changing standard search appearance

How to build a validated robots meta tag

Building a robots meta tag with the generator is a three-step task: pick the crawler scope, generate both HTML and HTTP syntax, then deploy while allowing crawl access.

Choose the crawler target and only the indexing, following and preview restrictions that match the resource policy. In the generator, pick robots for general crawlers, googlebot for Google's text results, or googlebot-news for Google News. Tick only the directives that match the page's actual intent — noindex to keep the URL out of results, nofollow to ask crawlers not to follow its links, nosnippet to suppress text and video preview snippets, max-snippet with a numeric character limit, max-video-preview in seconds, max-image-preview set to none, standard or large, notranslate, noimageindex, and indexifembedded only alongside noindex. Leave every other box empty. The generator refuses to emit both nosnippet and a numeric max-snippet because the more restrictive rule would make the numeric choice misleading, and it refuses indexifembedded without noindex because Google states that combination has no effect on its own.
Generate and copy either the HTML meta element for a page or the X-Robots-Tag header for a server response. The generator emits both forms from the same set of choices: the HTML meta element goes into the page <head> where you control markup, and the X-Robots-Tag line is delivered by the web server, application or CDN as part of the HTTP response. Use the HTML element on HTML pages you edit directly, and use the header for PDFs, images, video files or any deployment where response headers are the controlled channel. The max-snippet field accepts -1 or a non-negative whole number, where zero is equivalent to no snippet and -1 means Google can choose an effective length. Max-video-preview uses seconds, with zero permitting at most a static image and -1 meaning no duration limit. Max-image-preview accepts none, standard or large, where large permits a larger preview up to viewport width. Any value outside what Google documents is refused rather than copied into a rule Google would ignore.
Deploy while allowing crawl access, then inspect the public source and headers for duplicate or conflicting rules until recrawling occurs. A crawler must be allowed to fetch the resource before it can read either form. If robots.txt disallows crawling of the URL, Google may never see the rule, so the URL can stay known from external links without the page content being processed. To remove an accessible page from results, permit crawling and serve the directive until the search engine processes it. After deployment, fetch the public URL the way the intended user agent would, inspect the delivered HTML and the response headers, and confirm there is no contradictory declaration from a CMS, SEO plugin, reverse proxy or CDN.

Matching HTTP syntax with the X-Robots-Tag header

X-Robots-Tag is the response-header counterpart to the HTML meta element. The generator serializes the same directives in both forms, so the choice of name attribute also fixes the header value. A general crawler choice yields an unqualified header value such as X-Robots-Tag: noindex, while a googlebot choice yields X-Robots-Tag: googlebot: noindex, and a googlebot-news choice yields X-Robots-Tag: googlebot-news: noindex. The header is configured at the server, application or CDN layer, and pasting the displayed header text into HTML page content does nothing at all — it must travel as a real HTTP response header.

The header form is the right channel for non-HTML resources such as PDFs, images and video files, and for any deployment where headers are easier to manage than markup. It is also the right channel when a reverse proxy or CDN writes directives on your behalf, because inspecting the delivered header is the only way to see the final value. When resource rules can be hidden inside a CMS plugin or a server config, always inspect the actual response the public receives rather than only the editor setting. The displayed header text in the generator is exactly the line you need to configure at the server level; do not paste it into a page and expect crawlers to honor it.

Deploying, verifying, and handling conflicting rules

After deployment, fetch the public URL the way the targeted user agent can, and inspect the served source or response headers. Look for duplicate or contradictory declarations: a CMS, an SEO plugin, a reverse proxy and a CDN can each add their own robots meta tag or X-Robots-Tag header, so the editor view is not the only source of truth. Google says the more restrictive rule applies when conflicting rules exist, which means a noindex from one channel and an index from another still leaves the page noindexed. Inventory every existing directive before adding a new tag, because a single misplaced rule can override the rest.

Conflict rules baked into the generator itself prevent the worst contradictions before the tag ships. It refuses to combine nosnippet with a numeric max-snippet, because the more restrictive nosnippet would make the numeric choice misleading. Indexifembedded is gated on noindex because Google states it only has an effect when accompanied by noindex. The generator also rejects non-integer numeric values, values below the documented minimum, and max-snippet or max-video-preview inputs that fall outside what Google documents — it surfaces those rejections rather than copying a value Google would ignore.

Generation is entirely local, and the output can be copied without contacting a search service, which fits the workflow described in Create Valid Robots Meta Tags Without an API for publishers who do not want their URLs sent to a third party. The generator produces exact syntax, but it cannot guarantee deindexing, recrawl timing, or a particular search-result presentation once the rule is live — search engines process changes on their own crawl schedule, and other engines may interpret Google-specific directives differently.