Robots Meta Tag Generator
Controls that hide a page from search results — and the ones that only change how it is displayed.
Drops the page from results after this date. Useful for time-limited offers and events.
Generated tag
Equivalent HTTP header (for PDFs and other non-HTML files):
What this does in plain English
What the robots Meta Tag Generator does
The robots meta tag controls what a search engine does with a page once it has crawled it. That
distinction matters: it is not a way to stop crawling, it is a set of instructions applied
afterwards. Two directives do most of the work. noindex keeps the page out of results
entirely, and nofollow stops crawlers following the links on it.
The rest change how a page is presented rather than whether it appears. noarchive
suppresses the cached copy. nosnippet removes the description text, which almost always
costs clicks and is worth using only when the text itself is sensitive.
max-snippet, max-image-preview and max-video-preview set upper
bounds on how much Google may show; publishers often set max-image-preview:large to
encourage a bigger thumbnail. unavailable_after drops a page from results on a given
date, which suits an event listing or a limited offer.
One mistake causes more trouble than all the others: combining noindex with a
Disallow for the same URL in robots.txt. Blocking the URL means the crawler never
fetches the page, so it never sees the noindex, and the URL can remain indexed as a bare
link with no description. To remove a page you must let it be crawled. Get it out of the index first,
then block it later if you want to save crawl budget.
The tool also outputs the equivalent X-Robots-Tag HTTP header, which is how you apply
the same directives to files that have no HTML head — PDFs, images and downloads.
How to use it
- Tick the directives you want. In most cases that is
noindexalone, or nothing at all. - Set snippet and preview limits if you want to control how the result is displayed rather than whether it appears.
- Read the plain-English panel to confirm the tag does what you expect.
- Copy the meta tag into your
<head>, or use the HTTP header version for PDFs and other files.
A worked example
Three situations and the right tag for each:
| Situation | Tag |
|---|---|
| Thank-you page after a form submission | <meta name="robots" content="noindex, follow"> |
| Article where you want a large image thumbnail | <meta name="robots" content="max-image-preview:large, max-snippet:-1"> |
| Event page that expires | <meta name="robots" content="unavailable_after: 01 Nov 2026 00:00:00 GMT"> |
Note noindex, follow in the first row rather than noindex, nofollow. The
thank-you page should not be in search results, but it probably links to your main content and there
is no reason to cut those links off. Reaching for nofollow automatically is a habit
worth breaking.
Frequently asked questions
What is the difference between robots.txt and the robots meta tag?
robots.txt controls crawling — whether a bot may fetch the URL at all. The robots meta tag controls indexing — what happens after it has been fetched. They solve different problems, and using both on the same URL usually backfires.
Why is my noindex page still in Google?
Most often because the URL is also disallowed in robots.txt, so Googlebot cannot fetch the page and never sees the directive. Remove the block, let it be crawled, and the page will drop out on the next visit. Removal can take days to weeks either way.
Does nofollow on a page stop it passing any authority?
It stops crawlers following the links on that page, so those links pass no signals. That is a blunt instrument — it applies to every link including your own navigation. Use rel="nofollow" on individual links instead when only some should be excluded.
Can I use these directives on a PDF?
Not as a meta tag, since a PDF has no HTML head. Use the X-Robots-Tag HTTP header instead, which this tool also generates. Configure it in your server or CDN for the file or directory.