Pilotiko feature : available

Noindex: how do you keep a page out of search results?

A one-line rule that tells search engines: crawl this page if you like, but don't show it.

Noindex is a rule for search engines, delivered either by a <meta name="robots" content="noindex"> tag or by an X-Robots-Tag HTTP header. When Googlebot sees it on a crawl, Google drops the page from its results even if other sites link to it. The page itself stays online for anyone who has the address.

Try 30 days free

30 days free, no credit card.

By Pilotiko editorial team · Updated on

In short

  • Two methods, same effect: a robots meta tag or an X-Robots-Tag header.
  • Google doesn't support a noindex rule inside robots.txt.
  • A page blocked by robots.txt can't pass on its noindex, because the crawler never sees it.
  • none is shorthand for noindex, nofollow.
  • Nothing changes until the next crawl, which can take months for a low-traffic page.

How do you write a noindex rule?

In the page's <head>, the general form addresses every compliant search engine: <meta name="robots" content="noindex">. To target Google alone, swap robots for googlebot [1]. Google warns that other engines may read the rule differently, so the page might still appear elsewhere [1].

For files that aren't HTML, such as a PDF price list or an image, a meta tag isn't possible, so the server sends an X-Robots-Tag: noindex header with the response instead [1]. Both methods have the same effect, and noindex can be combined with other rules such as nofollow [1].

Common robots meta rules and what they do in Google
RuleEffect
allNo restrictions; it's the default, so there's no need to state it
noindexThe page doesn't appear in search results
nofollowGoogle doesn't follow the page's links to discover other pages
noneSame as noindex, nofollow
nosnippetNo text snippet, including in AI Overviews
indexifembeddedOnly with noindex: allows indexing of the content when embedded elsewhere via iframe

Source: Google's robots meta tag specifications [2].

Why don't noindex and robots.txt mix?

This is the classic trap. For noindex to work, the page must not be blocked by robots.txt: if the crawler isn't allowed in, it never sees the tag, and the page can keep showing in results when other pages link to it [1]. Google adds that noindex inside robots.txt isn't supported [1].

Put simply, robots.txt controls crawling, while noindex controls whether a page shows up in results. To remove a page, leave it crawlable and add noindex. If a page lingers in results, Google points to two likely causes: it hasn't recrawled the page yet, or robots.txt is stopping it from reading the tag [1].

Which pages on a service business site should be noindexed?

The usual candidates: the thank-you page after a quote request, booking confirmation pages, a staging copy of the site, internal search results, and pages for an expired promotion kept alive for customers who have the link. Each has a reason to exist, but none should be a stranger's first door into your business from Google.

On the other hand, don't use noindex to sort out duplicates between two versions of the same page; the canonical tag does that job. And take any noindexed page out of your XML sitemap so you aren't sending mixed signals.

How many pages should actually leave the index?

Which noindex mistakes should you avoid?

  • Putting noindex on the home page during a redesign and forgetting it at launch, which takes the site's main entry point out of results.
  • Blocking a page in robots.txt and expecting its noindex to be read [1].
  • Expecting instant results; depending on the page's importance, Googlebot may take months to come back [1].
  • Noindexing a town or service page you actually want found, through a template error.
  • Using none when you only meant to hide the page; it also stops link following [2].

How do you confirm Google has picked up a noindex?

Google points to its URL Inspection tool to request a recrawl, and to its removals documentation when a page has to come down urgently [1]. The inspection tool is part of Google Search Console. Check the response headers, not just the HTML: a forgotten X-Robots-Tag header on the server has the same effect as a tag.

What does Pilotiko do with noindex?

On the site Pilotiko builds and hosts, utility pages carry noindex, follow and are left out of the sitemap and the llms.txt file. The temporary address of a rebuilt site is served with an X-Robots-Tag noindex header until you make it public, so Google doesn't see a duplicate of your old site. The audit of your current site flags pages that ask not to be indexed, and treats it as critical on the home page.

Frequently asked questions

Does noindex take the page off my website?

No. The page stays online and reachable by anyone with the address or a link. It only disappears from the results of search engines that honour the rule.

How long before a noindexed page leaves Google?

Googlebot has to recrawl the page first. Google says that can take months for a less important page; the URL Inspection tool lets you request a recrawl.

Can I put noindex in robots.txt?

No. Google doesn't support that rule in robots.txt. Use the robots meta tag or the X-Robots-Tag header.

What is the difference between noindex and nofollow?

Noindex keeps the page out of results. Nofollow asks crawlers not to follow the links it contains. You can combine them, and the none rule means both.

How do I noindex a PDF?

You can't put a meta tag in a PDF. Configure the server to send an X-Robots-Tag: noindex header with that file.

Sources

  1. Google Search Central, Block search indexing with noindex (accessed on October 15, 2026).
  2. Google Search Central, Robots meta tag, data-nosnippet and X-Robots-Tag specifications (accessed on October 15, 2026).

Your whole business, on one screen.

Each department is a room. Pilotiko prepares the work; you approve with one click.

Try 30 days free

The Pilotiko office of a sample company: each department is a room.
  • Call answered
  • Appointment confirmed
  • Email reply ready
  • Invoice prepared

Try the office with your team

30 days free.

Try 30 days free