afaqrashid.com  ·  portfolio · 2026 ed.

How to Noindex a Page in Next.js (and the Traps I Tested)

By Afaq Rashid ·

To noindex a page in the Next.js App Router, set robots in that page's metadata:

// app/thank-you/page.tsx
export const metadata: Metadata = {
  robots: { index: false },
};

On Next.js 16.4.4 that printed <meta name="robots" content="noindex"/> in the head. Don't also block the page in robots.txt, or Google never sees the tag. That mistake, and the other cases below, I tested on a fresh app with next build and next start.

Noindex a whole section

Put the same robots value in that folder's layout.tsx. Every page under it inherits it, unless a page sets its own robots. A page with no robots of its own got the layout's tags in my test.

// app/account/layout.tsx
export const metadata: Metadata = {
  robots: { index: false, follow: false },
};

Noindex a staging or preview site

Decide in the root layout from an environment variable, so production is the only place that can be indexed. On Vercel, VERCEL_ENV is "production" only for production deployments:

// app/layout.tsx
const isProduction = process.env.VERCEL_ENV === "production";

export const metadata: Metadata = {
  robots: isProduction ? { index: true, follow: true } : { index: false, follow: false },
};

Check both after you deploy. A staging site that leaks into Google is annoying. A production site that ships noindex by mistake drops out of the index.

Trap: a page's robots replaces the layout's

Metadata merges shallowly, so robots is replaced as one object. My root layout set googleBot rules like max-image-preview:large. A page with robots: { index: false } got only this:

<meta name="robots" content="noindex"/>

The googlebot tag from the layout was gone. For a noindex page that doesn't matter. It does matter the other way round: if a page sets robots to change one thing, it loses everything else the layout set, so repeat what you need.

You can also pass a string, robots: "noindex, follow", and Next.js prints it as it is.

Trap: blocking it in robots.txt as well

It feels safer to do both. It isn't. Google says the page "must not be blocked by a robots.txt file" for noindex to work, because a crawler that can't fetch the page can't read the tag. The URL can then still show up in results if other pages link to it.

Use noindex to keep a page out of the index, and robots.txt to stop crawling. If you want a page gone, allow crawling until Google has seen the noindex.

Noindex PDFs and other files

Files in public/ have no head to put a meta tag in. Google accepts an X-Robots-Tag header for them instead, which you can set in next.config:

// next.config.ts
const config: NextConfig = {
  async headers() {
    return [
      { source: "/files/:path*", headers: [{ key: "X-Robots-Tag", value: "noindex" }] },
    ];
  },
};

A PDF in public/files/ came back with X-Robots-Tag: noindex and its normal content type.

404 pages are handled for you

When notFound() runs, or a URL doesn't exist, Next.js adds its own noindex tag and returns a 404 status. In my test that tag sat next to the layout's "index, follow", which looks wrong in the source. The 404 status keeps the page out of Google either way.

Check it worked

curl -s https://example.com/thank-you | grep -oE '<meta name="(robots|googlebot)"[^>]*>'
curl -sI https://example.com/files/guide.pdf | grep -i x-robots-tag

Google only sees the change when it recrawls the page, and Google notes that "it may take months" for some pages. Search Console's URL Inspection can request a recrawl and shows the HTML Googlebot got. The metadata checker below flags a noindex tag in pasted HTML too.

Related

Sources