Skip to content

Duplicate Content Checker

Find the pages on your own site that say almost the same thing, and the exact passages they share, with menus and footers left out of the comparison.

Loading the tool

What the checker compares across your pages

Navigation, header, footer, sidebar and form markup are stripped first, so the comparison is about the words a reader came for, not the template around them.

Unique, duplicate and common text
Every 5-word passage is sorted into one of three groups: found on one page only, shared by a few pages, or so widespread it counts as template text.
Common text threshold
A passage that appears on half or more of the compared pages, and on at least three of them, is treated as common boilerplate rather than duplication.
Near-duplicate pairs
Two pages are compared on the passages that are not template text. Pairs from 30% similar are listed, and 80% or more counts as a near duplicate.
Identical pages
Pages whose whole main text matches word for word are grouped together, which often points to parameter URLs or copied templates.
Repeated passages
The longest runs of shared words are rebuilt as readable text, with how many pages carry each one and sample addresses.
Closest match per page
Each page shows the other page it overlaps with most and by how much, so you can jump straight to the pair that matters.
Word count per page
How many words of main text were read on each page, up to 5,000 per page.
Pages left out
Pages with under 40 words of main text, files that are not HTML and pages that could not be read are counted separately, never mixed into the figures.

How to check a site for duplicate content

  1. Enter the start page

    Usually the home page or the top of a section such as /blog/. The crawl follows links from there and stays on the same site.

  2. Pick how many pages to compare

    Choose 10, 25 or 50. A tight section, such as one product category, often gives a clearer picture than a sample of the whole site.

  3. Let the crawl finish

    The crawler reads two pages in parallel, waits between requests and obeys robots.txt, crawl delay included, so a small server is never flooded.

  4. Work from the top of the pages table

    It is sorted by duplicate share. Open the near-duplicate pairs first, then the repeated passages, to see what the overlap actually is.

Links are followed up to six clicks deep. When robots.txt is unreadable, only the start page is fetched and there is nothing to compare.

How to read the duplicate report

Unique Text
The share of all passages across the compared pages that appear on one page only. Higher means each page carries more of its own words.
Duplicate Text
Passages shared by two or more pages but not common enough to be template. This is the overlap worth looking at.
Common Text
Boilerplate that survived the template stripping: cookie notices, sign-up blocks, repeated disclaimers. Usually harmless, sometimes worth trimming.
Near-Duplicate Pairs
The count of page pairs at 80% similarity or more. The list below it shows every pair from 30%, highest first.
Identical Groups
Sets of addresses serving the same main text. One of them is usually the real page and the rest are copies.
Partial result
The crawl ended early on one of its limits (pages, requests or time). Figures describe the pages compared, not the whole site.

Duplicate content problems and how to fix them

  • The same page reachable at several addresses

    Fix: Pick one address, redirect the others to it with a 301, or point their canonical tag at the address you keep.

  • Location or service pages that only swap a city name

    Fix: Add what is really local: the team, prices, examples, directions or reviews for that place. If there is nothing local to say, merge them into one page.

  • Product variants with copied descriptions

    Fix: Keep one main product page and handle colour or size as options on it, or write a description that explains what is different about each variant.

  • Two blog posts competing for one topic

    Fix: Combine the stronger parts into a single post, redirect the weaker one, and update internal links to point at the survivor.

  • A long block repeated across many pages

    Fix: Shorten it, move it to one page and link to that page, or keep it only where it helps the reader.

  • Tag and archive pages that mirror each other

    Fix: Set thin archive pages to noindex, or give each one an introduction and a reason to exist.

Who runs a duplicate content check

  • Store owners whose product templates reuse the same copy across many items.
  • Agencies reviewing a site with hundreds of near-identical location pages.
  • Bloggers cleaning up years of posts that cover the same question twice.
  • Developers checking that faceted filters and parameters did not create copies of real pages.

Duplicate is not a penalty by default

Search engines usually pick one version of repeated content and show that one. The real cost is that you, not they, should choose which page ranks, and link signals get split across copies. Treat high overlap as a decision to make, not an emergency.

Content Duplicate Checker FAQ

Short answers about the Content Duplicate Checker.

No. It compares the pages of the one site you enter with each other. It is built for internal duplication, such as copied templates and overlapping posts.