Blog

Hunt Orphaned Files and Duplicates Before You Migrate

· 6 min read · By the Replatform Radar team

Orphaned assets and duplicate content are the two failures most likely to quietly erode SEO during a replatform because neither triggers an obvious error at cutover. Orphaned assets are files (PDFs, images, old landing pages) that no live page links to but that still earn traffic or backlinks; migrations tend to drop them because crawlers building the migration inventory never reach them. Duplicate content is the same or near-identical text served at multiple URLs; migrations tend to multiply it through new URL patterns, faceted parameters, and lost canonical tags. Both problems compound at scale, and both are cheap to fix before cutover and expensive to diagnose after.

Why these two go unnoticed

Most pre-migration inventories start from a crawl of internal links, an XML sitemap, or a CMS content export. All three share the same blind spot: they only see what the current site chooses to surface. An orphaned PDF that ranks for a high-intent query, or a legacy campaign page that still collects backlinks, sits outside those sources entirely. If it isn't in your inventory, it isn't in your redirect map, and it dies silently on launch day.

Duplicate content fails in the opposite direction — it's too visible, in too many places. A single article might exist at a trailing-slash and non-trailing-slash URL, at HTTP and HTTPS, with and without tracking parameters, and again in a printer-friendly template. On the old platform, canonical tags and a mature redirect layer quietly consolidated all that authority to one URL. A migration resets that plumbing, and the duplicates start competing with each other for the same rankings.

Finding orphaned assets

To catch orphans, you have to look at sources your live site doesn't link to. Union these datasets and diff them against your crawl:

  • Server access logs — the ground truth of what URLs actually get requested, including files no menu points to.
  • Google Search Console — the Pages report and Performance data reveal indexed URLs and query-earning pages that may be absent from your sitemap.
  • Analytics landing-page reports — anything that received organic entrances in the last 12 months deserves a redirect decision.
  • Backlink exports — from Ahrefs, Semrush, or Search Console's links report; a page with referring domains is an asset even if it's orphaned internally.
  • CDN and media-library listings — the raw file store often holds thousands of assets the CMS no longer references.

Any URL that appears in these sources but not in your internal-link crawl is a candidate orphan. Rank candidates by organic traffic, backlink count, and conversion value, then assign each a migrate, redirect, or retire decision. The goal is not to save everything — most orphans genuinely deserve to be killed — but to make that call deliberately instead of by accident.

Finding and resolving duplicates

Duplicate detection needs to run on rendered content, not just URLs. Two distinct URLs with 95% identical body text are functional duplicates even if the paths look unrelated. A content-similarity pass (shingling or embeddings) across your crawl surfaces clusters; then you decide the canonical winner for each cluster and route the rest to it.

Duplicate typeCommon migration causeFix before cutover
Protocol / host variantsNew host not forcing one canonical origin301 to a single HTTPS host; set canonical
Parameter URLsNew faceting or tracking params indexedCanonical to clean URL; disallow in robots where safe
Pagination & print viewsTemplate proliferation on new platformSelf-referencing canonicals; noindex print
Near-duplicate contentThin variants and boilerplate pages migrated as-isConsolidate, rewrite, or kill before launch
Staging leakagePreview/staging URLs left crawlableBlock and password-protect non-production

Do the work before, not after

The sequence matters. Resolve duplicates and orphans in the inventory, before you build the redirect map, so every decision flows downstream cleanly:

  1. Build a union inventory from crawl, logs, GSC, analytics, and backlinks — not a single source.
  2. Score each URL on traffic, links, and content uniqueness to separate assets worth keeping from noise.
  3. Cluster duplicates and pick one canonical per cluster.
  4. Map redirects only after the keep/kill and canonical decisions are locked.
  5. Validate post-launch by re-crawling and re-diffing against the same sources to confirm nothing regressed.

This is precisely the gap a pre-migration audit closes: pulling orphaned files and duplicate clusters into one scored inventory so they get an explicit decision instead of vanishing at cutover. You can see how that inventory-and-scoring approach works in the migration planning guides.

What good looks like at go-live

A clean launch has one indexable URL per piece of content, a canonical tag on every page pointing to itself or its cluster winner, and a 301 for every retired-but-valuable orphan pointing to the closest relevant replacement. No high-traffic PDF 404s. No two pages fighting over the same query. When you re-crawl a week after cutover, the union of logs and Search Console should show requests resolving to live 200s or intentional 301s — never a wall of surprise 404s from files you forgot existed.

Neither of these killers is glamorous, and that's exactly why they get skipped. But the traffic they leak is real, recurring, and hard to trace back to its cause once the migration is three months in the rearview. Spend the hours on the inventory now.

Questions and discussion

Have a question about this article, or a migration you’re planning? Ask below — we read and answer every one.

Loading discussion…

Want this analysis for your exact site before you migrate?

Request a scan →