Skip to content

Bulk URL Cleaner and Editor

Paste a messy list of addresses and get back one consistent, de-duplicated list, with anything that is not a URL set aside instead of slipped in.

Loading the tool

The eleven operations, in the order they run

The order is fixed because it changes the answer. Rewrites happen before duplicates are removed, so two addresses that only differed by www or a query string collapse into one.

Trim each line
Removes spaces and tabs before and after each address, a common leftover from spreadsheets.
Drop blank lines
Empty lines never reach the cleaned list, whether or not this box is ticked.
Lowercase
Puts the whole address in lower case, path and query included.
Strip the query string
Removes everything from the question mark on, including utm tags, session IDs and filter parameters.
Strip the #fragment
Removes jump links such as #reviews, which point to the same page.
Strip a trailing slash
Turns /page/ into /page. It never touches the slash after a bare domain, and skips addresses that still have a query or fragment.
Force https
Rewrites http addresses to https.
Add www. or Remove www.
Puts every host on one form. If both are ticked, removal wins and the shorter form is kept.
Remove duplicates and Sort A to Z
Duplicates are removed after all rewrites, then the list is sorted last so the result is stable however the paste was ordered.

How to clean a list of URLs

  1. Paste the addresses

    One per line, up to 5,000 lines. A bare example.com/page is read as https://example.com/page.

  2. Tick the operations

    Trim, drop blank lines, strip fragments and remove duplicates are on by default. Add the rest to match how your site writes its URLs.

  3. Filter by domain if needed

    Type part of a host name to keep only addresses whose host contains that text.

  4. Run and copy the result

    Copy the cleaned list, and look at the lines reported as not an address before you throw them away.

The paste can be up to 200,000 characters. Lines beyond 5,000 are cut off and the result says so.

What the counts mean

Lines In
Addresses that were read, including ones later filtered out or found to be invalid. Blank lines are not counted.
Kept
How many addresses are in the cleaned list after every operation.
Duplicates Removed
Addresses that became identical after the rewrites and were merged.
Filtered Out
Addresses whose host did not contain your domain filter text.
Not An Address
Lines that could not be read as an http or https URL, such as a mailto link, a javascript link or a sentence. They are listed so you can fix them.

URL list problems it solves

  • Analytics exports full of utm tags

    Fix: Tick Strip the query string and Remove duplicates. Twenty tracked versions of one landing page become one line.

  • A sitemap or crawl list with mixed http and https

    Fix: Use Force https plus your preferred www choice so the list matches your canonical addresses.

  • Duplicates that differ only by a trailing slash

    Fix: Pick one form for your site and apply it. Check which form your canonical tags use before you choose.

  • Other domains mixed into a backlink or crawl list

    Fix: Enter your domain in the filter. Remember it matches any host containing that text, so subdomains stay in.

  • Junk lines pasted along with the URLs

    Fix: They are pulled out and listed separately, so they never leak into the list you paste into the next tool.

Who cleans URL lists

  • SEOs preparing a list for a redirect map, a disavow file or an indexing request.
  • Analysts merging page reports from several tools that write URLs differently.
  • Developers building a test list of canonical addresses after a migration.
  • Content managers turning a pile of copied links into a tidy inventory.

Be careful with Lowercase

Domain names ignore case, but paths often do not. On many servers /About and /about are different pages, and one may not exist. Only lowercase a list when you know the site treats paths that way.

Ultimate URL Editor FAQ

Short answers about the Ultimate URL Editor.

Up to 5,000 lines and 200,000 characters per run. A longer paste is cut at the line limit and flagged in the result.