List Clean-Up Tools - Sort, Deduplicate and Reformat Any List

Exported lists are almost never usable as-is. They arrive with the wrong separator, stray whitespace, repeated rows and no order. These four tools handle those problems in the browser, in a sequence you can repeat every time, without opening a spreadsheet or writing a throwaway script.

The Order That Works

Clean-up steps are not commutative — running them in the wrong order silently produces the wrong result. Deduplicating before you trim whitespace, for example, leaves apple and apple  as two separate entries. This sequence avoids every common trap:

The five-second sanity check

• One item per line? If not, split it with the delimiter converter before anything else.
• Trim turned on? Invisible spaces defeat both sorting and deduplication.
• Case consistent? Apple and apple survive as two rows otherwise.
• Counted the rows? Note the number now so you can tell what changed.
• Know your output format? Decide before you start whether you need CSV, one-per-line or a SQL clause.

Common Clean-Up Jobs

De-duplicating an email or contact export

Marketing exports routinely contain the same address several times with inconsistent casing. Trim first, then remove duplicate lines with case-insensitive matching on, and check the removed panel before you commit — it is the fastest way to spot that a supposed duplicate was actually two different people at the same domain. Count the result with the line counter to confirm the drop is the size you expected.

Building a SQL IN clause from a spreadsheet column

Copy the column, paste it into the delimiter converter, choose a comma join and single-quote wrapping, and you have a paste-ready IN ('a', 'b', 'c'). Enable dedupe in the same pass if the column is not already distinct.

Converting between CSV, TSV and pipe-delimited

Spreadsheets copy as tab-separated, European locales export semicolon-separated, and legacy feeds often use pipes. All three are one switch of the split mode in the delimiter converter. Verify the row count afterwards with the line counter.

Normalising a keyword or tag list

Keyword exports need lowercasing, deduplicating and sorting before they are useful. Run case conversion to lowercase first — otherwise "Running Shoes" and "running shoes" survive as two entries — then dedupe and sort. If the output is destined for URLs, the slug generator handles the whole list in bulk.

Checking Your Work

Every step here changes how many items you have, so a count before and after is the cheapest possible sanity check. The line counter gives you the row count, and the other text counters cover characters and words when a field-length limit is involved. If a number moves in a direction you did not expect, you have caught a problem before it reached your database instead of after.

Frequently Asked Questions

Should I sort before or after removing duplicates?

Remove duplicates first, then sort. Deduplicating is order-independent, so sorting first just means sorting rows you are about to throw away. The one exception is when you want to see duplicates grouped together visually before deciding — sort first for the inspection, then dedupe.

Why do identical-looking lines survive deduplication?

Almost always trailing whitespace, or a Windows carriage return on one line and not the other. Enable the trim option before deduplicating and both disappear.

Can I clean up a comma-separated string without splitting it first?

Not reliably. The Delimiter.site sort and deduplicate tools work line by line, so a single-line CSV string counts as one item. Split it with the delimiter converter first, clean it, then rejoin.

Is there a size limit on these tools?

Everything runs in your browser, so the practical limit is your device's memory rather than a server quota. Lists of tens of thousands of lines are fine; multi-megabyte pastes will start to feel slow.