List Clean-Up Tools - Sort, Deduplicate and Reformat Any List
Exported lists are almost never usable as-is. They arrive with the wrong separator, stray whitespace, repeated rows and no order. These four tools handle those problems in the browser, in a sequence you can repeat every time, without opening a spreadsheet or writing a throwaway script.
The Order That Works
Clean-up steps are not commutative — running them in the wrong order silently produces the
wrong result. Deduplicating before you trim whitespace, for example, leaves
apple and apple as two separate entries. This sequence
avoids every common trap:
| # | Step | Why it goes here |
|---|---|---|
| 1 | Split to one item per line | Every other tool works line by line. Do this first or nothing else applies. |
| 2 | Trim whitespace | Invisible leading and trailing spaces defeat both dedupe and sort. |
| 3 | Remove duplicate lines | Cheaper on a trimmed list, and the removed-items panel is actually readable. |
| 4 | Sort lines | Sorting last means you sort only the rows you are keeping. |
| 5 | Bulk edit | Add quotes, prefixes or suffixes once the set is final. |
| 6 | Rejoin with your separator | Back to CSV, a SQL IN clause, or a code array. |
The five-second sanity check
- One item per line? If not, split it with the delimiter converter before anything else.
- Trim turned on? Invisible spaces defeat both sorting and deduplication.
- Case consistent? Apple and apple survive as two rows otherwise.
- Counted the rows? Note the number now so you can tell what changed.
- Know your output format? Decide before you start whether you need CSV, one-per-line or a SQL clause.
Common Clean-Up Jobs
De-duplicating an email or contact export
Marketing exports routinely contain the same address several times with inconsistent casing. Trim first, then remove duplicate lines with case-insensitive matching on, and check the removed panel before you commit — it is the fastest way to spot that a supposed duplicate was actually two different people at the same domain. Count the result with the line counter to confirm the drop is the size you expected.
Building a SQL IN clause from a spreadsheet column
Copy the column, paste it into the delimiter converter, choose a
comma join and single-quote wrapping, and you have a paste-ready
IN ('a', 'b', 'c'). Enable dedupe in the same pass if the column is not already
distinct.
Converting between CSV, TSV and pipe-delimited
Spreadsheets copy as tab-separated, European locales export semicolon-separated, and legacy feeds often use pipes. All three are one switch of the split mode in the delimiter converter. Verify the row count afterwards with the line counter.
Normalising a keyword or tag list
Keyword exports need lowercasing, deduplicating and sorting before they are useful. Run case conversion to lowercase first — otherwise "Running Shoes" and "running shoes" survive as two entries — then dedupe and sort. If the output is destined for URLs, the slug generator handles the whole list in bulk.
Checking Your Work
Every step here changes how many items you have, so a count before and after is the cheapest possible sanity check. The line counter gives you the row count, and the other text counters cover characters and words when a field-length limit is involved. If a number moves in a direction you did not expect, you have caught a problem before it reached your database instead of after.
Frequently Asked Questions
Should I sort before or after removing duplicates?
Remove duplicates first, then sort. Deduplicating is order-independent, so sorting first just means sorting rows you are about to throw away. The one exception is when you want to see duplicates grouped together visually before deciding — sort first for the inspection, then dedupe.
Why do identical-looking lines survive deduplication?
Almost always trailing whitespace, or a Windows carriage return on one line and not the other. Enable the trim option before deduplicating and both disappear.
Can I clean up a comma-separated string without splitting it first?
Not reliably. The Delimiter.site sort and deduplicate tools work line by line, so a single-line CSV string counts as one item. Split it with the delimiter converter first, clean it, then rejoin.
Is there a size limit on these tools?
Everything runs in your browser, so the practical limit is your device's memory rather than a server quota. Lists of tens of thousands of lines are fine; multi-megabyte pastes will start to feel slow.