Text List Cleaner - remove duplicate lines, sort, trim

What to do with the list Remove duplicate lines Keeps the first copy, drops the rest. Two lines count as the same only when every character matches, capitals and edge spaces included, unless the two boxes below change that. Sort alphabetically Code order, not dictionary order: digits, then A-Z, then a-z. So Banana lands before apple until you tick Ignore case. Ties keep their pasted order. Remove empty lines Drops a line with no characters, and also a line holding only spaces or tabs. This does not wait on the trim box. A newline at the very end is not counted as a line. Trim whitespace Strips spaces and tabs from both ends of every line. Runs first, so apple and apple become the same line for the duplicate check. Ignore case Not an action on its own. It changes how the two boxes above compare text: duplicates ignore capitals, and sorting becomes case insensitive. With nothing else ticked it does nothing. Update as I type On by default. Untick it on a very large list and use the Clean list button instead, so the page is not recomputing on every keystroke.

Your list

0 lines

Cleaned list

0 lines

Lines in

0

Counted before anything is removed.

Lines out

0

What the box on the right holds.

Removed

0

Lines in minus lines out.

Nothing to clean yet.

The exact rules this page uses

Most list cleaners give you the same five switches and never say what they mean, so you find out that Apple survived next to apple only after you have pasted the result somewhere. Here is the whole specification. It is short on purpose.

The order the steps run in

Always this order, whichever boxes are ticked:

Trim whitespace. Spaces and tabs are stripped from both ends of every line. The inside of a line is never touched, so New York keeps its double space.
Remove empty lines. A line goes if it has no characters, or if everything in it is whitespace.
Remove duplicate lines. The first copy stays, every later copy goes.
Sort alphabetically. Runs last, on whatever survived.

The order is what makes the switches work together rather than fight. Trimming before the duplicate check is what lets apple and apple be recognised as the same entry. Removing blanks before the duplicate check is what makes twelve blank lines disappear instead of collapsing into one surviving blank line.

What counts as a duplicate

Which boxes make two lines count as the same line
Boxes ticked	Two lines are the same when…	`Apple` vs `apple`
Neither	Every character matches exactly, capitals and edge spaces included	Different
Trim whitespace	Every character matches after both ends are stripped	Different
Ignore case	The lowercased text matches, edge spaces still count	Different
Both	The lowercased text matches after both ends are stripped	Duplicate

When copies are spelled differently and Ignore case is on, the surviving line is spelled the way the first copy was. Apple, then APPLE, then apple leaves you Apple.

What counts as an empty line

A line of zero characters is empty.
A line made only of spaces or tabs is empty as well. This holds whether or not Trim whitespace is on, so the two boxes cannot disagree with each other.
A newline at the very end of the box ends the last line rather than starting a new empty one. A list of three items that you finished with Enter is three lines here, not four.
\r\n and a bare \r are both read as one line break, so a list copied out of Windows behaves the same as one copied out of anywhere else. The result is always joined back with \n.

What alphabetical means here

Lines are compared character by character in code order, which is the ordering you get from the JavaScript < operator, and it is not dictionary order. Every capital letter sits below every lowercase letter, and digits sit below both. With Ignore case off, [Banana, apple, Cherry] sorts to Banana, Cherry, apple. Tick Ignore case and the same list sorts to apple, Banana, Cherry, which is what most people mean by alphabetical. The sort is stable, so two lines that compare equal stay in the order you pasted them.

Locale rules are deliberately not applied. A locale aware sort puts Å in a different place for a Swedish reader than for an English one, and the answer would then depend on the language settings of whichever machine opened the page, which is not something a list cleaner should be quietly doing.

Accented letters, and characters you cannot see

Text arrives spelled in more than one way, and this page never quietly respells it. The rules below are the consequences of that, written out so a surprising result is at least a documented one.

How the page treats accents, invisible characters and unusual spaces
Case	What the page does	Why
The same accented word spelled two ways, `café` as one `é` versus `e` plus a combining accent	Two different lines. Not duplicates, and they sort apart	No Unicode normalisation is applied. Normalising would edit characters you did not ask to have edited, and the two spellings really are different text
A zero width space or a zero width joiner inside a line	An ordinary character. It is never trimmed, a line holding only zero width characters is not empty, and it makes the line different from its clean twin	Those characters are not whitespace. Dropping them would be a silent edit, and a line that looks blank but is not is worth seeing
A non breaking space, an ideographic space, or a byte order mark at the end of a line	Trimmed, same as an ordinary space	Trim uses the whitespace set the language itself defines, which includes all three
Turkish `İ` and `ı` with Ignore case on	`İ` is not the same as `I`, and `ı` is not the same as `i`	Lowercasing is done without a locale, for the same reason sorting is. A Turkish rule applied to an English list would be just as wrong the other way round
Emoji, and characters above `U+FFFF` generally	Kept whole, and sorted by code unit, which places them before the last slice of ordinary characters rather than after everything	Code order is the ordering of the underlying storage units, and those characters are stored as a pair. The ordering is at least exactly reproducible
A right to left override left in a pasted line	Kept as a character. It may reorder how the line looks, but comparison and sorting use the stored order	Removing it would change the text; ignoring how it displays is what keeps the result predictable

Questions

What counts as a duplicate line?

By default two lines are duplicates only when every character matches exactly, including capital letters and including spaces or tabs at the start and end of the line. So apple and Apple are two different lines, and apple and apple with a trailing space are also two different lines. Turning on Ignore case makes the comparison case insensitive. Turning on Trim whitespace strips the spaces and tabs from both ends of every line before the comparison happens. The two boxes can be combined.

Which copy of a duplicate line is kept?

The first one, counting from the top of the box. Later copies are dropped. When Ignore case is on and the copies are spelled differently, the surviving line keeps the spelling of the first occurrence.

What counts as an empty line?

A line with no characters at all, and also a line that holds nothing but whitespace such as spaces and tabs. That rule does not depend on the Trim whitespace box. A single newline at the very end of the box is treated as the end of the last line rather than as an extra empty line.

How does sort alphabetically order the lines?

Lines are compared character by character in code order, which is the ordering the JavaScript less than operator uses. That is not dictionary order: every capital letter sorts before every lowercase letter, so Banana comes before apple. Turning on Ignore case compares the lowercased text instead. The sort is stable, so lines that compare equal stay in the order they were pasted in.

What order do the options run in?

Always the same fixed order, no matter which boxes you tick: trim whitespace first, then remove empty lines, then remove duplicate lines, then sort.

Is my text uploaded anywhere?

No. The whole tool is a few array operations that run inside the page. There is no account, no upload, no analytics and no server side storage, and the site has no endpoint that accepts a request body at all. Nothing is written to the address bar either, so a pasted list does not end up in browser history.

Built by

This page is one of a small set of single purpose tools that run entirely in the browser, with no account and no server side storage. The set is listed below. If you write a lot of text rather than a lot of lists, the character limit checker in the same set measures a draft against the published limits of the platform you are about to paste it into.