Remove Duplicate Lines
Clean up a list that has repeats in it, such as emails, product codes or names, without changing its order. Choose how capitals, spaces and empty lines are treated.
Runs in your browser. Your data is not uploaded.How our tools work
Line statistics
- Original lines
- 0
- Duplicates removed
- 0
- Remaining lines
- 0
How do I remove duplicate lines from a list?
Paste your list with one item per line and every repeated line is removed straight away, keeping the first occurrence of each in its original order. Choose whether Apple and apple count as the same, whether to ignore spaces at the ends of lines, and whether to remove empty lines. Then copy the cleaned list.
Key features
Original order kept
The first occurrence of each line stays where it was. Nothing is sorted.
Case and spacing options
Treat Apple and apple as the same line or not, and ignore spaces at the start and end when comparing.
Empty lines handled your way
Keep blank lines where they are, or remove them. They are never treated as duplicates of each other.
Counts before and after
See how many lines you started with, how many duplicates were removed and how many remain.
Common uses
- Clean up email lists, keyword lists or product codes
- Remove repeated entries from log files or exports
- Deduplicate names before a mail merge or import
How lines are compared
Two lines count as duplicates when they are identical, character for character. With “Case-sensitive” turned off, capital and small letters are treated as the same. With “Ignore spaces at the start and end” turned on, leading and trailing spaces and tabs are ignored, but the line you keep is left exactly as it was.
Accented letters can be stored in two ways that look identical, for example é as one character or as e plus an accent. Both forms are treated as the same line.
Related tools
Frequently Asked Questions
Does it sort my lines?
No. The lines stay in their original order, and the first time each line appears is the one that is kept.
What happens to empty lines?
By default they stay where they are and are not counted as duplicates, so paragraph spacing survives. Turn on “Remove empty lines” to delete them.
Why wasn’t a line removed?
It probably differs slightly, for example in capital letters or a space at the end. Turn off “Case-sensitive” or turn on “Ignore spaces at the start and end”, and check again.
Is there a limit on the number of lines?
There is no fixed limit. Very long lists of several hundred thousand lines work, although pasting them can take a moment.
Last updated: