uniq: Report or Filter Out Repeated Lines in a File
August 12th, 2024 2:02 PM Mr. Q Categories: Command
Description:
The uniq command filters out or reports repeated lines in a file. It is often used in combination with the sort command to remove duplicates from sorted data.
Command:
Copy001uniq [options] [input_file] [output_file]
Sample Input
file.txt:
Copy001apple
002banana
003banana
004orange
005apple
006grape
007grape
Sample Output (Default):
Copy001apple
002banana
003orange
004apple
005grape
Options:
-c: Prefix lines with the number of occurrences.-d: Only print duplicate lines.-u: Only print unique lines (lines that are not repeated).-i: Ignore case while comparing lines.
Example with -c Option:
To count occurrences of each line:
Copy001uniq -c file.txt
Sample Output with -c:
Copy001 1 apple
002 2 banana
003 1 orange
004 1 apple
005 2 grape
Example with -d Option:
To display only duplicate lines:
Copy001uniq -d file.txt
Sample Output with -d:
Copy001banana
002grape
Example with -u Option:
To display only unique lines (lines that are not repeated):
Copy001uniq -u file.txt
Sample Output with -u:
Copy001orange
Example with -i Option:
To ignore case while comparing lines:
Copy001uniq -i file.txt
Sample Input file.txt:
Copy001Apple
002banana
003BANANA
004orange
005apple
006GRAPE
007grape
Sample Output with -i:
Copy001Apple
002banana
003orange
004GRAPE
The uniq command is useful for processing text files to eliminate or analyze duplicate lines, especially when used in conjunction with sorting.