uniq

uniq 與 sort 一起使用時，可以控制那些重複的資訊。

基本操作

$ cat file2
ChhatrapatiShahuMaharaj
Dr.B.R.Ambedkar
Budhha
Dr.B.R.Ambedkar
Budhha
Dr.B.R.Ambedkar
Budhha

$ uniq file2
ChhatrapatiShahuMaharaj
Dr.B.R.Ambedkar
Budhha
Dr.B.R.Ambedkar
Budhha
Dr.B.R.Ambedkar
Budhha

$ sort file2
Budhha
Budhha
Budhha
ChhatrapatiShahuMaharaj
Dr.B.R.Ambedkar
Dr.B.R.Ambedkar
Dr.B.R.Ambedkar

$ sort file2 | uniq
Budhha
ChhatrapatiShahuMaharaj
Dr.B.R.Ambedkar

With -c, --count option

# using the -c option to count the repeated lines
$ sort file2 | uniq -c
    3 Budhha
    1 ChhatrapatiShahuMaharaj
    3 Dr.B.R.Ambedkar

With -d, --repeated option

# The -d option prints only lines that are repeated.
$ sort file2 | uniq -d
Budhha
Dr.B.R.Ambedkar

# using the -c option to cross-check whether the -d option is only printing the repeated lines or not
$ sort file2 | uniq -cd
    3 Budhha
    3 Dr.B.R.Ambedkar

With -D, --all-repeated option

# The -D option prints repeated lines and discards the non-duplicate lines
$ sort file2 | uniq -D
Budhha
Budhha
Budhha
Dr.B.R.Ambedkar
Dr.B.R.Ambedkar
Dr.B.R.Ambedkar

With -u, --unique option

# the -u option prints unique lines i.e., non-duplicate lines
$ sort file2 | uniq -u
ChhatrapatiShahuMaharaj

With -i, --ignore-case option

# Using the -i option, we can ignore the case sensitivity of characters
$ cat file3
aaaa
aaaa
AAAA
AAAA
bbbb
BBBB

$ uniq file3
aaaa
AAAA
bbbb
BBBB

$ uniq -i file3
aaaa
bbbb

With -f, --skip-fields=N

# skip some fields to filter duplicate lines
$ cat file5
Amit aaaa
Ajit aaaa
Advi bbbb
Kaju bbbb

$ uniq file5
Amit aaaa
Ajit aaaa
Advi bbbb
Kaju bbbb

$ uniq -f 1 file5
Amit aaaa
Advi bbbb

With -s, --skip-char=N option

# skip characters
$ cat file4
22aa
33aa
44bb
55bb

$ uniq file4
22aa
33aa
44bb
55bb

$ uniq -s 2 file4
22aa
44bb

With -w, --check-chars=N option

# consider characters as well using the -w option
$ cat file6
aa12
aa34
bb56
bb78

$ uniq file6
aa12
aa34
bb56
bb78

$ uniq -w 2 file6
aa12
bb56