2
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

sort | uniq -c しないで重複行をカウントする

Posted at

出力が大きくて sort してると重いときに使います

alias sortuniqc="perl -nlE '\$c{\$_}++; END { say \$_ . q! ! . \$c{\$_} for (sort keys %c) }'"

それなりに軽くなる様子

  • 5000行とかでも2倍以上早くなる
  • 100万行とかだと顕著に違う
$ time cat ~/result-item-search | egrep -o 'skip|preserved' | wc -l
    4214

real	0m0.051s
user	0m0.039s
sys	0m0.009s
00:19:59-sakamotoakira@SakamotoAkira-no-Mac-mini:~  \(^o^)/
$ time cat ~/result-item-search | egrep -o 'skip|preserved' | sort | uniq -c
2758 preserved
1461 skip

real	0m0.133s
user	0m0.111s
sys	0m0.015s
00:20:03-sakamotoakira@SakamotoAkira-no-Mac-mini:~  \(^o^)/
$ time cat ~/result-item-search | egrep -o 'skip|preserved' | sortuniqc
preserved 2758
skip 1461

real	0m0.045s
user	0m0.046s
sys	0m0.010s

参考

2
2
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
2
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?