Wikipediaにある表から必要なデータを取り出すまで。

とある事情からWikiから医薬品名だけを抜き出して，クエリーに投げる必要が出てきた。ちょっと探したら，Wiki上にステキな一覧を発見。

List of largest selling pharmaceutical products

WikiなのにWikipediaって表の中身を昇順等に並べ替えられるのね。なんてことを思って感心した（今更？）のだけど，ここからどうやってGeneric Nameを取り出そうかなぁ。とちょい考えた。
↑

世界で売れている医薬品名だけを取り出そう。

構造化されたデータは無いかしら

最初はDBpediaからRDF等，構造化されたデータを取り出そうと思ったのだが，データが古くてしかも一覧がなさそうだわ，というわけで諦めた。構造化されたデータがあると便利なのだけど。

というわけでWikipediaにログインして（登録して）Editして中身を見てみた。

|align=center|1||Nexium||[[Esomeprazole]]||align=right|1,420,907||align=right|2.93%||[[AstraZeneca]]||Gastrointestinal disorders||align=right|Mar-2000||align=right|May-2014
|-
|align=center|2||Abilify||[[Aripiprazole]]||align=right|1,405,226||align=right|1.89%||[[Otsuka Pharmaceutical]], [[Bristol-Myers Squibb]]||Psychosis, Depression||align=right|Nov-2002||align=right|Oct-2014…

てな感じだった。これをテキストファイル (ここではdruglist.txtにしました)，[[]] で囲まれている部分を取り出せば良いのではないかしらということで，Rubyでレッツ実装。

色々忘れていたrubyの正規表現記号
¥wが英文字とか忘れてた。

irbで[[]]で囲まれた部分を取り出す正規表現を試す。
matchって便利なのね〜

irb
?> a
=> "asahi [[ocha]] umai"
>> a.match(/\[\[(\w+)\]\]/)[0]
=> "[[ocha]]"
>> a.match(/\[\[(\w+)\]\]/)[1]
=> "ocha"
# ↑これだっ

druglist.rbで書いてみるよ〜一行ずつ文字列にして最初の[[]]で囲まれた文字列だけ順次出します

druglist.rb

File.open("druglist.txt"){|file|
while line = file.gets　#一行ずつ取得
    if line.match(/\[\[(\w+)\]\]/)　#マッチしたら
    print line.match(/\[\[(\w+)\]\]/)[1]
    puts
    end 
end
}

無事に取り出せた。

maori-itos-macbook-2:Downloads Maori$ ruby druglist.rb
Esomeprazole
Aripiprazole
Rosuvastatin
Duloxetine
Adalimumab
Fluticasone
Etanercept
Infliximab
Glatiramer
Filgrastim
Rituximab

後は目的にそぐう形に変形すべし。

何をしたかったというと，私たちが開発している検索エンジンに薬のクエリーを一覧表示するものが作りたかったのである。

druglist.rb

File.open("druglist.txt"){|file|
 print "| Drug Name | Sagace URL |\n|:----------:|:------------:|\n"
 
while line = file.gets
   if line.match(/\[\[(\w+)\]\]/)
    drug = line.match(/\[\[(\w+)\]\]/)[1]
    print "|"+drug+"|"+"http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword="+drug+"|"
    puts
    end 
end
}

出来たぞなもし。

Drug Name	Sagace URL
Esomeprazole	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Esomeprazole
Aripiprazole	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Aripiprazole
Rosuvastatin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Rosuvastatin
Duloxetine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Duloxetine
Adalimumab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Adalimumab
Fluticasone	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Fluticasone
Etanercept	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Etanercept
Infliximab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Infliximab
Glatiramer	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Glatiramer
Filgrastim	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Filgrastim
Rituximab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Rituximab
Oxycodone	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Oxycodone
Tiotropium	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Tiotropium
sitagliptin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=sitagliptin
Bevacizumab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Bevacizumab
Tenofovir	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Tenofovir
Erythropoietin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Erythropoietin
Pregabalin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Pregabalin
Celecoxib	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Celecoxib
Montelukast	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Montelukast
Valsartan	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Valsartan
Imatinib	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Imatinib
Trastuzumab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Trastuzumab
Memantine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Memantine
Buprenorphine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Buprenorphine
Enoxaparin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Enoxaparin
Ranibizumab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Ranibizumab
Escitalopram	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Escitalopram
Valsartan	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Valsartan
Lisdexamfetamine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Lisdexamfetamine
Ezetimibe	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Ezetimibe
Testosterone	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Testosterone
Methylphenidate	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Methylphenidate
Pioglitazone	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Pioglitazone
Lidocaine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Lidocaine
Fenofibrate	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Fenofibrate
Serono	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Serono
Budesonide	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Budesonide
Salbutamol	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Salbutamol
Erythropoietin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Erythropoietin
Quetiapine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Quetiapine
Pemetrexed	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Pemetrexed
Sildenafil	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Sildenafil
amphetamine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=amphetamine
Salbutamol	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Salbutamol
GlaxoSmithKline	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=GlaxoSmithKline
Ipratropium	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Ipratropium
Niacin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Niacin
Erythropoietin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Erythropoietin
Metoprolol	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Metoprolol
Filgrastim	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Filgrastim
Mometasone	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Mometasone
Fluticasone	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Fluticasone
Atazanavir	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Atazanavir
Raltegravir	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Raltegravir
Ezetimibe	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Ezetimibe
Modafinil	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Modafinil
metformin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=metformin
Tadalafil	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Tadalafil
Formoterol	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Formoterol
fingolimod	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=fingolimod
Rabeprazole	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Rabeprazole
Liraglutide	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Liraglutide
Budesonide	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Budesonide
Oxaliplatin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Oxaliplatin
vaccine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=vaccine
Dexlansoprazole	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Dexlansoprazole
Amphetamine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Amphetamine
Fentanyl	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Fentanyl
Rabeprazole	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Rabeprazole
Zostavax	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Zostavax
Raloxifene	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Raloxifene
Olmesartan	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Olmesartan
Olmesartan	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Olmesartan
Atorvastatin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Atorvastatin
Budesonide	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Budesonide
Levothyroxine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Levothyroxine
Omalizumab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Omalizumab
Olmesartan	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Olmesartan
Eszopiclone	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Eszopiclone
vaccine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=vaccine
Capecitabine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Capecitabine
Sensipar	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Sensipar
Linezolid	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Linezolid
Atorvastatin	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Atorvastatin
Ustekinumab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Ustekinumab
Amgen	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Amgen
Tadalafil	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Tadalafil
Tadalafil	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Tadalafil
Cetuximab	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Cetuximab
Levothyroxine	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Levothyroxine
Pantoprazole	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Pantoprazole
Oseltamivir	http://sagace.nibio.go.jp/cgi-bin/sagace/search.cgi?action=search&keyword=Oseltamivir