経緯
2018年、フェイクニュースを自動生成するAI、GPT-2がOpenAIから発表されました。
あまりにリアルさに、そのすべての学習モデルの公開は延期されています(2019年7月現在は345M版まで公開されている)。
しかし学習用のコードとしてnshepperd氏がnshepperd/gpt-2が公開しており、
これを応用してGwern氏が英語の詩の自動生成を行っていました。
面白そうだったのでKaggleで公開されていた歌詞データから、19,289作のHiphopの歌詞を抽出して、GPT-2に学習させてみることにしました。
参考
Gwern氏による英語詩の自動生成
GPT-2 Neural Network Poetry
免責事項
歌詞データには(英語ですが)性的卑語、人種差別的と思われる言葉、性差別表現などが含まれる可能性があります。
文化研究の一環ということで特に修正はしておりません。データを利用する際はこの点に注意してご使用ください。
コード
Google Colabでサンプルを実行できます。
Noutrious-GPT2 sample
前処理
以下の条件で前処理をします。
- 英語のみ抽出(データにはフランス語、ドイツ語、スペイン語と思われる歌詞データが含まれており、これらに該当するだろう歌詞を排除する)
- Verse1といった歌詞ではないワードを削除
- 括弧のような記号も削除
import pandas as pd
# 歌詞データを読み込む
df = pd.read_csv("lyrics.csv")
# Hiphopジャンルのみ抽出し、歌詞が欠損している行を削除
df_hiphop = df[df['genre'] == "Hip-Hop"].dropna(subset=['lyrics'])
# 歌詞ではないと思われる要素を一括置き換え
patterns = '\[[^]]+\]|\([^)]+\)|\{[^}]+\}|[+]+|[. ]{2,}|(VERSE|Verse|verse)[ ]*\d'
df_hiphop['lyrics'] = df_hiphop['lyrics'].replace(patterns, '', regex=True)
# 言語については、特徴的な単語をもとに雑に判断しています
language_keywords = ["ich", "suis", " que ", " je ", " ca "]
for keyword in language_keywords:
df_hiphop = df_hiphop[~df_hiphop['lyrics'].str.contains(keyword)]
次に歌詞の行ごとに処理します。
一度データを一行にまとめて、不要な改行を削除します。
import re
# 歌詞をリストに変換し、改行で結合
text = "\n".join(df_hiphop['lyrics'].values)
# 連続する改行と特定のパターンを置換
text = re.sub("\n{2,}", "\n", text) # 2つ以上の改行を1つに置換
text = re.sub("\\'", "'", text) # エスケープされたシングルクォートを置換
text = re.sub(" \n", "\n", text) # 改行前の空白を削除
次に行を配列にしてから短すぎる行などを削除します。
今回悩みましたが、重複する内容も削除しています(これは歌詞的にダメかもしれない……)
# Delete too short or too long lines.
lines = text.split("\n");
lines = [line for line in lines if line]
lines = [line for line in lines if len(line) > 3 and len(line) < 369]
lines = [line for line in lines if " " in line]
# 重複を削除(できる限り順番を維持して)
seen = set()
seen_add = seen.add
lines = [ x for x in lines if x not in seen and not seen_add(x)]
text = "\n".join(lines)
最後に、ファイルに保存します。
with open("lyrics.txt","w",encoding="utf-8") as f:
f.write(text)
学習開始
Google Colabで公開しています。
コード通りに実行すれば、自動的に学習が始まります。
!PYTHONPATH=src ./train.py --dataset lyrics.txt.npz --batch_size 2 --save_every 1040 --sample_every 100 --learning_rate 0.0001
学習実行のためコマンドになっています。
batch_sizeに関しては、この数字より大きくすると自分の場合は止まってしまいました。適切な数字を入れるようにしてください。
結果
卑語などが含まれます。
筆者は英語に通じていないので判断し辛いのに加えて、黒人英語なのか、それとも単に出力として失敗しているのかよく分からない点も多いです。
Wrap up in plastic
Tucked in some cotton
Grown man I know I could hang
The nigga just got out of jail
He's gonna make a million this year
I'll take a shot
I'll just kill for ya
Like a man
I'll just kill for ya
Nigga if I have to
Fuck around and lose my life
I wanna die for ya
Nigga I wanna kill, nigga fuck around and lose my friend if you want me to nigga I can't stay on the streets I gott
a get to that money
And I got a brother that'll ride with me
He a gangsta for life so I can't let him go
And I know a girl I'm gonna take to my town tonight
And she know I gotta be up in my way tonight
'Cause I'm the kind of nigga that'll kill for ya
But I can't let 'em ride with you
I wanna die for ya I wanna kill
Whip it out ya
I'm gonna show them nigga what you about
I got a little brother too
So I'm gonna give him a nigga way
You gon' be in the cut wit a big nigga body
I know I got to be up in this shit
Might slip and slide
But I'm a be on the down down
No I ain't leavin'
I wanna just help out like I said in my last song
But I just got to do my own thing
Please don't want to hurt me, I just got to do my own thing
And I can't give you shit thoughI feel nothing
Nothing for you nothing for us nothing that you want you just listen to me and listen to me
I wanna just help out like I said in my last song
But I just got to do my own thingPlease don't want to hurt me, I just got to do my own thing
Please don't want to hurt me, I just got to do my own thing
It's my right to do what I feel
This is not a game baby please don't call it that
There might not be no stars but still I'll always be your number one
And I just wanna let you know that
I will always love you so much more than words
Yes, yes I'll always love you with all my heart
The next time you see me I'll know that I really mean it
Yeah, baby I know that there will never be another you
But I will always love you with all my heart
I've done a bunch of things I regret
That I'll never, I've been on my own
If I would have known that I'd be what I am
money as a fuckin' dream
P-p-passing laws, it's just what I'm on
And I gots to go in and get a fuckin' mill
Get the money, it's on, I'm coming back
This that motherfucker shit that you need
To get the bread, just to get up on a fuckin' table
You want me, get a hundred and fifty
Fuck that, that's right, that's two hundred and fifty I think
You really want me, motherfucker get one
If you really want me, nigga go get one
You want me, motherfucker get one
You really want me to go get one
To all my people out the ghetto
All the ghetto niggas out the ghetto
In the ghetto, on the west, on the east
It's real all around, and it's here to stay
All around, and it's here to stay
In the ghetto, in the ghetto
It's real all around
Now I was born in the ghetto
This is where my mother came from
The ghetto was the city
From a ghetto bed to a ghetto house
My father started off in the ghetto
Now I come from a ghetto bed
That's why I'm a ghetto star
Now I'm from a house where the rent's by the door
And I'm from a ghetto mind
That's why I'm a ghetto nigga
I'm from a ghetto man
That's why I'm a ghetto nigga
Now a ghetto nigga never had nothin'th to do
But play the ghetto card and go and get my hustle on
I'm on the block with my tool on
I just had to get my hustle on
ちょっと意味不明で面白いです。
I'm a man, I'm a man
I'm a man, she's a woman
I'm a man, I'm a manShe's a woman
I'm a woman
Man I get so hot in the summer
She get so hot all the time
Cause I got a thing for you
I'm a man and a woman
I'm a woman, man I got that thing for you
Uh I'm a guy, I'm a girl
I'm a guy, I'm a girl, I'm like
A girl, a girl
I'm like I wanna run up on you
Tell you what I'm gonna do
I'm gonna grab a pen
Wanna write somethin', man?
I'm gonna write somethin', uh
I'm a writer, you gonna run up on 'em
Shoot a few somethin', make something
Wanna make somethin', I wanna spend somethin', uh
I'm a man, I'm a girl
A girl, a girlYo, I'm a man, I'm a girl, I'm like
I got a thing for you
A girl, a girlYo, I'm a man, I'm a girl, I'm likeShe's a girlYou know, uh, uh, I'm a man, I'm a girl, I'm a girl
A girl, a girlIt's the way I write I'm a man
You could never see me at the bar, no
Cause I got this thing on, man
My girls like they know you
Girl you know they got you
So here's a little something to consider:
I got this book that tells the story of
the race between man and rock
And the man who conquered was not a man at all
He was the race of god and he lived in heaven
But his race was not made just because his God lived in heaven
He was a god who beat God
Then God beat God
And that was all the reason you had to believe the story
But if that's the same story
I don't wanna read the story to keep you away
So tell me what you say
Ain't no way we can leave the place we call home
Even though you're gone, no matter how many we lost
One more friend to bring home a home that I will remember
A baby girl, and her mother was his wife
Then she was killed in a car crash
A boy with a dream, he tried to fly and now he dies
A young girl, who didn't come from the city just to give him a ring
A young man who always said that love was something he would never achieve
And that his life is full of pain
But he could see a way out, and he did in his life
In our day and
In our place
Even though you're gone
Even though you love your time
In our place and we love our time
If it was up to me
なんか悲しい感じの歌詞です。
me at home
I got no money, so it's time to eat
And here I am again as a visitor
On a Saturday night, no one notices me
I wonder to myself, who cares
I'm just a homeless man in a rain jacket
With the sun shining like sunshine
I'm not worried about getting left and right
In the dark darkness of a dark room
Waiting for that man called fate
This might be a sad movie, but reality hits
And it's not the bad guys, no way
But I can't help but smile
Sometimes I wonder what I am
A homeless man in a cold and wet world
If I could I'd be happy
I could have a house, a car, a girlfriend, a wife
A crib and a dog, I'm happy
I'd be all that
I'd be so happy, that's my goal
I wonder what I am
It seems to me, everybody in this world is sick
And maybe in tomorrow people will know
Just what their friends and family can bring
Some will laugh while some will hate
So I guess I live my life
Until I go away, I won't know
If I'm happy or if I'm sad
And I wonder who it is inside me
Sometimes I wonder how I know
Why I had to live in a bad way
終わりに
まぁまぁいい感じにできましたが、なかなか収束しないのがもどかしいです。
韻を踏んでいないのが気になる所です。