[Survey] Kaggle - Quora 14位解法まとめ

Kaggle - Quora Question Pairs¹の14位解法²の調査記事です．

Author: Wojtek Rosinski
Title: 14th Place Solution - Code
Kaggle Discussion: https://www.kaggle.com/c/quora-question-pairs/discussion/34549
Code: https://github.com/Wrosinski/Kaggle-Quora

特徴量

Data Encoding

• Textacy³を用いたテキストクリーニング
• 品詞解析
• ステミング
• NER Encoding

NLP

• カーネル上の特徴量(by Abhishek⁴, SRK^{5 6}, Jared Turkewitz⁷, the_1owl⁸, Mephistopheles⁹など)
• LSIやLDA, tSVD
• Word2Vec
• Doc2Vec
• 様々な距離や類似度
• テキスト関連の特徴量
• KNNを用いた特徴量

リーク情報

• Jared Turkewitz's frequency features
• NetworkX features

モデル

• XGBoost & LightGBM
• Decomposable Attention¹⁰(SpaCy¹¹)
• LSTM
• MLP

Stacking

• sklearn modelのアンサンブル
• LightGBMでStacking
• spacyの最小化関数を用いてアンサンブルに対する重みを算出

References

---

1. Kaggle, Quora Question Pairs, 2017. ↩

2. Rosinski, 14th Place Solution - Code ↩

3. charbeat-labs, textacy ↩

4. radder, Abhishek's features ↩

5. SRK, Simple Leaky Exploration Notebook - Quora ↩

6. SRK, Some interesting solutions from the web ↩

7. Jared Turkewitz, Magic Features (0.03 gain) ↩

8. the_1owl, Matching ¿Que? for Quora - End to End 0.33719 PB ↩

9. Mephistopheles, 0.29936 solution ↩

10. Parikh et al., A Decomposable Attention Model for Natural Language Inference, 2016. ↩

11. spacy, https://github.com/explosion/spaCy/tree/master/examples/keras_parikh_entailment ↩