2
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

MultiDict: クローラでフォームPOSTする時にはまるなど

Last updated at Posted at 2014-09-21

MultiDict

複数選択があり得るので、werkzeug のMultiDictを使ってみる:

$ pip install werkzeug

リストからdict:

>>> from werkzeug.datastructures import MultiDict
>>> MultiDict([('a',1), ('a', 2), ('a', 3)])
MultiDict([('a', 1), ('a', 2), ('a', 3)])

追加:

>>> d.update({'a': '4'})
>>> d
MultiDict([('a', 1), ('a', 2), ('a', 3), ('a', '4')])

更新

>>> d['a']=5
>>> d
MultiDict([('a', 5)])

BeautifulSoupでフォームデータを取得

BeautifulSoupのselect()でHTMLからフォームを取得:

    def form_data(self, form_node):
        ''' form_node : HTMLからselect()したformノード
        '''

        data = MultiDict(list(
           (t['name'], t.get('value', '')) 
           for i in ['text', 'hidden', 'password', ]
           for t in form_node.select('input[type=%s]' % i)
        ) + list(
           (t['name'], t.text) 
           for t in form_node.select('textarea')
        ) + list(
           (t['name'], t.get('value', '')) 
           for t in form_node.select('input[checked]' % i)
        ) + list(
            (s.parent['name'], s.get('value', s.text))
            for s in form_node.select('option[selected]')
        ))  

        # TODO: "file", "image", 
        # TODO: HTML5 -- input(list) and datalist

        return data

requestsでPOSTすると最初のパラメータしかわたっていない

requestsはMultiDictに対応しているということなのだが、手持ちの環境で動かない。。。

しょうがないので、とりあえずdictに変換して対応:

>>> dict(d.lists())
{'a': [1, 2, 3, '4']}
    def post(self, uri, data):

        if isinstance(data, MultiDict):
            data = dict(data.lists())

        self.res = self.req.post(
            uri, data=data,
            verify=False, allow_redirects=False)
        self.last_uri = uri
2
2
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
2
2

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?