Colaboratory から、大きなサイズのファイルを、PyDrive を使って Google ドライブにアップロードしようとすると、RedirectMissingLocation 例外が発生して失敗する。
回避方法はこちら。
#問題の詳細
サンプルコードは以下のリンクを参照。
外部データ: ローカル ファイル、ドライブ、スプレッドシート、Cloud Storage - Colaboratory
> Google ドライブ > PyDrive
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
なお、投稿時点（2021.7.17）でインストールされていた PyDrive のバージョンは 1.3.1（詳細）。
Colaboratory 上にあるファイルを Google ドライブにアップロードするには、次のようにする。
dfile = drive.CreateFile()
dfile.SetContentFile("path.to.large.file")
dfile.Upload()
ここで path.to.large.file のファイルサイズが大きいと、RedirectMissingLocation 例外が発生して失敗する（詳細）。
RedirectMissingLocation: Redirected but the response is missing a Location: header.
#原因と回避方法
原因は PyDrive が依存している httplib2 のバグのようである。
詳細は以下の issue を参照。
httplib2 v0.16.0 breaks the library · Issue #803 · googleapis/google-api-python-client · GitHub
この issue に書かれているように、httplib2 をダウングレードすることで回避できる可能性がある。
なお、投稿時点でインストールされていた httplib2 のバージョンは 0.17.4（詳細）。
pip install -U httplib2==0.15.0
ダウングレードが終わったら、ランタイムを再起動すること。
#例外の詳細
---------------------------------------------------------------------------
RedirectMissingLocation Traceback (most recent call last)
<ipython-input-34-d2b7f6f09d2e> in <module>()
1 dfile = drive.CreateFile()
2 dfile.SetContentFile("path.to.large.file")
----> 3 dfile.Upload()
10 frames
/usr/local/lib/python3.7/dist-packages/pydrive/files.py in Upload(self, param)
283 self._FilesPatch(param=param)
284 else:
--> 285 self._FilesInsert(param=param)
286
287 def Trash(self, param=None):
/usr/local/lib/python3.7/dist-packages/pydrive/auth.py in _decorated(self, *args, **kwargs)
73 self.http = self.auth.Get_Http_Object()
74
---> 75 return decoratee(self, *args, **kwargs)
76 return _decorated
77
/usr/local/lib/python3.7/dist-packages/pydrive/files.py in _FilesInsert(self, param)
367 param['media_body'] = self._BuildMediaBody()
368 metadata = self.auth.service.files().insert(**param).execute(
--> 369 http=self.http)
370 except errors.HttpError as error:
371 raise ApiRequestError(error)
/usr/local/lib/python3.7/dist-packages/googleapiclient/_helpers.py in positional_wrapper(*args, **kwargs)
132 elif positional_parameters_enforcement == POSITIONAL_WARNING:
133 logger.warning(message)
--> 134 return wrapped(*args, **kwargs)
135
136 return positional_wrapper
/usr/local/lib/python3.7/dist-packages/googleapiclient/http.py in execute(self, http, num_retries)
877 body = None
878 while body is None:
--> 879 _, body = self.next_chunk(http=http, num_retries=num_retries)
880 return body
881
/usr/local/lib/python3.7/dist-packages/googleapiclient/_helpers.py in positional_wrapper(*args, **kwargs)
132 elif positional_parameters_enforcement == POSITIONAL_WARNING:
133 logger.warning(message)
--> 134 return wrapped(*args, **kwargs)
135
136 return positional_wrapper
/usr/local/lib/python3.7/dist-packages/googleapiclient/http.py in next_chunk(self, http, num_retries)
1056 try:
1057 resp, content = http.request(
-> 1058 self.resumable_uri, method="PUT", body=data, headers=headers
1059 )
1060 except:
/usr/local/lib/python3.7/dist-packages/oauth2client/transport.py in new_request(uri, method, body, headers, redirections, connection_type)
173 resp, content = request(orig_request_method, uri, method, body,
174 clean_headers(headers),
--> 175 redirections, connection_type)
176
177 # A stored token may expire between the time it is retrieved and
/usr/local/lib/python3.7/dist-packages/oauth2client/transport.py in request(http, uri, method, body, headers, redirections, connection_type)
280 return http_callable(uri, method=method, body=body, headers=headers,
281 redirections=redirections,
--> 282 connection_type=connection_type)
283
284
/usr/local/lib/python3.7/dist-packages/httplib2/__init__.py in request(self, uri, method, body, headers, redirections, connection_type)
1989 headers,
1990 redirections,
-> 1991 cachekey,
1992 )
1993 except Exception as e:
/usr/local/lib/python3.7/dist-packages/httplib2/__init__.py in _request(self, conn, host, absolute_uri, request_uri, method, body, headers, redirections, cachekey)
1688 ),
1689 response,
-> 1690 content,
1691 )
1692 # Fix-up relative redirects (which violate an RFC 2616 MUST)
RedirectMissingLocation: Redirected but the response is missing a Location: header.
#バージョンの詳細
PyDrive==1.3.1
- google-api-python-client [required: >=1.2, installed: 1.12.8]
- google-api-core [required: >=1.21.0,<2dev, installed: 1.26.3]
- google-auth [required: >=1.21.1,<2.0dev, installed: 1.32.1]
- cachetools [required: >=2.0.0,<5.0, installed: 4.2.2]
- pyasn1-modules [required: >=0.2.1, installed: 0.2.8]
- pyasn1 [required: >=0.4.6,<0.5.0, installed: 0.4.8]
- rsa [required: >=3.1.4,<5, installed: 4.7.2]
- pyasn1 [required: >=0.1.3, installed: 0.4.8]
- setuptools [required: >=40.3.0, installed: 57.2.0]
- six [required: >=1.9.0, installed: 1.15.0]
- googleapis-common-protos [required: >=1.6.0,<2.0dev, installed: 1.53.0]
- protobuf [required: >=3.12.0, installed: 3.17.3]
- six [required: >=1.9, installed: 1.15.0]
- packaging [required: >=14.3, installed: 21.0]
- pyparsing [required: >=2.0.2, installed: 2.4.7]
- protobuf [required: >=3.12.0, installed: 3.17.3]
- six [required: >=1.9, installed: 1.15.0]
- pytz [required: Any, installed: 2018.9]
- requests [required: >=2.18.0,<3.0.0dev, installed: 2.23.0]
- certifi [required: >=2017.4.17, installed: 2021.5.30]
- chardet [required: >=3.0.2,<4, installed: 3.0.4]
- idna [required: >=2.5,<3, installed: 2.10]
- urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.24.3]
- setuptools [required: >=40.3.0, installed: 57.2.0]
- six [required: >=1.13.0, installed: 1.15.0]
- google-auth [required: >=1.16.0, installed: 1.32.1]
- cachetools [required: >=2.0.0,<5.0, installed: 4.2.2]
- pyasn1-modules [required: >=0.2.1, installed: 0.2.8]
- pyasn1 [required: >=0.4.6,<0.5.0, installed: 0.4.8]
- rsa [required: >=3.1.4,<5, installed: 4.7.2]
- pyasn1 [required: >=0.1.3, installed: 0.4.8]
- setuptools [required: >=40.3.0, installed: 57.2.0]
- six [required: >=1.9.0, installed: 1.15.0]
- google-auth-httplib2 [required: >=0.0.3, installed: 0.0.4]
- google-auth [required: Any, installed: 1.32.1]
- cachetools [required: >=2.0.0,<5.0, installed: 4.2.2]
- pyasn1-modules [required: >=0.2.1, installed: 0.2.8]
- pyasn1 [required: >=0.4.6,<0.5.0, installed: 0.4.8]
- rsa [required: >=3.1.4,<5, installed: 4.7.2]
- pyasn1 [required: >=0.1.3, installed: 0.4.8]
- setuptools [required: >=40.3.0, installed: 57.2.0]
- six [required: >=1.9.0, installed: 1.15.0]
- httplib2 [required: >=0.9.1, installed: 0.17.4]
- six [required: Any, installed: 1.15.0]
- httplib2 [required: >=0.15.0,<1dev, installed: 0.17.4]
- six [required: >=1.13.0,<2dev, installed: 1.15.0]
- uritemplate [required: >=3.0.0,<4dev, installed: 3.0.1]
- oauth2client [required: >=4.0.0, installed: 4.1.3]
- httplib2 [required: >=0.9.1, installed: 0.17.4]
- pyasn1 [required: >=0.1.7, installed: 0.4.8]
- pyasn1-modules [required: >=0.0.5, installed: 0.2.8]
- pyasn1 [required: >=0.4.6,<0.5.0, installed: 0.4.8]
- rsa [required: >=3.1.4, installed: 4.7.2]
- pyasn1 [required: >=0.1.3, installed: 0.4.8]
- six [required: >=1.6.1, installed: 1.15.0]
- PyYAML [required: >=3.0, installed: 3.13]