仕事でSeleniumを扱い、色々と調べたことを備忘録も兼ねて書いておく。
言語はPython。

実行環境

この記事に書いてある通り、Docker上でSeleniumを動かしている。

シークレットモードで実行

set_secretmode_sample.py

# Options の設定(ブラウザ設定)
options = webdriver.ChromeOptions()
options.add_argument('--incognito') # シークレットモードに設定する

SeleniumでJavascriptを実行する

execute_script_sample.py

# Selenium Server に接続する
driver = webdriver.Remote(command_executor='http://localhost:4444/wd/hub',
                          desired_capabilities=capabilities,
                          options=options)

# 開発者ツールの「Console」に「hoge」と表示させる
driver.execute_script("console.log('hoge');")

要素の存在・表示・有効確認

存在確認のポイントは、find_element ではなく find_elements を使うこと。
find_element の場合、対象の要素が存在しない場合例外となるが、find_elements の場合は空のリストとなるためサイズで判定可能となる。
表示されているかは、is_displayed()。有効になっているかは、is_enabled()。

find_elements_sample.py

element_id = 'hoge'
elements = driver.find_elements(By.ID, element_id)
if len(elements) > 0:
    element = elements[0]
    if element.is_displayed() and element.is_enabled():
        print(f'{element_id} is displayed and enabled.')
        element.click()
    else:
        print(f'{element_id} is not displayed or enabled.')
else:
    print(f'{element_id} is not found.')

パフォーマンス測定を行う

ブラウザ上でのパフォーマンス測定を自動で行いたくてSeleniumを扱ったが、けっこう苦労したので他のツールを使った方がいいのかもしれない。
参考にしたのは以下ページ。

https://stackoverflow.com/questions/52633697/selenium-python-how-to-capture-network-traffics-response
https://qiita.com/nextpenguin/items/4827fb1da062581518e7

performanceログを有効にすることで、開発者ツールの「Network」に表示されるような情報がログとして取得できる。

get_log_sample.py

# Capabilityの設定
capabilities = webdriver.DesiredCapabilities.CHROME.copy()
capabilities['goog:loggingPrefs'] = {'performance': 'ALL'}  # performanceログを有効化

for log in driver.get_log('performance'):
    print(log)

出力例

{'level': 'INFO', 'message': '{"message":{"method":"Page.frameResized","params":{}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384599}
{'level': 'INFO', 'message': '{"message":{"method":"Network.loadingFinished","params":{"encodedDataLength":0,"requestId":"5209C4FC23FD75E606224D419A0AFEE7","shouldReportCorbBlocking":false,"timestamp":54550.15917}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384604}
{'level': 'INFO', 'message': '{"message":{"method":"Page.loadEventFired","params":{"timestamp":54550.182073}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384604}
{'level': 'INFO', 'message': '{"message":{"method":"Page.frameStoppedLoading","params":{"frameId":"F6BB344875CC9300FE1BFF97A9BB9A85"}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384604}
{'level': 'INFO', 'message': '{"message":{"method":"Page.domContentEventFired","params":{"timestamp":54550.18242}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384604}
{'level': 'INFO', 'message': '{"message":{"method":"Page.frameResized","params":{}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384824}
{'level': 'INFO', 'message': '{"message":{"method":"Page.frameResized","params":{}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384824}
{'level': 'INFO', 'message': '{"message":{"method":"Network.requestWillBeSent","params":{"documentURL":"https://www.google.co.jp/","frameId":"F6BB344875CC9300FE1BFF97A9BB9A85","hasUserGesture":false,"initiator":{"type":"other"},"loaderId":"94F1D64DEC188CE1ACDCFC9B570C6924","request":{"headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36","sec-ch-ua":"\\"Chromium\\";v=\\"92\\", \\" Not A;Brand\\";v=\\"99\\", \\"Google Chrome\\";v=\\"92\\"","sec-ch-ua-mobile":"?0"},"initialPriority":"VeryHigh","method":"GET","mixedContentType":"none","referrerPolicy":"strict-origin-when-cross-origin","url":"https://www.google.co.jp/"},"requestId":"94F1D64DEC188CE1ACDCFC9B570C6924","timestamp":54550.454592,"type":"Document","wallTime":1631801384.870629}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384870}{'level': 'INFO', 'message': '{"message":{"method":"Network.requestWillBeSentExtraInfo","params":{"associatedCookies":[],"headers":{":authority":"www.google.co.jp",":method":"GET",":path":"/",":scheme":"https","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","sec-ch-ua":"\\"Chromium\\";v=\\"92\\", \\" Not A;Brand\\";v=\\"99\\", \\"Google Chrome\\";v=\\"92\\"","sec-ch-ua-mobile":"?0","sec-fetch-dest":"document","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36","x-client-data":"CJCBywE="},"requestId":"94F1D64DEC188CE1ACDCFC9B570C6924"}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801384965}
{'level': 'INFO', 'message': '{"message":{"method":"Network.responseReceivedExtraInfo","params":{"blockedCookies":[],"headers":{"alt-svc":"h3=\\":443\\"; ma=2592000,h3-29=\\":443\\"; ma=2592000,h3-T051=\\":443\\"; ma=2592000,h3-Q050=\\":443\\"; ma=2592000,h3-Q046=\\":443\\"; ma=2592000,h3-Q043=\\":443\\"; ma=2592000,quic=\\":443\\"; ma=2592000; v=\\"46,43\\"","bfcache-opt-in":"unload","cache-control":"private, max-age=0","content-encoding":"br","content-length":"39494","content-type":"text/html; charset=UTF-8","date":"Thu, 16 Sep 2021 14:09:45 GMT","expires":"-1","p3p":"CP=\\"This is not a P3P policy! See g.co/p3phelp for more info.\\"","server":"gws","set-cookie":"1P_JAR=2021-09-16-14; expires=Sat, 16-Oct-2021 14:09:45 GMT; path=/; domain=.google.co.jp; Secure; SameSite=none\\nNID=223=mCqes7W1pPbU-ZL6zI7cN4JStT7P8Tv9OwHHgcDdNajXwqdsMttfnsSySMI4_wRIe6Bn2M7KJ8nNobn2Y394hCL7eEfUNMvdLhf7cPg2hnES-Ad1fDkdyh3Oj2DxsqFDH_9hCkwOIb85GFvOoKhyI6K3I7CeYZmRP0b2pXC36YE; expires=Fri, 18-Mar-2022 14:09:45 GMT; path=/; domain=.google.co.jp; Secure; HttpOnly; SameSite=none","strict-transport-security":"max-age=31536000","x-frame-options":"SAMEORIGIN","x-xss-protection":"0"},"requestId":"94F1D64DEC188CE1ACDCFC9B570C6924","resourceIPAddressSpace":"Public"}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801385075}
{'level': 'INFO', 'message': '{"message":{"method":"Network.responseReceived","params":{"frameId":"F6BB344875CC9300FE1BFF97A9BB9A85","loaderId":"94F1D64DEC188CE1ACDCFC9B570C6924","requestId":"94F1D64DEC188CE1ACDCFC9B570C6924","response":{"connectionId":27,"connectionReused":false,"encodedDataLength":682,"fromDiskCache":false,"fromPrefetchCache":false,"fromServiceWorker":false,"headers":{"alt-svc":"h3=\\":443\\"; ma=2592000,h3-29=\\":443\\"; ma=2592000,h3-T051=\\":443\\"; ma=2592000,h3-Q050=\\":443\\"; ma=2592000,h3-Q046=\\":443\\"; ma=2592000,h3-Q043=\\":443\\"; ma=2592000,quic=\\":443\\"; ma=2592000; v=\\"46,43\\"","bfcache-opt-in":"unload","cache-control":"private, max-age=0","content-encoding":"br","content-length":"39494","content-type":"text/html; charset=UTF-8","date":"Thu, 16 Sep 2021 14:09:45 GMT","expires":"-1","p3p":"CP=\\"This is not a P3P policy! See g.co/p3phelp for more info.\\"","server":"gws","set-cookie":"1P_JAR=2021-09-16-14; expires=Sat, 16-Oct-2021 14:09:45 GMT; path=/; domain=.google.co.jp; Secure; SameSite=none\\nNID=223=mCqes7W1pPbU-ZL6zI7cN4JStT7P8Tv9OwHHgcDdNajXwqdsMttfnsSySMI4_wRIe6Bn2M7KJ8nNobn2Y394hCL7eEfUNMvdLhf7cPg2hnES-Ad1fDkdyh3Oj2DxsqFDH_9hCkwOIb85GFvOoKhyI6K3I7CeYZmRP0b2pXC36YE; expires=Fri, 18-Mar-2022 14:09:45 GMT; path=/; domain=.google.co.jp; Secure; HttpOnly; SameSite=none","strict-transport-security":"max-age=31536000","x-frame-options":"SAMEORIGIN","x-xss-protection":"0"},"mimeType":"text/html","protocol":"h2","remoteIPAddress":"142.250.196.131","remotePort":443,"requestHeaders":{":authority":"www.google.co.jp",":method":"GET",":path":"/",":scheme":"https","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","sec-ch-ua":"\\"Chromium\\";v=\\"92\\", \\" Not A;Brand\\";v=\\"99\\", \\"Google Chrome\\";v=\\"92\\"","sec-ch-ua-mobile":"?0","sec-fetch-dest":"document","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36","x-client-data":"CJCBywE="},"responseTime":1.631801385074467e+12,"securityDetails":{"certificateId":0,"certificateTransparencyCompliance":"compliant","cipher":"AES_128_GCM","issuer":"GTS CA 1C3","keyExchange":"","keyExchangeGroup":"X25519","protocol":"TLS 1.3","sanList":["*.google.co.jp","google.co.jp"],"signedCertificateTimestampList":[{"hashAlgorithm":"SHA-256","logDescription":"Google \'Xenon2021\' log","logId":"7D3EF2F88FFF88556824C2C0CA9E5289792BC50E78097F2E6A9768997E22F0D7","origin":"Embedded in certificate","signatureAlgorithm":"ECDSA","signatureData":"304402201D45EBC2DCE861F94D034D136CF1DCD9EB2E874D64AA965692B2EAA94917254402200BA7A2CD36235026830D597EF2C974B1471449D0D6957423D6C925EC672453B9","status":"Verified","timestamp":1.630301689401e+12},{"hashAlgorithm":"SHA-256","logDescription":"DigiCert Yeti2021 Log","logId":"5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA","origin":"Embedded in certificate","signatureAlgorithm":"ECDSA","signatureData":"3045022017F5289BF579D271C0C6D689D873E435EB8521CD547FAC9CE7833F897AB94F24022100951012654261E0E7F05F09083CCAC0AFE3850B873BCA069ED24877DDAF6167EF","status":"Verified","timestamp":1.630301689379e+12}],"subjectName":"*.google.co.jp","validFrom":1630298088,"validTo":1637555687},"securityState":"secure","status":200,"statusText":"","timing":{"connectEnd":91.857,"connectStart":35.885,"dnsEnd":35.885,"dnsStart":35.873,"proxyEnd":-1,"proxyStart":-1,"pushEnd":0,"pushStart":0,"receiveHeadersEnd":202.589,"requestTime":54550.456082,"sendEnd":92.808,"sendStart":92.622,"sslEnd":91.85,"sslStart":45.083,"workerFetchStart":-1,"workerReady":-1,"workerRespondWithSettled":-1,"workerStart":-1},"url":"https://www.google.co.jp/"},"timestamp":54550.660791,"type":"Document"}},"webview":"F6BB344875CC9300FE1BFF97A9BB9A85"}', 'timestamp': 1631801385077}

こんな感じのJson形式になっている。
注目するのは、「"method":"Network.requestWillBeSent"」。このレコードがリクエスト送信時のログ。
逆にレスポンスを受け取った時のログは、「"method":"Network.responseReceived"」になっている（たぶん）。
リクエスト送信時のログ中の"requestId"を使えばレスポンスとの紐付けを行える。

実際に計測してみると開発者ツールのNetworkの情報とそう変わらない時間になったので思考停止したけど、これらのmethodの意味をちゃんと理解せず雰囲気で実装したので、誤っていそうな気も…

詳しい方がいれば教えて下さい！！

以上です。