0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

闇のプログラミングAdvent Calendar 2023

Day 5

ライブドアブログの記事全部文字列で引っこ抜く

Last updated at Posted at 2023-12-04
import requests
from bs4 import BeautifulSoup
from pprint import pprint
import re


def content_get(url):
    html = requests.get(url).content
    soup = BeautifulSoup(html, 'html.parser')
    return soup


def cleanhtml(raw_html):
    cleanr = re.compile("<.*?>")
    cleantext = re.sub(cleanr, "", raw_html)
    return cleantext


url = "https://{id}.livedoor.blog/"
html = requests.get(url).content
soup = BeautifulSoup(html, 'html.parser')
sidebody = soup.find_all('div', {'class': "sidebody"})
pprint(sidebody)
urls = []
for i in sidebody:
    for a_tag in i.find_all('a'):
        url = a_tag.get('href')
        if url:
            urls.append(url)
pprint(urls)
for i in urls:
    true_url = i
    true_content = content_get(true_url)
    true_body = true_content.find('div', {'class': 'article-body-inner'})
    print(true_body.get_text())

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?