python beautifulsoup4 findall_python-beautifulsoup4不返回内容

需要JavaScript来呈现网页内容.使用prerenderio服务是一种从页面获取所需数据的简便方法.

import requests

from bs4 import BeautifulSoup

# the target we want to open

# changed to use prerenderio service

url='http://service.prerender.io/https://hackerone.com/directory?offers_bounties=true&asset_type=URL&order_direction=DESC&order_field=started_accepting_at'

#open with GET method

resp=requests.get(url)

#http_respone 200 means OK status

if resp.status_code==200:

print("Successfully opened the web page")

print("The news are as follow :-

")

# we need a parser,Python built-in HTML parser is enough .

soup=BeautifulSoup(resp.text,'html.parser')

# l is the list which contains all the text i.e news

l=soup.find("tr","spec-directory-entry daisy-table__row fade fade--show")

#now we want to print only the text part of the anchor.

#find all the elements of a, i.e anchor

for i in l:

print(i.text)

else:

print("Error")

上面的代码返回的数据：

Successfully opened the web page

The news are as follow :-

LivestreamManaged

04 / 2019

73

$100

$150-$250

这是仅获取“ Livestream”表行的值的代码.

import requests

from bs4 import BeautifulSoup

# the target we want to open

# changed to use prerenderio service

url='http://service.prerender.io/https://hackerone.com/directory?offers_bounties=true&asset_type=URL&order_direction=DESC&order_field=started_accepting_at'

#open with GET method

resp=requests.get(url)

#http_respone 200 means OK status

if resp.status_code==200:

print("Successfully opened the web page")

print("The news are as follow :-

")

# we need a parser,Python built-in HTML parser is enough .

soup=BeautifulSoup(resp.text,'html.parser')

# l is the list which contains all "tr" tags

l=soup.findAll("tr","spec-directory-entry daisy-table__row fade fade--show")

# looping through the list of table rows

for i in l:

# checking if the current row is for 'Livestream'

if i.find('a').text == 'Livestream':

# printing the row's values except the first "td" tag

for e in i.findAll('td')[1:]:

print(e.text)

else:

print("Error")

结果：

Successfully opened the web page

The news are as follow :-

04 / 2019

73

$100

$150-$250