How to use XPath with BeautifulSoup and Python?

To use XPath with BeautifulSoup and Python, we can replace BeautifulSoup with lxml.

For instance, we write

from lxml import html
import requests

page = requests.get('http://example.com')
tree = html.fromstring(page.content)
names = tree.xpath('//div[@title="name"]/text()')
prices = tree.xpath('//span[@class="price"]/text()')

print('Names: ', names)
print('Prices: ', prices)

to call html.fromstring to parse the HTML string into an object.

Then we call xpath with the XPath to get the items by XPath.

Post Views: 31