choppervsselectolax
Chopper is a tool to extract elements from HTML by preserving ancestors and CSS rules.
Compared to other HTML parsers Chopper is designed to retain original HTML tree but eliminate elements that do not match parsing rules. Meaning, we can parse HTML elements and keep thei structure for machine learning or other tasks where data structure is needed as well as the data value.
selectolax is a fast and lightweight library for parsing HTML and XML documents in Python. It is designed to be a drop-in replacement for the popular BeautifulSoup library, with significantly faster performance.
selectolax uses a Cython-based parser to quickly parse and navigate through HTML and XML documents. It provides a simple and intuitive API for working with the document's structure, similar to BeautifulSoup.
To use selectolax, you first need to install it via pip by running pip install selectolax``.
Once it is installed, you can use theselectolax.html.fromstring()function to parse an HTML document and create a selectolax object.
For example:
selectolax.html.fromstring()from selectolax.parser import HTMLParser
html_string = "<html><body>Hello, World!</body></html>"
root = HTMLParser(html_string).root
print(root.tag) # html
with file-like objects, bytes or file paths,
as well asselectolax.xml.fromstring()`` for parsing XML documents.
Once you have a selectolax object, you can use the select() method to search for elements in the document using CSS selectors,
similar to BeautifulSoup. For example:
body = root.select("body")[0]
print(body.text()) # "Hello, World!"
Like BeautifulSoups find and find_all methods selectolax also supports searching using the search()`` method, which returns the first matching element,
and thesearch_all()`` method, which returns all matching elements.
Example Use
HTML = """
<html>
<head>
<title>Test</title>
</head>
<body>
<div id="header"></div>
<div id="main">
<div class="iwantthis">
HELLO WORLD
<a href="/nope">Do not want</a>
</div>
</div>
<div id="footer"></div>
</body>
</html>
"""
CSS = """
div { border: 1px solid black; }
div#main { color: blue; }
div.iwantthis { background-color: red; }
a { color: green; }
div#footer { border-top: 2px solid red; }
"""
extractor = Extractor.keep('//div[@class="iwantthis"]').discard('//a')
html, css = extractor.extract(HTML, CSS)
# will result in:
html
"""
<html>
<body>
<div id="main">
<div class="iwantthis">
HELLO WORLD
</div>
</div>
</body>
</html>"""
css
"""
div{border:1px solid black;}
div#main{color:blue;}
div.iwantthis{background-color:red;}
"""
from selectolax.parser import HTMLParser
html_string = "<html><body>Hello, World!</body></html>"
root = HTMLParser(html_string).root
print(root.tag) # html
# use css selectors:
body = root.select("body")[0]
print(body.text()) # "Hello, World!"
# find first matching element:
body = root.search("body")
print(body.text()) # "Hello, World!"
# or all matching elements:
html_string = "<html><body><p>paragraph1</p><p>paragraph2</p></body></html>"
root = HTMLParser(html_string).root
for el in root.search_all("p"):
print(el.text())
# will print:
# paragraph 1
# paragraph 2