cheeriovschopper
cheerio is a popular JavaScript library that allows you to interact with and manipulate HTML and XML documents in a similar way to how you would with jQuery in a browser. It is a fast, flexible, and lean implementation of core jQuery designed specifically for the server.
One of the main benefits of using cheerio is that it allows you to use jQuery-like syntax to navigate and m anipulate the Document Object Model (DOM) of an HTML or XML document, making it easy to work with.
cheerio supports CSS selectors though not XPath.
Chopper is a tool to extract elements from HTML by preserving ancestors and CSS rules.
Compared to other HTML parsers Chopper is designed to retain original HTML tree but eliminate elements that do not match parsing rules. Meaning, we can parse HTML elements and keep thei structure for machine learning or other tasks where data structure is needed as well as the data value.
Example Use
const cheerio = require('cheerio');
const $ = cheerio.load('<html><head><title>My title</title></head><body><h1 class='name'>Hello World!</h1></body></html>');
// use css selectors
console.log($('title').text()); // My title
console.log($('.name').text()); // Hello World!
// select multiple elements
const $ = cheerio.load('<html><body><ul><li>item 1</li><li>item 2</li></ul></body></html>');
$('li').each(function(i, elem) {
console.log($(this).text());
});
// modify elements
const $ = cheerio.load('<html><body><h1>Hello World!</h1></body></html>');
$('h1').text('Hello, Cheerio!');
console.log($.html());
HTML = """
<html>
<head>
<title>Test</title>
</head>
<body>
<div id="header"></div>
<div id="main">
<div class="iwantthis">
HELLO WORLD
<a href="/nope">Do not want</a>
</div>
</div>
<div id="footer"></div>
</body>
</html>
"""
CSS = """
div { border: 1px solid black; }
div#main { color: blue; }
div.iwantthis { background-color: red; }
a { color: green; }
div#footer { border-top: 2px solid red; }
"""
extractor = Extractor.keep('//div[@class="iwantthis"]').discard('//a')
html, css = extractor.extract(HTML, CSS)
# will result in:
html
"""
<html>
<body>
<div id="main">
<div class="iwantthis">
HELLO WORLD
</div>
</div>
</body>
</html>"""
css
"""
div{border:1px solid black;}
div#main{color:blue;}
div.iwantthis{background-color:red;}
"""