parse5vschompjs

MIT 34 7 3,886

419.4 million (month) Jul 03 2013 8.0.0(2026-02-21 19:30:52 ago)

218 1 5 MIT

Jul 30 2007 47.0 thousand (month) 1.4.0(2025-08-04 21:07:54 ago)

parse5 is a Node.js library for parsing and manipulating HTML and XML documents. It is designed to be fast and flexible, and it is commonly used in web scraping and web development projects.

parse5 is used by popular libraries such as Angular, Lit, Cheerio and many more. Unlike Cheerio parse5 is a low level html parsing library that might be useful directly in web scraping without higher level abstraction.

chompjs can be used in web scrapping for turning JavaScript objects embedded in pages into valid Python dictionaries.

In web scraping this is particularly useful for parsing Javascript variables like: python import chompjs js = """ var myObj = { myMethod: function(params) { // ... }, myValue: 100 } """ chompjs.parse_js_object(js, json_params={'strict': False}) {'myMethod': 'function(params) {\n // ...\n }', 'myValue': 100}

In practice this can be used to extract hidden JSON data like data from <script id=__NEXT_DATA__> elements from nextjs (and similar) websites. Unlike json.loads command chompjs can ingest json documents that contain javascript natives like functions making it a super easy way to scrape hidden web data objects.

Example Use

```javascript const parse5 = require("parse5"); // parse string const document = parse5.parse('Hello World!'); console.log(document); // html tree can be traversed as javascript object: const body = document.childNodes[1]; console.log(body.childNodes[0].value); // "Hello World!" // and modified const newElement = parse5.parseFragment('

New Element

'); body.appendChild(newElement.childNodes[0]); console.log(parse5.serialize(document)); ```

```python # basic use import chompjs js = """ var myObj = { myMethod: function(params) { // ... }, myValue: 100 } """ chompjs.parse_js_object(js, json_params={'strict': False}) {'myMethod': 'function(params) {\n // ...\n }', 'myValue': 100} # example how to use with hidden data parsing: import httpx import chompjs from parsel import Selector response = httpx.get("http://example.com") hidden_script = Selector(response.text).css("script#__NEXT_DATA__::text").get() data = chompjs.parse_js_object(hidden_script) print(data['props']) ```