chompjsvscascadia

MIT 5 1 218

47.0 thousand (month) Jul 30 2007 1.4.0(2025-08-04 21:07:54 ago)

754 1 1 BSD-2-Clause

Feb 20 2018 58.1 thousand (month) Start(2018-02-20 18:47:44 ago)

chompjs can be used in web scrapping for turning JavaScript objects embedded in pages into valid Python dictionaries.

In web scraping this is particularly useful for parsing Javascript variables like: python import chompjs js = """ var myObj = { myMethod: function(params) { // ... }, myValue: 100 } """ chompjs.parse_js_object(js, json_params={'strict': False}) {'myMethod': 'function(params) {\n // ...\n }', 'myValue': 100}

In practice this can be used to extract hidden JSON data like data from <script id=__NEXT_DATA__> elements from nextjs (and similar) websites. Unlike json.loads command chompjs can ingest json documents that contain javascript natives like functions making it a super easy way to scrape hidden web data objects.

cascadia is a library for Go that provides a CSS selector engine, allowing you to use CSS selectors to select elements from an HTML document.

It is built on top of the html package in the Go standard library, and provides a more efficient and powerful way to select elements from an HTML document.

Example Use

```python # basic use import chompjs js = """ var myObj = { myMethod: function(params) { // ... }, myValue: 100 } """ chompjs.parse_js_object(js, json_params={'strict': False}) {'myMethod': 'function(params) {\n // ...\n }', 'myValue': 100} # example how to use with hidden data parsing: import httpx import chompjs from parsel import Selector response = httpx.get("http://example.com") hidden_script = Selector(response.text).css("script#__NEXT_DATA__::text").get() data = chompjs.parse_js_object(hidden_script) print(data['props']) ```

```go package main import ( "fmt" "github.com/andybalholm/cascadia" "golang.org/x/net/html" "strings" ) func main() { // Create an HTML string html := `

Hello, World!

Example

` // Parse the HTML string into a node tree doc, err := html.Parse(strings.NewReader(html)) if err != nil { fmt.Println("Error:", err) return } // Compile the CSS selector sel, err := cascadia.Compile("p") if err != nil { fmt.Println("Error:", err) return } // Use the Selector.Match method to select elements from the document matches := sel.Match(doc) if len(matches) > 0 { fmt.Println(matches[0].FirstChild.Data) // > Hello, World! } } ```