Skip to content

chompjs

178 1 4 MIT
1.2.3 (31 Jan 2024) Jul 30 2007 20.9 thousand (month)

chompjs can be used in web scrapping for turning JavaScript objects embedded in pages into valid Python dictionaries.

In web scraping this is particularly useful for parsing Javascript variables like:

import chompjs
js = """
  var myObj = {
    myMethod: function(params) {
    // ...
    },
    myValue: 100
  }
"""
chompjs.parse_js_object(js, json_params={'strict': False})
{'myMethod': 'function(params) {\n        // ...\n    }', 'myValue': 100}

In practice this can be used to extract hidden JSON data like data from <script id=__NEXT_DATA__> elements from nextjs (and similar) websites. Unlike json.loads command chompjs can ingest json documents that contain javascript natives like functions making it a super easy way to scrape hidden web data objects.

Example Use


# basic use
import chompjs
js = """
  var myObj = {
    myMethod: function(params) {
    // ...
    },
    myValue: 100
  }
"""
chompjs.parse_js_object(js, json_params={'strict': False})
{'myMethod': 'function(params) {\n        // ...\n    }', 'myValue': 100}

# example how to use with hidden data parsing:
import httpx
import chompjs
from parsel import Selector

response = httpx.get("http://example.com")
hidden_script = Selector(response.text).css("script#__NEXT_DATA__::text").get()
data = chompjs.parse_js_object(hidden_script)
print(data['props'])

Alternatives / Similar


- 4.12.3 (4 months ago) Jul 26 2019 compare
2,589 5.2.2 (a month ago) Dec 13 2022 compare
5,395 0.13.0 (2 years ago) Jul 30 2007 compare
1,096 1.1 (3 years ago) Jul 30 2007 compare
284 1.2.0 (1 year, 7 months ago) Apr 14 2012 compare
1,861 6.0.11 (6 months ago) Jun 15 2007 compare
2,274 2.0.0 (1 year, 5 months ago) Dec 05 2008 compare
1,090 1.9.1 (2 months ago) Jul 26 2019 compare
13,605 0.10.0 (5 years ago) Feb 25 2018 compare
608 1.2.1 (1 year, 11 months ago) Jun 09 2011 compare
979 0.3.21 (3 months ago) Mar 01 2018 compare
671 0.4.12 (6 months ago) Jun 03 2007 compare
736 1.1 (3 years ago) Dec 28 2012 compare
22 0.6.0 (1 year, 1 month ago) Jul 24 2014 compare

Other Languages

3,569 7.1.2 (11 months ago) Jul 03 2013 compare
1,074 1.3.0 (8 months ago) Feb 09 2011 compare
4,304 9.1.0 (5 months ago) Aug 28 2011 compare
27,887 1.0.0-rc.12 (7 months ago) Oct 08 2011 compare
6,108 1.16.4 (2 months ago) Jul 25 2009 compare
218 1.3.6 (11 months ago) Apr 20 2015 compare
1,479 1.0.4 (1 year, 9 months ago) Nov 22 2014 compare
1,492 2.9.0 (2 months ago) Jun 01 2013 compare
3,915 v7.1.0-RC1 (a month ago) Sep 26 2011 compare
683 Start (6 years ago) Feb 20 2018 compare
13,635 v1.9.2 (a month ago) Aug 29 2016 compare
701 v1.3.1 (2 months ago) Feb 07 2019 compare
2,134 v1.2.5 (2 years ago) Apr 29 2017 compare
656 (2 months ago) Jun 08 2019 compare
2,065 v4.4.10 (6 months ago) Oct 26 2013 compare
152 2.2.4 (3 years ago) Dec 22 2019 compare
Was this page helpful?