Python Libraries Used in Web Scraping html-extractor html2text 1,897 2024.2.26 (1 year, 6 months ago) Dec 14 2008 extruct 884 0.18.0 (9 months ago) Oct 27 2015 newspaper 14,364 0.2.8 (6 years ago) Dec 28 2012 trafilatura 3,791 2.0.0 (8 months ago) Jul 17 2019 readability 2,724 0.8.1 (5 years ago) Jun 30 2011 sumy 3,548 0.11.0 (2 years ago) Oct 20 2013 extractnet 263 2.0.7 (2 years ago) Dec 11 2020 photon 11,149 1.1.9 (6 years ago) Aug 24 2018 text-parsing sumy 3,548 0.11.0 (2 years ago) Oct 20 2013 scrapy scrapy 54,211 2.12.0 (9 months ago) Jul 26 2019 scrapyd 2,980 1.5.0 (10 months ago) Sep 04 2013 scrapydweb 3,218 1.6.0 (6 months ago) Sep 30 2018 gerapy 3,365 0.9.13 (2 years ago) Jul 04 2017 splash 4,122 3.5 (5 years ago) Apr 25 2014 framework scrapy 54,211 2.12.0 (9 months ago) Jul 26 2019 scrapyd 2,980 1.5.0 (10 months ago) Sep 04 2013 autoscraper 6,638 1.1.14 (3 years ago) Jul 26 2019 gracy 247 1.34.0 (8 months ago) Feb 05 2023 scrapydweb 3,218 1.6.0 (6 months ago) Sep 30 2018 gerapy 3,365 0.9.13 (2 years ago) Jul 04 2017 ruia 1,754 0.8.5 (2 years ago) Oct 17 2018 photon 11,149 1.1.9 (6 years ago) Aug 24 2018 dude 428 0.1.3 (2 years ago) Feb 20 2022 gui scrapydweb 3,218 1.6.0 (6 months ago) Sep 30 2018 gerapy 3,365 0.9.13 (2 years ago) Jul 04 2017 html-parser beautifulsoup - 4.13.3 (6 months ago) Jul 26 2019 lxml 2,737 5.3.0 (1 year, 18 days ago) Dec 13 2022 xmltodict 5,577 0.14.2 (10 months ago) Jul 30 2007 html5lib 1,153 1.1 (5 years ago) Jul 30 2007 cssselect 293 1.2.0 (2 years ago) Apr 14 2012 feedparser 2,048 6.0.11 (1 year, 8 months ago) Jun 15 2007 pyquery 2,312 2.0.1 (11 months ago) Dec 05 2008 parsel 1,187 1.10.0 (7 months ago) Jul 26 2019 requests-html 13,780 0.10.0 (6 years ago) Feb 25 2018 selectolax 1,186 0.3.27 (8 months ago) Mar 01 2018 untangle 619 1.2.1 (3 years ago) Jun 09 2011 html5-parser 683 0.4.12 (1 year, 9 months ago) Jun 03 2007 chompjs 202 1.3.0 (1 year, 11 days ago) Jul 30 2007 gazpacho 764 1.1 (4 years ago) Dec 28 2012 chopper 22 0.6.0 (2 years ago) Jul 24 2014 data jmespath 2,211 1.0.1 (3 years ago) Feb 09 2022 jsonpath-ng 636 1.7.0 (9 months ago) Feb 09 2022 xhtml2pdf 2,280 0.2.16 (1 year, 2 months ago) May 19 2011 nested-lookup 209 0.2.25 (3 years ago) Feb 09 2022 json-parser jmespath 2,211 1.0.1 (3 years ago) Feb 09 2022 jsonpath-ng 636 1.7.0 (9 months ago) Feb 09 2022 nested-lookup 209 0.2.25 (3 years ago) Feb 09 2022 jmespath jmespath 2,211 1.0.1 (3 years ago) Feb 09 2022 selenium selenium 31,604 4.28.1 (7 months ago) Apr 25 2008 undetected-chromedriver 10,683 3.5.5 (1 year, 6 months ago) Sep 04 2020 requestium 1,834 0.4.0 (1 year, 6 months ago) Dec 28 2012 selenium-driverless 718 1.9.4 (10 months ago) Jul 22 2022 browser-automation selenium 31,604 4.28.1 (7 months ago) Apr 25 2008 playwright 12,131 1.49.1 (8 months ago) Feb 24 2021 undetected-chromedriver 10,683 3.5.5 (1 year, 6 months ago) Sep 04 2020 requestium 1,834 0.4.0 (1 year, 6 months ago) Dec 28 2012 selenium-driverless 718 1.9.4 (10 months ago) Jul 22 2022 splash 4,122 3.5 (5 years ago) Apr 25 2014 stealth curl-impersonate 4,221 v0.6.1 (1 year, 5 months ago) Feb 23 2022 undetected-chromedriver 10,683 3.5.5 (1 year, 6 months ago) Sep 04 2020 curl-cffi 1,751 0.7.1 (1 year, 1 month ago) Feb 23 2022 hrequests 780 0.9.2 (8 months ago) Feb 23 2022 selenium-driverless 718 1.9.4 (10 months ago) Jul 22 2022 blocking ua-parser 596 1.0.1 (6 months ago) Dec 29 2012 cloudscraper 4,683 1.2.71 (2 years ago) Dec 28 2012 http ua-parser 596 1.0.1 (6 months ago) Dec 29 2012 headers ua-parser 596 1.0.1 (6 months ago) Dec 29 2012 http-client requests 52,519 2.32.3 (1 year, 2 months ago) Feb 14 2011 aiohttp 15,425 3.11.12 (6 months ago) Jul 26 2019 httpx 13,703 0.28.1 (8 months ago) Jul 26 2019 curl-impersonate 4,221 v0.6.1 (1 year, 5 months ago) Feb 23 2022 pycurl 1,094 7.45.4 (8 months ago) Feb 25 2003 curl-cffi 1,751 0.7.1 (1 year, 1 month ago) Feb 23 2022 treq 590 24.9.1 (11 months ago) Dec 28 2012 hrequests 780 0.9.2 (8 months ago) Feb 23 2022 scraper trafilatura 3,791 2.0.0 (8 months ago) Jul 17 2019 youtube-dl 134,254 2021.12.17 (3 years ago) Feb 22 2012 you-get 54,934 0.4.1743 (7 months ago) Sep 01 2012 cli-tool youtube-dl 134,254 2021.12.17 (3 years ago) Feb 22 2012 you-get 54,934 0.4.1743 (7 months ago) Sep 01 2012 educational cloudscraper 4,683 1.2.71 (2 years ago) Dec 28 2012 youtube-dl 134,254 2021.12.17 (3 years ago) Feb 22 2012 you-get 54,934 0.4.1743 (7 months ago) Sep 01 2012 html-utils bleach 2,663 6.2.0 (9 months ago) Feb 21 2010 seo advertools 1,186 0.16.4 (8 months ago) Dec 28 2012 http-utils httptools 1,229 0.6.4 (10 months ago) Dec 28 2012 requests-cache 1,369 1.2.1 (1 year, 2 months ago) Feb 14 2011 jsonpath jsonpath-ng 636 1.7.0 (9 months ago) Feb 09 2022 css cssutils 84 2.11.1 (1 year, 2 months ago) Aug 04 2007 captcha captcha_solver 244 0.1.5 (6 years ago) Feb 17 2015 unicaps 227 1.3.0 (1 year, 5 months ago) Jul 27 2020 scrapers crosslinked 1,346 0.3.0 (2 years ago) Feb 14 2011 microformats extruct 884 0.18.0 (9 months ago) Oct 27 2015 data-output xhtml2pdf 2,280 0.2.16 (1 year, 2 months ago) May 19 2011 playwright playwright 12,131 1.49.1 (8 months ago) Feb 24 2021 Was this page helpful? Thanks for your feedback! Thanks for your feedback! Feel free to email suggestions to hello@webscraping.fyi