Python Libraries Used in Web Scraping captcha captcha_solver 227 0.1.5 (4 years ago) Feb 17 2015 unicaps 179 1.2.2 (2 months ago) Jul 27 2020 html-parser lxml 2,484 4.9.3 (4 months ago) Dec 13 2022 beautifulsoup - 4.12.2 (7 months ago) Jul 26 2019 xmltodict 5,269 0.13.0 (1 year, 6 months ago) Jul 30 2007 html5lib 1,072 1.1 (3 years ago) Jul 30 2007 cssselect 275 1.2.0 (1 year, 1 month ago) Apr 14 2012 feedparser 1,734 6.0.10 (1 year, 6 months ago) Jun 15 2007 pyquery 2,243 2.0.0 (11 months ago) Dec 05 2008 parsel 1,020 1.8.1 (7 months ago) Jul 26 2019 requests-html 13,416 0.10.0 (4 years ago) Feb 25 2018 html5-parser 655 0.4.12 (10 days ago) Jun 03 2007 untangle 605 1.2.1 (1 year, 4 months ago) Jun 09 2011 selectolax 895 0.3.17 (a month ago) Mar 01 2018 chompjs 159 1.2.2 (5 months ago) Jul 30 2007 gazpacho 716 1.1 (3 years ago) Dec 28 2012 chopper 22 0.6.0 (7 months ago) Jul 24 2014 html-extractor html2text 1,518 2020.1.16 (3 years ago) Dec 14 2008 extruct 783 0.16.0 (4 months ago) Oct 27 2015 trafilatura 2,210 1.6.2 (2 months ago) Jul 17 2019 newspaper 13,294 0.2.8 (5 years ago) Dec 28 2012 readability 2,386 0.8.1 (3 years ago) Jun 30 2011 sumy 3,289 0.11.0 (1 year, 1 month ago) Oct 20 2013 photon 10,120 1.1.9 (5 years ago) Aug 24 2018 extractnet 116 2.0.7 (1 year, 23 days ago) Dec 11 2020 text-parsing sumy 3,289 0.11.0 (1 year, 1 month ago) Oct 20 2013 http-client requests 50,569 2.31.0 (6 months ago) Feb 14 2011 aiohttp 14,116 3.9.0 (11 days ago) Jul 26 2019 httpx 11,446 0.25.1 (26 days ago) Jul 26 2019 pycurl 1,019 7.45.2 (11 months ago) Feb 25 2003 treq 565 23.11.0 (25 days ago) Dec 28 2012 data-output xhtml2pdf 2,102 0.2.13 (20 days ago) May 19 2011 data jmespath 1,972 1.0.1 (1 year, 5 months ago) Feb 09 2022 jsonpath-ng 477 1.6.0 (2 months ago) Feb 09 2022 nested-lookup 207 0.2.25 (1 year, 4 months ago) Feb 09 2022 xhtml2pdf 2,102 0.2.13 (20 days ago) May 19 2011 framework scrapy 49,364 2.11.0 (2 months ago) Jul 26 2019 scrapyd 2,734 1.4.3 (2 months ago) Sep 04 2013 autoscraper 5,654 1.1.14 (1 year, 4 months ago) Jul 26 2019 ruia 1,701 0.8.5 (1 year, 2 months ago) Oct 17 2018 scrapydweb 2,864 1.4.1 (2 months ago) Sep 30 2018 gerapy 3,118 0.9.13 (4 months ago) Jul 04 2017 photon 10,120 1.1.9 (5 years ago) Aug 24 2018 gracy 224 1.27.1 (3 months ago) Feb 05 2023 dude 386 0.1.3 (3 months ago) Feb 20 2022 educational youtube-dl 124,737 2021.12.17 (1 year, 11 months ago) Feb 22 2012 cloudscraper 3,596 1.2.71 (7 months ago) Dec 28 2012 you-get 48,498 0.4.1650 (11 months ago) Sep 01 2012 cli-tool youtube-dl 124,737 2021.12.17 (1 year, 11 months ago) Feb 22 2012 you-get 48,498 0.4.1650 (11 months ago) Sep 01 2012 scraper youtube-dl 124,737 2021.12.17 (1 year, 11 months ago) Feb 22 2012 trafilatura 2,210 1.6.2 (2 months ago) Jul 17 2019 you-get 48,498 0.4.1650 (11 months ago) Sep 01 2012 blocking ua-parser 518 0.18.0 (4 months ago) Dec 29 2012 cloudscraper 3,596 1.2.71 (7 months ago) Dec 28 2012 gui scrapydweb 2,864 1.4.1 (2 months ago) Sep 30 2018 gerapy 3,118 0.9.13 (4 months ago) Jul 04 2017 scrapy scrapy 49,364 2.11.0 (2 months ago) Jul 26 2019 scrapyd 2,734 1.4.3 (2 months ago) Sep 04 2013 scrapydweb 2,864 1.4.1 (2 months ago) Sep 30 2018 splash 3,944 3.5 (3 years ago) Apr 25 2014 gerapy 3,118 0.9.13 (4 months ago) Jul 04 2017 seo advertools 948 0.13.5 (3 months ago) Dec 28 2012 http-utils httptools 1,142 0.6.1 (a month ago) Dec 28 2012 requests-cache 1,183 1.1.1 (10 days ago) Feb 14 2011 css cssutils 63 2.9.0 (a month ago) Aug 04 2007 microformats extruct 783 0.16.0 (4 months ago) Oct 27 2015 html-utils bleach 2,563 6.1.0 (a month ago) Feb 21 2010 browser-automation selenium 28,273 4.15.2 (25 days ago) Apr 25 2008 playwright 9,689 1.40.0 (7 days ago) Feb 24 2021 requestium 1,755 0.2.1 (6 months ago) Dec 28 2012 splash 3,944 3.5 (3 years ago) Apr 25 2014 jsonpath jsonpath-ng 477 1.6.0 (2 months ago) Feb 09 2022 json-parser jmespath 1,972 1.0.1 (1 year, 5 months ago) Feb 09 2022 jsonpath-ng 477 1.6.0 (2 months ago) Feb 09 2022 nested-lookup 207 0.2.25 (1 year, 4 months ago) Feb 09 2022 selenium selenium 28,273 4.15.2 (25 days ago) Apr 25 2008 requestium 1,755 0.2.1 (6 months ago) Dec 28 2012 http ua-parser 518 0.18.0 (4 months ago) Dec 29 2012 headers ua-parser 518 0.18.0 (4 months ago) Dec 29 2012 jmespath jmespath 1,972 1.0.1 (1 year, 5 months ago) Feb 09 2022 scrapers crosslinked 1,029 0.3.0 (5 months ago) Feb 14 2011 playwright playwright 9,689 1.40.0 (7 days ago) Feb 24 2021 Was this page helpful? Thanks for your feedback! Thanks for your feedback! Feel free to email suggestions to hello@webscraping.fyi