Python Libraries Used in Web Scraping seo advertools 1,148 0.16.1 (2 months ago) Dec 28 2012 http-client requests 52,128 2.32.3 (4 months ago) Feb 14 2011 aiohttp 15,096 3.10.10 (16 days ago) Jul 26 2019 httpx 13,195 0.27.2 (2 months ago) Jul 26 2019 curl-impersonate 3,751 v0.6.1 (7 months ago) Feb 23 2022 pycurl 1,079 7.45.3 (8 months ago) Feb 25 2003 curl-cffi 1,751 0.7.1 (3 months ago) Feb 23 2022 treq 587 23.11.0 (11 months ago) Dec 28 2012 hrequests 672 0.8.2 (6 months ago) Feb 23 2022 framework scrapy 52,353 2.11.2 (5 months ago) Jul 26 2019 scrapyd 2,955 1.5.0 (25 days ago) Sep 04 2013 autoscraper 6,193 1.1.14 (2 years ago) Jul 26 2019 gerapy 3,344 0.9.13 (1 year, 3 months ago) Jul 04 2017 ruia 1,749 0.8.5 (2 years ago) Oct 17 2018 scrapydweb 3,144 1.5.1 (21 days ago) Sep 30 2018 gracy 246 1.33.0 (8 months ago) Feb 05 2023 photon 11,012 1.1.9 (6 years ago) Aug 24 2018 dude 424 0.1.3 (1 year, 2 months ago) Feb 20 2022 html-parser beautifulsoup - 4.12.3 (9 months ago) Jul 26 2019 lxml 2,691 5.3.0 (2 months ago) Dec 13 2022 xmltodict 5,491 0.14.2 (11 days ago) Jul 30 2007 html5lib 1,118 1.1 (4 years ago) Jul 30 2007 cssselect 290 1.2.0 (2 years ago) Apr 14 2012 feedparser 1,966 6.0.11 (10 months ago) Jun 15 2007 pyquery 2,298 2.0.1 (a month ago) Dec 05 2008 parsel 1,143 1.9.1 (6 months ago) Jul 26 2019 selectolax 1,118 0.3.25 (2 days ago) Mar 01 2018 requests-html 13,726 0.10.0 (5 years ago) Feb 25 2018 untangle 612 1.2.1 (2 years ago) Jun 09 2011 chompjs 194 1.3.0 (2 months ago) Jul 30 2007 html5-parser 681 0.4.12 (11 months ago) Jun 03 2007 gazpacho 740 1.1 (4 years ago) Dec 28 2012 chopper 22 0.6.0 (1 year, 6 months ago) Jul 24 2014 html-utils bleach 2,650 6.1.0 (1 year, 21 days ago) Feb 21 2010 captcha captcha_solver 243 0.1.5 (5 years ago) Feb 17 2015 unicaps 220 1.3.0 (7 months ago) Jul 27 2020 educational cloudscraper 4,388 1.2.71 (1 year, 6 months ago) Dec 28 2012 youtube-dl 131,571 2021.12.17 (2 years ago) Feb 22 2012 you-get 50,259 0.4.1730 (2 months ago) Sep 01 2012 blocking ua-parser 556 0.18.0 (1 year, 3 months ago) Dec 29 2012 cloudscraper 4,388 1.2.71 (1 year, 6 months ago) Dec 28 2012 scrapers crosslinked 1,270 0.3.0 (1 year, 4 months ago) Feb 14 2011 css cssutils 78 2.11.1 (4 months ago) Aug 04 2007 stealth curl-impersonate 3,751 v0.6.1 (7 months ago) Feb 23 2022 undetected-chromedriver 9,858 3.5.5 (8 months ago) Sep 04 2020 curl-cffi 1,751 0.7.1 (3 months ago) Feb 23 2022 hrequests 672 0.8.2 (6 months ago) Feb 23 2022 selenium-driverless 568 1.9.4 (5 days ago) Jul 22 2022 html-extractor html2text 1,828 2024.2.26 (7 months ago) Dec 14 2008 trafilatura 3,580 1.12.2 (a month ago) Jul 17 2019 newspaper 14,130 0.2.8 (6 years ago) Dec 28 2012 extruct 847 0.17.0 (4 months ago) Oct 27 2015 sumy 3,516 0.11.0 (2 years ago) Oct 20 2013 readability 2,657 0.8.1 (4 years ago) Jun 30 2011 extractnet 229 2.0.7 (1 year, 11 months ago) Dec 11 2020 photon 11,012 1.1.9 (6 years ago) Aug 24 2018 microformats extruct 847 0.17.0 (4 months ago) Oct 27 2015 gui gerapy 3,344 0.9.13 (1 year, 3 months ago) Jul 04 2017 scrapydweb 3,144 1.5.1 (21 days ago) Sep 30 2018 scrapy scrapy 52,353 2.11.2 (5 months ago) Jul 26 2019 scrapyd 2,955 1.5.0 (25 days ago) Sep 04 2013 gerapy 3,344 0.9.13 (1 year, 3 months ago) Jul 04 2017 scrapydweb 3,144 1.5.1 (21 days ago) Sep 30 2018 splash 4,092 3.5 (4 years ago) Apr 25 2014 http-utils httptools 1,203 0.6.4 (10 days ago) Dec 28 2012 requests-cache 1,321 1.2.1 (4 months ago) Feb 14 2011 json-parser jmespath 2,179 1.0.1 (2 years ago) Feb 09 2022 jsonpath-ng 600 1.7.0 (16 days ago) Feb 09 2022 nested-lookup 209 0.2.25 (2 years ago) Feb 09 2022 jmespath jmespath 2,179 1.0.1 (2 years ago) Feb 09 2022 data jmespath 2,179 1.0.1 (2 years ago) Feb 09 2022 jsonpath-ng 600 1.7.0 (16 days ago) Feb 09 2022 xhtml2pdf 2,251 0.2.16 (4 months ago) May 19 2011 nested-lookup 209 0.2.25 (2 years ago) Feb 09 2022 jsonpath jsonpath-ng 600 1.7.0 (16 days ago) Feb 09 2022 playwright playwright 11,763 1.48.0 (6 days ago) Feb 24 2021 browser-automation selenium 30,594 4.25.0 (a month ago) Apr 25 2008 playwright 11,763 1.48.0 (6 days ago) Feb 24 2021 undetected-chromedriver 9,858 3.5.5 (8 months ago) Sep 04 2020 requestium 1,834 0.4.0 (8 months ago) Dec 28 2012 selenium-driverless 568 1.9.4 (5 days ago) Jul 22 2022 splash 4,092 3.5 (4 years ago) Apr 25 2014 selenium selenium 30,594 4.25.0 (a month ago) Apr 25 2008 undetected-chromedriver 9,858 3.5.5 (8 months ago) Sep 04 2020 requestium 1,834 0.4.0 (8 months ago) Dec 28 2012 selenium-driverless 568 1.9.4 (5 days ago) Jul 22 2022 text-parsing sumy 3,516 0.11.0 (2 years ago) Oct 20 2013 scraper trafilatura 3,580 1.12.2 (a month ago) Jul 17 2019 youtube-dl 131,571 2021.12.17 (2 years ago) Feb 22 2012 you-get 50,259 0.4.1730 (2 months ago) Sep 01 2012 http ua-parser 556 0.18.0 (1 year, 3 months ago) Dec 29 2012 headers ua-parser 556 0.18.0 (1 year, 3 months ago) Dec 29 2012 data-output xhtml2pdf 2,251 0.2.16 (4 months ago) May 19 2011 cli-tool youtube-dl 131,571 2021.12.17 (2 years ago) Feb 22 2012 you-get 50,259 0.4.1730 (2 months ago) Sep 01 2012 Was this page helpful? Thanks for your feedback! Thanks for your feedback! Feel free to email suggestions to hello@webscraping.fyi