Framework Libraries Used in Web Scraping

php

crwlr-crawler
277 v1.3.0 (a month ago) Apr 18 2022
php-spider
1,312 v0.7.1 (3 months ago) Mar 16 2013
phpscraper
443 2.0.0 (5 months ago) May 04 2020
roach
1,271 2.0.1 (9 months ago) Dec 27 2021
panther
2,829 v2.1.0 (5 months ago) Jul 17 2018

go

ferret
5,513 v0.18.0 (8 months ago) Aug 06 2019
gocrawl
2,021 (2 years ago) Nov 20 2016
dataflowkit
622 2023-11-24 (4 days ago) Feb 09 2017
geziyor
2,296 2023-11-24 (4 days ago) Jun 06 2019
pholcus
7,476 v1.3.4 (3 years ago) Feb 15 2020
colly
21,215 v2.1.0 (3 years ago) May 14 2018

r

ralger
150 2.2.4 (2 years ago) Dec 22 2019
rvest
1,428 1.0.3 (2 years ago) Nov 22 2014

javascript

ayakashi
186 1.0.0-beta8.4 (5 months ago) Apr 18 2019
node-crawler
6,541 1.4.0 (10 months ago) Sep 10 2012

ruby

spidr
769 0.7.0 (10 months ago) Jul 25 2009
wombat
1,298 3.0.0 (1 year, 3 months ago) Dec 27 2011

python

dude
386 0.1.3 (3 months ago) Feb 20 2022
gracy
224 1.27.1 (3 months ago) Feb 05 2023
photon
10,120 1.1.9 (5 years ago) Aug 24 2018
gerapy
3,118 0.9.13 (4 months ago) Jul 04 2017
scrapydweb
2,864 1.4.1 (2 months ago) Sep 30 2018
ruia
1,701 0.8.5 (1 year, 2 months ago) Oct 17 2018
autoscraper
5,654 1.1.14 (1 year, 4 months ago) Jul 26 2019
scrapyd
2,734 1.4.3 (2 months ago) Sep 04 2013
scrapy
49,364 2.11.0 (2 months ago) Jul 26 2019