Framework Libraries Used in Web Scraping

go

ferret
5,716 v0.18.0 (1 year, 8 months ago) Aug 06 2019
gocrawl
2,037 (3 years ago) Nov 20 2016
dataflowkit
663 2024-12-03 (6 days ago) Feb 09 2017
geziyor
2,641 2024-12-04 (6 days ago) Jun 06 2019
pholcus
7,566 v1.3.4 (4 years ago) Feb 15 2020
colly
23,410 v2.1.0 (4 years ago) May 14 2018

javascript

ayakashi
212 1.0.0-beta8.4 (1 year, 5 months ago) Apr 18 2019
node-crawler
6,713 2.0.2 (4 months ago) Sep 10 2012

php

crwlr-crawler
342 v2.1.3 (a month ago) Apr 18 2022
php-spider
1,335 v0.7.2 (a year ago) Mar 16 2013
phpscraper
544 3.0.0 (8 months ago) May 04 2020
roach
1,353 v3.2.0 (8 months ago) Dec 27 2021
panther
2,951 v2.1.1 (1 year, 7 days ago) Jul 17 2018

python

dude
426 0.1.3 (1 year, 4 months ago) Feb 20 2022
photon
11,112 1.1.9 (6 years ago) Aug 24 2018
gracy
246 1.33.0 (9 months ago) Feb 05 2023
ruia
1,753 0.8.5 (2 years ago) Oct 17 2018
scrapydweb
3,176 1.5.1 (2 months ago) Sep 30 2018
gerapy
3,362 0.9.13 (1 year, 4 months ago) Jul 04 2017
autoscraper
6,193 1.1.14 (2 years ago) Jul 26 2019
scrapyd
2,980 1.5.0 (2 months ago) Sep 04 2013
scrapy
52,353 2.11.2 (6 months ago) Jul 26 2019

r

ralger
156 2.2.4 (3 years ago) Dec 22 2019
rvest
1,494 1.0.4 (2 years ago) Nov 22 2014

ruby

wombat
1,315 3.0.0 (2 years ago) Dec 27 2011
spidr
809 0.7.1 (10 months ago) Jul 25 2009
Was this page helpful?