Framework Libraries Used in Web Scraping

ruby

---

wombat

1,316 3.0.0 (3 years ago) Dec 27 2011

spidr

813 0.7.2 (6 months ago) Jul 25 2009

javascript

---

ayakashi

213 1.0.0-beta8.4 (2 years ago) Apr 18 2019

node-crawler

6,733 2.0.2 (1 year, 1 month ago) Sep 10 2012

python

---

dude

428 0.1.3 (2 years ago) Feb 20 2022

photon

11,149 1.1.9 (6 years ago) Aug 24 2018

ruia

1,754 0.8.5 (2 years ago) Oct 17 2018

gerapy

3,365 0.9.13 (2 years ago) Jul 04 2017

scrapydweb

3,218 1.6.0 (6 months ago) Sep 30 2018

gracy

247 1.34.0 (8 months ago) Feb 05 2023

autoscraper

6,638 1.1.14 (3 years ago) Jul 26 2019

scrapyd

2,980 1.5.0 (10 months ago) Sep 04 2013

scrapy

54,211 2.12.0 (9 months ago) Jul 26 2019

php

---

crwlr-crawler

356 v3.2.3 (6 months ago) Apr 18 2022

php-spider

1,335 v0.7.2 (1 year, 8 months ago) Mar 16 2013

phpscraper

554 3.0.0 (1 year, 4 months ago) May 04 2020

roach

1,384 v3.2.0 (1 year, 4 months ago) Dec 27 2021

panther

2,977 v2.2.0 (6 months ago) Jul 17 2018

go

---

gocrawl

2,039 (4 years ago) Nov 20 2016

ferret

5,716 v0.18.0 (2 years ago) Aug 06 2019

dataflowkit

676 2025-02-16 (6 months ago) Feb 09 2017

geziyor

2,667 2025-02-18 (6 months ago) Jun 06 2019

pholcus

7,580 v1.3.4 (5 years ago) Feb 15 2020

colly

23,747 v2.1.0 (5 years ago) May 14 2018

r

---

ralger

156 2.2.4 (4 years ago) Dec 22 2019

rvest

1,498 1.0.4 (3 years ago) Nov 22 2014