Skip to content

photon

10,534 3 52 MIT
1.1.9 (21 Oct 2018) Aug 24 2018 322 (month)

Photon is a Python library for web scraping. It is designed to be lightweight and fast, and can be used to extract data from websites and web pages. Photon can extract the following data while crawling:

  • URLs (in-scope & out-of-scope)
  • URLs with parameters (example.com/gallery.php?id=2)
  • Intel (emails, social media accounts, amazon buckets etc.)
  • Files (pdf, png, xml etc.)
  • Secret keys (auth/API keys & hashes)
  • JavaScript files & Endpoints present in them
  • Strings matching custom regex pattern
  • Subdomains & DNS related data

The extracted information is saved in an organized manner or can be exported as json.

Example Use


from photon import Photon

#Create a new Photon instance
ph = Photon()

#Extract data from a specific element of the website
url = "https://www.example.com"
selector = "div.main"
data = ph.get_data(url, selector)

#Print the extracted data
print(data)


#Extract data from multiple websites asynchronously
urls = ["https://www.example1.com", "https://www.example2.com"]
data = ph.get_data_async(urls)

Alternatives / Similar


1,595 2024.2.26 (2 months ago) Dec 14 2008 compare
51,043 2.11.1 (2 months ago) Jul 26 2019 compare
13,679 0.2.8 (5 years ago) Dec 28 2012 compare
2,911 1.9.0 (9 days ago) Jul 17 2019 compare
2,572 0.8.1 (3 years ago) Jun 30 2011 compare
3,429 0.11.0 (1 year, 6 months ago) Oct 20 2013 compare
823 0.16.0 (10 months ago) Oct 27 2015 compare
2,857 1.4.3 (7 months ago) Sep 04 2013 compare
5,964 1.1.14 (1 year, 9 months ago) Jul 26 2019 compare
3,012 1.5.0 (2 months ago) Sep 30 2018 compare
3,219 0.9.13 (9 months ago) Jul 04 2017 compare
235 1.33.0 (2 months ago) Feb 05 2023 compare
1,733 0.8.5 (1 year, 8 months ago) Oct 17 2018 compare
172 2.0.7 (1 year, 6 months ago) Dec 11 2020 compare
412 0.1.3 (9 months ago) Feb 20 2022 compare

Other Languages

22,276 v2.1.0 (3 years ago) May 14 2018 compare
7,533 v1.3.4 (4 years ago) Feb 15 2020 compare
2,488 2024-05-09 (3 days ago) Jun 06 2019 compare
645 2024-05-11 (16 hours ago) Feb 09 2017 compare
1,476 1.0.4 (1 year, 8 months ago) Nov 22 2014 compare
5,630 v0.18.0 (1 year, 1 month ago) Aug 06 2019 compare
2,026 (2 years ago) Nov 20 2016 compare
2,400 v1.3.0 (2 months ago) Apr 20 2016 compare
6,623 1.5.0 (23 hours ago) Sep 10 2012 compare
2,891 v2.1.1 (5 months ago) Jul 17 2018 compare
1,304 3.0.0 (1 year, 8 months ago) Dec 27 2011 compare
792 0.7.1 (3 months ago) Jul 25 2009 compare
152 2.2.4 (3 years ago) Dec 22 2019 compare
1,321 v3.2.0 (a month ago) Dec 27 2021 compare
493 3.0.0 (a month ago) May 04 2020 compare
197 1.0.0-beta8.4 (10 months ago) Apr 18 2019 compare
1,324 v0.7.2 (5 months ago) Mar 16 2013 compare
298 v1.7.2 (a month ago) Apr 18 2022 compare