puppeteer-stealthvscurl-impersonate

MIT 301 30 94,086

3.6 million (month) May 29 2018 2.11.2(2023-04-11 04:13:00 ago)

5,944 1 79 MIT

Feb 23 2022 v0.6.1(2024-03-02 18:08:29 ago)

Puppeteer Stealth is puppeteer plugin that fortifies headles browser for web scraping. This makes detection of puppeteer scrapers more difficult allowing to scrape targets which use headless browser detection techniques.

Puppeteer-stealth does this by applying various javascript patches to cover up traces of headless browser presence in the web scraping browser's environment.

Curl-impersonate is a special build of libcurl and cURL HTTP client that impersonates the four major browsers: - Google Chrome - Microsoft Edge - Safari - Firefox Curl-impersonate achieves this by patching TLS and HTTP fingerprints to be identical to that of one of these real browsers.

Unlike other HTTP clients curl-impersonate can bypass TSL and HTTP fingerprinting and detection techniques though it does not implement anything for Javascript fingerprint or bypass.

Highlights

bypasshttp2tls-fingerprinthttp-fingerprintlow-level

Example Use

```javascript const puppeteer = require('puppeteer-extra') // add stealth plugin and use defaults (all evasion techniques) const StealthPlugin = require('puppeteer-extra-plugin-stealth') puppeteer.use(StealthPlugin()) // puppeteer usage as normal puppeteer.launch({ headless: true }).then(async browser => { console.log('Running tests..') const page = await browser.newPage() await page.goto('https://bot.sannysoft.com') await page.waitForTimeout(5000) await page.screenshot({ path: 'result.png', fullPage: true }) await browser.close() console.log("success - check the result.png screenshot") }) ```

curl-impersonate installs itself under `curl_` terminal commands like `curl_chrome116`: ```shell $ curl_chrome116 https://www.wikipedia.org ``` To use it in HTTP client libraries that use `libcurl` replace curl path with one of these. To use it in python directly see curl-cffi Python package

Alternatives / Similar

puppeteer-extra new

7,293 compare

curl-impersonate

5,944 compare

puppeteer

94,086 compare

primp new

504 compare

undetected-chromedriver

12,519 compare

camoufox new

7,089 compare

curl-cffi

1,751 compare

nodriver new

4,003 compare

botasaurus new

4,321 compare

hrequests

1,001 compare

selenium-driverless

718 compare

ayakashi

217 compare

pydoll new

- compare

primp new

504 compare

curl-cffi

1,751 compare

hrequests

1,001 compare

requests

53,883 compare

node-fetch

8,860 compare

httpx

15,183 compare

aiohttp

16,395 compare

axios

108,987 compare

got

14,897 compare

needle

1,631 compare

superagent

16,610 compare

faraday

5,927 compare

pycurl

1,147 compare

puppeteer-stealth

94,086 compare

excon

1,172 compare

httpclient

707 compare

httparty

5,889 compare

puppeteer-extra new

7,293 compare

typhoeus

4,131 compare

httr

985 compare

undetected-chromedriver

12,519 compare

camoufox new

7,089 compare

guzzle

23,447 compare

rvest

1,517 compare

nodriver new

4,003 compare

wreck

378 compare

em-http-request

1,219 compare

symfony-http

2,033 compare

mechanize new

4,440 compare

treq

605 compare

http-2

908 compare

req

4,781 compare

resty

11,632 compare

nestful

505 compare

botasaurus new

4,321 compare

crul

107 compare

requests

3,577 compare

buzz

1,924 compare

selenium-driverless

718 compare

httpful

1,803 compare

ralger

165 compare

http.rb

3,104 compare

pydoll new

- compare