selenium-driverlessvscurl-impersonate
Selenium Driverless is a Selenium inspired browser automation library with focus on web scraping detection bypass. It shares most of Selenium API and UX but implements several extensions that make the scraper more difficult to detect and extra usability features like: - Bypass Cloudflare - Multiple Tab scraping - Multiple context support - Proxy auth - Network interception
Curl-impersonate is a special build of libcurl and cURL HTTP client that impersonates the four major browsers:
- Google Chrome
- Microsoft Edge
- Safari
- Firefox
Curl-impersonate achieves this by patching TLS and HTTP fingerprints to be identical to that of one of these real browsers.
Unlike other HTTP clients curl-impersonate can bypass TSL and HTTP fingerprinting and detection techniques though it does not implement anything for Javascript fingerprint or bypass.
Highlights
Example Use
# It works the same as Selenium just with a different import.
import undetected_chromedriver as uc
driver = uc.Chrome(headless=True, use_subprocess=False)
driver.get('https://nowsecure.nl')
driver.save_screenshot('screenshot.png')
driver.close()
$ curl_chrome116 https://www.wikipedia.org