Skip to content

curl-cffivscamoufox

MIT 34 2 1,751
594.9 thousand (month) Feb 23 2022 0.7.1(2024-07-13 09:07:25 ago)
7,089 3 244 MPL-2.0
Mar 01 2024 1.0 million (month) 0.4.11(2025-01-29 09:33:18 ago)

Curl-cffi is a Python library for implementing curl-impersonate which is a HTTP client that appears as one of popular web browsers like: - Google Chrome - Microsoft Edge - Safari - Firefox Unlike requests and httpx which are native Python libraries, curl-cffi uses cURL and inherits it's powerful features like extensive HTTP protocol support and detection patches for TLS and HTTP fingerprinting.

Using curl-cffi web scrapers can bypass TLS and HTTP fingerprinting.

Camoufox is an anti-detect browser built on Firefox, designed specifically for web scraping and browser automation while evading bot detection systems. Unlike Chrome-based anti-detect solutions, Camoufox uses a patched version of Firefox with over 100 modifications to prevent fingerprinting and detection.

Key features include:

  • Firefox-based stealth Uses Firefox instead of Chrome, which has different fingerprinting characteristics and is less commonly targeted by anti-bot systems. The browser is patched at the C++ level to prevent JavaScript-based fingerprint leaks.
  • Realistic fingerprints Generates and applies realistic browser fingerprints including screen resolution, WebGL renderer, fonts, and other browser properties that match real user configurations.
  • Playwright integration Works seamlessly with Playwright for Python, providing the familiar Playwright API while running through the stealth-patched Firefox instance.
  • Proxy support Built-in proxy support with authentication, including residential proxy rotation.
  • Geolocation spoofing Can spoof browser geolocation to match proxy IP location for consistent fingerprints.
  • Human-like behavior Includes utilities for adding human-like mouse movements and typing patterns.

Camoufox is particularly effective against sophisticated anti-bot systems that detect Chrome-based automation tools. By using Firefox as the base, it avoids many Chrome-specific detection vectors while the deep browser patches prevent general fingerprinting techniques.

Highlights


bypasshttp2tls-fingerprinthttp-fingerprintsyncasync
anti-detectstealth

Example Use


curl-cffi can be accessed as low-level curl client as well as an easy high-level HTTP client: ```python from curl_cffi import requests response = requests.get('https://httpbin.org/json') print(response.json()) # or using sessions session = requests.Session() response = session.get('https://httpbin.org/json') # also supports async requests using asyncio import asyncio from curl_cffi.requests import AsyncSession urls = [ "http://httpbin.org/html", "http://httpbin.org/html", "http://httpbin.org/html", ] async with AsyncSession() as s: tasks = [] for url in urls: task = s.get(url) tasks.append(task) # scrape concurrently: responses = await asyncio.gather(*tasks) # also supports websocket connections from curl_cffi.requests import Session, WebSocket def on_message(ws: WebSocket, message): print(message) with Session() as s: ws = s.ws_connect( "wss://api.gemini.com/v1/marketdata/BTCUSD", on_message=on_message, ) ws.run_forever() ```
```python from camoufox.sync_api import Camoufox # Launch stealth Firefox browser with Camoufox(headless=True) as browser: page = browser.new_page() # Navigate - the browser appears as a real Firefox user page.goto("https://example.com") # Use standard Playwright API for interaction page.fill("input[name='search']", "web scraping") page.click("button[type='submit']") # Extract data results = page.query_selector_all(".result") for result in results: title = result.query_selector("h3").text_content() print(title) # With proxy and custom fingerprint from camoufox.sync_api import Camoufox with Camoufox( headless=True, proxy={"server": "http://proxy.example.com:8080"}, geoip=True, # auto-match geolocation to proxy IP ) as browser: page = browser.new_page() page.goto("https://example.com") print(page.content()) ```

Alternatives / Similar


Was this page helpful?