pydollvsrod

None - - -

Jun 01 2024 0.0.0(2025-02-01 00:00:00 ago)

6,853 3 202 MIT

Sep 23 2022 v0.116.2(2024-07-12 11:52:28 ago)

Pydoll is a Python library for browser automation that uses the Chrome DevTools Protocol (CDP) directly, designed to be undetectable by anti-bot systems. Unlike Selenium-based tools, Pydoll does not use WebDriver and avoids the common detection vectors that anti-bot systems look for.

Key features include:

Native CDP communication Connects directly to Chrome/Chromium via CDP websocket without intermediary drivers, avoiding the automation flags and fingerprints that WebDriver-based tools leave behind.
Event-driven architecture Built around an async event system that can listen for and react to browser events like network requests, console messages, and DOM changes.
Network interception Can intercept, modify, and mock network requests and responses, useful for blocking unnecessary resources or modifying API responses during scraping.
Async-first design Fully asynchronous API built on Python's asyncio for efficient concurrent automation.
Clean API Provides a high-level, Pythonic API for common browser automation tasks while still allowing direct CDP command execution for advanced use cases.
Multi-browser support Can manage multiple browser instances and pages concurrently.

Pydoll fills a similar niche to nodriver and camoufox — browser automation with a focus on avoiding detection — but takes a different approach by providing more granular control over CDP communication and network interception.

Rod is a high-level Go library for browser automation built on the Chrome DevTools Protocol (CDP). It provides a simpler and more intuitive API compared to chromedp, making it easier to write browser automation and web scraping scripts in Go.

Key features include:

Simple API Rod's API is designed to be intuitive and requires less boilerplate than chromedp. Common operations like clicking, typing, and waiting are straightforward single-line calls.
Auto-wait Automatically waits for elements to be ready before interacting with them, reducing the need for explicit wait statements and making scripts more reliable.
Page pool Built-in page pool for managing multiple browser pages efficiently, useful for concurrent scraping tasks.
Stealth mode Includes a stealth plugin (rod/lib/launcher/flags) that can disable common automation detection vectors.
Element screenshots Can take screenshots of specific elements, not just full pages.
Network interception Supports hijacking network requests and responses for modification or monitoring.
Input emulation Realistic mouse and keyboard input emulation for interacting with complex web applications.

Rod is the recommended choice for new Go browser automation projects due to its simpler API and active maintenance. It is comparable to Playwright in terms of developer experience but native to the Go ecosystem.

Highlights

anti-detectcdpasync

cdpfast

Example Use

```python import asyncio from pydoll.browser import Chrome from pydoll.constants import By async def main(): async with Chrome() as browser: # Open a new page page = await browser.new_page() await page.go_to("https://example.com") # Find and interact with elements search_input = await page.find_element(By.CSS, "input[name='q']") await search_input.type_text("web scraping") submit_btn = await page.find_element(By.CSS, "button[type='submit']") await submit_btn.click() # Wait for results and extract content await page.wait_element(By.CSS, ".results") results = await page.find_elements(By.CSS, ".result-item") for result in results: title = await result.get_text() print(title) # Network interception example await page.enable_network_interception() # intercept and analyze API calls made by the page asyncio.run(main()) ```

```go package main import ( "fmt" "github.com/go-rod/rod" "github.com/go-rod/rod/lib/launcher" ) func main() { // Launch browser url := launcher.New().Headless(true).MustLaunch() browser := rod.New().ControlURL(url).MustConnect() defer browser.MustClose() // Navigate and auto-wait for the page to load page := browser.MustPage("https://example.com") page.MustWaitStable() // Find elements and extract text - auto-waits for element title := page.MustElement("h1").MustText() fmt.Println("Title:", title) // Fill in a form page.MustElement("input[name='search']").MustInput("web scraping") page.MustElement("button[type='submit']").MustClick() // Wait for results and extract page.MustWaitStable() results := page.MustElements(".result-item") for _, el := range results { text := el.MustText() href := el.MustElement("a").MustProperty("href").String() fmt.Printf("Result: %s (%s)\n", text, href) } // Take screenshot of specific element page.MustElement(".results").MustScreenshot("results.png") } ```