Skip to content

playwrightvssplash

Apache-2.0 38 5 10,799
3.1 million (month) Feb 24 2021 1.43.0(a month ago)
4,008 15 402 BSD
3.5(3 years ago) Apr 25 2014 581 (month)

playwright is a Python package that allows developers to automate web browsers for end-to-end testing, web scraping, and web performance analysis. It is built on top of WebKit, Mozilla's Gecko, and Microsoft's EdgeHTML, and it is designed to be fast, reliable, and easy to use.

playwright is similar to Selenium, but it provides a more modern and powerful API, with features such as automatic waiting for elements, automatic retries, and built-in support for browser contexts, which allow you to open multiple pages in a single browser instance.

Playwright also provides an asynchronous client which makes scaling playwright-powered web scrapers easier than alternatives (like Selenium)

Splash is a javascript rendering service with an HTTP API. It's a lightweight browser with an HTTP API, implemented in Python 3 using Twisted and QT5.

It is built on top of the QtWebkit library and allows developers to interact with web pages in a headless mode, which means that the web pages are rendered in the background, without displaying them on the screen.

splash is particularly useful for web scraping and web testing tasks, as it allows developers to interact with web pages in a way that is very similar to how a human user would interact with the browser.

It also allows you to execute javascript and interact with web pages even if they use heavy javascript.

Unlike Selenium or Playwright, splash is powered by webkit embedded browser instead of a real browser like Chrome or Firefox. As a down-side splash requests are easy to detect and block when scraping websites with anti-scraping features.

One benefit of splash is that it seemlesly integrates with Scrapy.

Example Use


from playwright import sync_playwright

# Start Playwright
with sync_playwright() as playwright:
    # Launch a browser instance
    browser = playwright.chromium.launch()
    # Open a new context (tab)
    context = browser.new_context()
    # Create a new page in the context
    page = context.new_page()

    # Navigate to a website
    page.goto("https://www.example.com")

    # Find an element by its id
    element = page.get_by_id("example-id")

    # Interact with the element
    element.click()

    # Fill an input form
    page.get_by_name("example-name").fill("example text")

    # Find and click a button
    page.get_by_xpath("//button[text()='Search']").click()

    # Wait for the page to load
    page.wait_for_selector("#results")

    # Get the page title
    print(page.title)

    # Close the browser
    browser.close()
# once splash server is started it can be requested to render pages through 
# HTTP requests:
import requests

url = "http://localhost:8050/render.html"
payload = {
    'url': 'https://www.example.com',
    'timeout': 30,
    'wait': 2
}

response = requests.get(url, params=payload)

# Get the page HTML
print(response.text)

Alternatives / Similar