Skip to content

hrequestsvshttp-2

MIT 28 1 713
15.1 thousand (month) Feb 23 2022 0.9.2(9 days ago)
899 6 - MIT
Sep 25 2013 139.8 thousand (month) 1.0.2(a month ago)

hrequests is a feature rich modern replacement for a famous requests library for Python. It provides a feature rich HTTP client capable of resisting popular scraper identification techniques: - Seamless transition between headless browser and http client based requests - Integrated HTML parser - Mimicking of real browser TLS fingerprints - Javascript rendering - HTTP2 support - Realistic browser headers

Pure Ruby, framework and transport agnostic, implementation of HTTP/2 protocol and HPACK header compression with support for:

  • Binary framing parsing and encoding
  • Stream multiplexing and prioritization
  • Connection and stream flow control
  • Header compression and server push
  • Connection and stream management
  • And more... see API docs

Protocol specifications:

  • Hypertext Transfer Protocol Version 2 (RFC 7540)
  • HPACK: Header Compression for HTTP/2 (RFC 7541)

Highlights


bypasshttp2tls-fingerprinthttp-fingerprintsyncasync
http2

Example Use


hrequests has almost identical API and UX as requests and here's a quick overview:
import hrequests

# perform HTTP client requests
resp = hrequests.get('https://httpbin.org/html')
print(resp.status_code)
# 200

# use headless browsers and sessions:
session = hrequests.Session('chrome', version=122, os="mac")

# supports asyncio and easy concurrency
requests = [
    hrequests.async_get('https://www.google.com/', browser='firefox'),
    hrequests.async_get('https://www.duckduckgo.com/'),
    hrequests.async_get('https://www.yahoo.com/'),
    hrequests.async_get('https://www.httpbin.org/'),
]
responses = hrequests.map(requests, size=3)  # max 3 conccurency
require 'http/2'

# GET request
client = HTTP2::Client.new
response = client.get("https://httpbin.org/get")
puts response.body

# POST reuqest
data = { name: "value" }
response = client.post("https://www.example.com", data)

Alternatives / Similar


Was this page helpful?