buzzvscurl-cffi
Buzz is a lightweight PHP library for issuing HTTP requests. It is built on top of the PHP curl extension and is designed to be easy to use and flexible.
While buzz isn't as feature rich as other clients it still supports http2 and is relatively easy to use.
Curl-cffi is a Python library for implementing curl-impersonate which is a
HTTP client that appears as one of popular web browsers like:
- Google Chrome
- Microsoft Edge
- Safari
- Firefox
Unlike requests and httpx which are native Python libraries, curl-cffi uses cURL and inherits it's powerful features
like extensive HTTP protocol support and detection patches for TLS and HTTP fingerprinting.
Using curl-cffi web scrapers can bypass TLS and HTTP fingerprinting.
Highlights
http2uses-curl
bypasshttp2tls-fingerprinthttp-fingerprintsyncasync
Example Use
use Buzz\Client\Curl;
use Buzz\Message\Request;
use Buzz\Message\Response;
$client = new Curl();
// GET request
$request = new Request('GET', 'http://httpbin.org/get');
$response = new Response();
$client->send($request, $response);
echo $response->getContent();
//POST request
$request = new Request('POST', '/api/resource', 'http://example.com');
$request->setContent(json_encode(['name' => 'John Doe']));
// we can also add headers or cookies:
$request->addHeader('Content-Type: application/json');
$request->addHeader('Cookie: name=foobar');
$response = new Response();
$client->send($request, $response);
echo $response->getContent();
// Buzz also supports http2 (see the 2.0 parameter)
$response = $client->sendRequest(
new Request('GET', 'https://http2.golang.org/serverpush', [], null, '2.0')
);
curl-cffi can be accessed as low-level curl client as well as an easy high-level HTTP client:
from curl_cffi import requests
response = requests.get('https://httpbin.org/json')
print(response.json())
# or using sessions
session = requests.Session()
response = session.get('https://httpbin.org/json')
# also supports async requests using asyncio
import asyncio
from curl_cffi.requests import AsyncSession
urls = [
"http://httpbin.org/html",
"http://httpbin.org/html",
"http://httpbin.org/html",
]
async with AsyncSession() as s:
tasks = []
for url in urls:
task = s.get(url)
tasks.append(task)
# scrape concurrently:
responses = await asyncio.gather(*tasks)
# also supports websocket connections
from curl_cffi.requests import Session, WebSocket
def on_message(ws: WebSocket, message):
print(message)
with Session() as s:
ws = s.ws_connect(
"wss://api.gemini.com/v1/marketdata/BTCUSD",
on_message=on_message,
)
ws.run_forever()