superagentvshrequests
superagent is an HTTP client library for Node.js that provides a simple, flexible, and powerful API for making HTTP requests. It supports all major HTTP methods, and has a clean and easy-to-use interface for handling responses and errors.
what differentiates superagent from other http clients is its simple declarative API.
hrequests is a feature rich modern replacement for a famous requests library for Python. It provides a feature rich HTTP client capable of resisting popular scraper identification techniques: - Seamless transition between headless browser and http client based requests - Integrated HTML parser - Mimicking of real browser TLS fingerprints - Javascript rendering - HTTP2 support - Realistic browser headers
Highlights
declarativeproxypopular
bypasshttp2tls-fingerprinthttp-fingerprintsyncasync
Example Use
const superagent = require('superagent');
// superagent supports both Promises and async/await
superagent.get('https://httpbin.org/get')
.then(res => console.log(res.text))
.catch(err => console.error(err));
const response = superagent.get('https://httpbin.org/get')
// post requests:
superagent.post('https://httpbin.org/post').send({ name: 'John Doe' })
// setting proxy
superagent.get('https://httpbin.org/ip').proxy('http://proxy.example.com:8080')
// settings headers and proxies
superagent.get('https://httpbin.org/headers').set('Cookie', 'myCookie=123').set('X-My-Header', 'myValue')
hrequests has almost identical API and UX as requests and here's a quick overview:
import hrequests
# perform HTTP client requests
resp = hrequests.get('https://httpbin.org/html')
print(resp.status_code)
# 200
# use headless browsers and sessions:
session = hrequests.Session('chrome', version=122, os="mac")
# supports asyncio and easy concurrency
requests = [
hrequests.async_get('https://www.google.com/', browser='firefox'),
hrequests.async_get('https://www.duckduckgo.com/'),
hrequests.async_get('https://www.yahoo.com/'),
hrequests.async_get('https://www.httpbin.org/'),
]
responses = hrequests.map(requests, size=3) # max 3 conccurency