requestsvshttpclient
The requests package is a popular library for making HTTP requests in Python.
It provides a simple, easy-to-use API for sending HTTP/1.1 requests, and it abstracts away many of the low-level details of working with HTTP.
One of the key features of requests is its simple API. You can send a GET request with a single line of code:
import requests
response = requests.get('https://webscraping.fyi/lib/requests/')
pip install requests
HTTPClient is a Ruby gem that provides a simple and flexible interface for making HTTP requests. It's a full-featured HTTP client library with support for cookies, redirects, proxy, and more. It's built on top of the libwww-perl library, which is a widely-used, robust and well-documented library.
Features:
- methods like GET/HEAD/POST/* via HTTP/1.1.
- HTTPS(SSL), Cookies, proxy, authentication(Digest, NTLM, Basic), etc.
- asynchronous HTTP request, streaming HTTP request.
- debug mode CLI.
- by contrast with net/http in standard distribution;
- Cookies support
- MT-safe
- streaming POST (POST with File/IO)
- Digest auth
- Negotiate/NTLM auth for WWW-Authenticate (requires net/ntlm module; rubyntlm gem)
- NTLM auth for Proxy-Authenticate (requires 'win32/sspi' module; rubysspi gem)
- extensible with filter interface
- you don't have to care HTTP/1.1 persistent connection (httpclient cares instead of you)
Highlights
syncease-of-useno-http2no-asyncpopular
Example Use
import requests
# get request:
response = requests.get("http://webscraping.fyi/")
response.status_code
200
response.text
"text"
response.content
b"bytes"
# requests can automatically convert json responses to Python dictionaries:
response = requests.get("http://httpbin.org/json")
print(response.json())
{'slideshow': {'author': 'Yours Truly', 'date': 'date of publication', 'slides': [{'title': 'Wake up to WonderWidgets!', 'type': 'all'}, {'items': ['Why <em>WonderWidgets</em> are great', 'Who <em>buys</em> WonderWidgets'], 'title': 'Overview', 'type': 'all'}], 'title': 'Sample Slide Show'}}
# for POST request it can ingest Python's dictionaries as JSON:
response = requests.post("http://httpbin.org/post", json={"query": "hello world"})
# or form data:
response = requests.post("http://httpbin.org/post", data={"query": "hello world"})
# Session object can be used to automatically keep track of cookies and set defaults:
from requests import Session
s = Session()
s.headers = {"User-Agent": "webscraping.fyi"}
s.get('http://httpbin.org/cookies/set/foo/bar')
print(s.cookies['foo'])
'bar'
print(s.get('http://httpbin.org/cookies').json())
{'cookies': {'foo': 'bar'}}
require 'httpclient'
client = HTTPClient.new
# GET requests
response = client.get("http://httpbin.org/get")
puts response.content
# POST requests
data = { name: "value" }
response = client.post("http://httpbin.org/post", data)
puts response.content