wreckvsem-http-request
Wreck is an HTTP client library for Node.js. It provides a simple, consistent API for making HTTP requests, including support for both the client and server side of an HTTP transaction.
Wreck is a very minimal but stable as it's part of Hapi web framework project. For web scraping, it doesn't offer required features like proxy configuration or http2 support so it's not recommended.
em-http-request is a Ruby gem for making asynchronous HTTP requests using EventMachine. It allows you to perform multiple requests simultaneously and handle the responses as they come in, rather than waiting for each request to complete before making the next one.
In short it supports: - Asynchronous HTTP API for single & parallel request execution - Keep-Alive and HTTP pipelining support - Auto-follow 3xx redirects with max depth - Automatic gzip & deflate decoding - Streaming response processing - Streaming file uploads - HTTP proxy and SOCKS5 support - Basic Auth & OAuth - Connection-level & global middleware support - HTTP parser via http_parser.rb - Works wherever EventMachine runs: Rubinius, JRuby, MRI
Example Use
const Wreck = require('wreck');
// get request
Wreck.get('http://example.com', (err, res, payload) => {
if (err) {
throw err;
}
console.log(payload.toString());
});
// post request
const options = {
headers: { 'content-type': 'application/json' },
payload: JSON.stringify({ name: 'John Doe' })
};
Wreck.post('http://example.com', options, (err, res, payload) => {
if (err) {
throw err;
}
console.log(payload.toString());
});
EventMachine.run {
http = EventMachine::HttpRequest.new('http://google.com/').get :query => {'keyname' => 'value'}
# add callback for errors:
http.errback { p 'Uh oh'; EM.stop }
# add callback for successful requests
http.callback {
p http.response_header.status
p http.response_header
p http.response
EventMachine.stop
}
}