Skip to content

gotvsexcon

MIT 108 10 13,904
88.1 million (month) Mar 27 2014 14.2.1(30 days ago)
1,152 19 24 MIT
0.110.0(24 days ago) Oct 31 2009 2.7 million (month)

Got is a lightweight and powerful HTTP client for Node.js. It is built on top of the http and https modules and provides a simple, consistent API for making HTTP requests.

Got is one of the most feature-rich http clients in NodeJS ecosystem offering http2, proxy and asynchronous support making it ideal for web scraping.

Got also supports many specific domain integrations like AWS, plugins for various public APIs like github.

Note that Got has some inconsistent behaviors when it comes to web scraping use.
For example, it normalizes http headers which is undesired functionality in scraping and should be disabled.

Excon is a Ruby library for making HTTP requests. It is designed to be fast and efficient, and is often used as a building block for other Ruby libraries and frameworks.

One of the main features of Excon is its support for persistent connections, which allows it to reuse the same connection for multiple requests, reducing the overhead of establishing a new connection for each request.

Excon also supports streaming requests and responses, which allows you to read or write data to the server incrementally, without having to load the entire response into memory at once.

Highlights


http2asyncpopularextendibletypescriptproxy

Example Use


const got = require('got');

// GET requests are default and can be made calling the module as is:
const response = await got('https://api.example.com');
console.log(response.body);

// POST requests can send 
const response = await got.post('https://api.example.com', {
    json: { name: 'John Doe' },
});
console.log(response.body);

// handling cookies
import {CookieJar} from 'tough-cookie';

const cookieJar = new CookieJar();

await cookieJar.setCookie('foo=bar', 'https://httpbin.org');
await got('https://httpbin.org/anything', {cookieJar});

// using proxy
import got from 'got';
import {HttpsProxyAgent} from 'hpagent';

await got('https://httpbin.org/ip', {
  agent: {
    https: new HttpsProxyAgent({
      keepAlive: true,
      keepAliveMsecs: 1000,
      maxSockets: 256,
      maxFreeSockets: 256,
      scheduling: 'lifo',
      proxy: 'https://localhost:8080'
    })
  }
});
require 'excon'

# GET requests
response = Excon.get('https://jsonplaceholder.typicode.com/posts/1')
puts response.body
puts response.status
puts response.headers

# POST requests
response = Excon.post('https://jsonplaceholder.typicode.com/posts',
  :body => { :title => 'foo', :body => 'bar', :userId => 1 }.to_json,
  :headers => { 'Content-Type' => 'application/json' } )
puts response.body

Alternatives / Similar