Skip to content

node-fetchvscurl-impersonate

MIT 206 10 8,753
228.5 million (month) Dec 28 2012 3.3.2(10 months ago)
3,639 1 64 MIT
Feb 23 2022 v0.6.1(7 months ago)

node-fetch is a lightweight library that provides a fetch()-like API for making HTTP requests in Node.js. It is a light-weight implementation of the Fetch API, which is mostly compatible with the browser's version.

node-fetch is primarily known as almost identical package fetch() is included in web browsers so it shares the same use common API. It's great starting point for people coming from front-end environment.

Curl-impersonate is a special build of libcurl and cURL HTTP client that impersonates the four major browsers: - Google Chrome - Microsoft Edge - Safari - Firefox Curl-impersonate achieves this by patching TLS and HTTP fingerprints to be identical to that of one of these real browsers.

Unlike other HTTP clients curl-impersonate can bypass TSL and HTTP fingerprinting and detection techniques though it does not implement anything for Javascript fingerprint or bypass.

Highlights


popular
bypasshttp2tls-fingerprinthttp-fingerprintlow-level

Example Use


const fetch = require('node-fetch');

// fetch supports both Promises and async/await
fetch('http://httpbin.org/get')
    .then(res => res.text())
    .then(body => console.log(body))
    .catch(err => console.error(err));
const response = await fetch('http://httpbin.org/get');

// for concurrent scraping Promise.all can be used
const results = await Promise.all([
  fetch('http://httpbin.org/html'),
  fetch('http://httpbin.org/html'),
  fetch('http://httpbin.org/html'),
])

// POST requests
await fetch('http://httpbin.org/post', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ name: 'John Doe' }),
})

// Proxy use:
const agent = new https.Agent({
    rejectUnauthorized: false,
    proxy: {
        host: 'proxy.example.com',
        port: 8080
    }
});

await fetch('https://httpbin.org/ip', { agent })

// setting headers and cookies
const headers = new fetch.Headers();
headers.append('Cookie', 'myCookie=123');
headers.append('X-My-Header', 'myValue');

await fetch('https://httpbin.org/headers', { headers })
curl-impersonate installs itself under `curl_` terminal commands like `curl_chrome116`:
$ curl_chrome116 https://www.wikipedia.org
To use it in HTTP client libraries that use `libcurl` replace curl path with one of these. To use it in python directly see curl-cffi Python package

Alternatives / Similar


Was this page helpful?