Skip to content

needlevsaxios

MIT 82 4 1,611
33.2 million (month) Dec 11 2011 3.3.1(3 months ago)
103,590 14 648 MIT
1.6.7(a month ago) Aug 29 2014 213.6 million (month)

needle is an HTTP client library for Node.js that provides a simple, flexible, and powerful API for making HTTP requests. It supports all major HTTP methods and has a clean and easy-to-use interface for handling responses and errors.

axios is a popular JavaScript library that allows you to make HTTP requests from a Node.js environment. It is a promise-based library that works in both the browser and Node.js. It is similar to the Fetch API, but with a more powerful feature set and better browser compatibility.

One of the main benefits of using axios is that it automatically transforms the response data into a JSON object, making it easy to work with.

Axios is known for user-friendly API and support for asynchronous async/await syntax making it very accessible in web scraping.

Example Use


const needle = require('needle');

// needle supports both Promises and async/await
needle.get('https://httpbin.org/get', (err, res) => {
    if (err) {
        console.error(err);
        return;
    }
    console.log(res.body);
});

const response = await needle.get('https://httpbin.org/get')

// concurrent requests can be sent using Promise.all
const results = await Promise.all([
  needle.get('http://httpbin.org/html'),
  needle.get('http://httpbin.org/html'),
  needle.get('http://httpbin.org/html'),
])

// POST requests
const data = { name: 'John Doe' };
await needle.post('https://api.example.com', data)

// proxy
const options = {
    proxy: 'http://proxy.example.com:8080'
};
await needle.get('https://httpbin.org/ip', options)

// headers and cookies
const options = {
  headers: {
      'Cookie': 'myCookie=123',
      'X-My-Header': 'myValue'
  }
};
await needle.get('https://httpbin.org/headers', options)
// axios can be used with promises:
axios.get('http://httpbin.org/json')
  .then(response => {
    console.log(response.data);
  })
  .catch(error => {
    console.log(error);
  });

// or async await syntax:
var resp = await axios.get('http://httpbin.org/json');
console.log(resp.data);

// to make requests concurrently Promise.all function can be used:
const results = await Promise.all([
  axios.get('http://httpbin.org/html'),
  axios.get('http://httpbin.org/html'),
  axios.get('http://httpbin.org/html'),
])

// axios also supports other type of requests like POST and even automatically serialize them:
await axios.post('http://httpbin.org/post', {'query': 'hello world'});
// or formdata
const data = {name: 'John Doe', email: 'johndoe@example.com'};

await axios.post('https://jsonplaceholder.typicode.com/users',
    querystring.stringify(data), 
    {
        headers: {
            'Content-Type': 'application/x-www-form-urlencoded'
        }
    }
);

// default values like headers can be configured globally
axios.defaults.headers.common['User-Agent'] = 'webscraping.fyi';
// or for session instance:
const instance = axios.create({
  headers: {"User-Agent": "webscraping.fyi"},
})

Alternatives / Similar