axiosvssuperagent
axios is a popular JavaScript library that allows you to make HTTP requests from a Node.js environment. It is a promise-based library that works in both the browser and Node.js. It is similar to the Fetch API, but with a more powerful feature set and better browser compatibility.
One of the main benefits of using axios is that it automatically transforms the response data into a JSON object, making it easy to work with.
Axios is known for user-friendly API and support for asynchronous async/await syntax making it very accessible in web scraping.
superagent is an HTTP client library for Node.js that provides a simple, flexible, and powerful API for making HTTP requests. It supports all major HTTP methods, and has a clean and easy-to-use interface for handling responses and errors.
what differentiates superagent from other http clients is its simple declarative API.
Highlights
Example Use
// axios can be used with promises:
axios.get('http://httpbin.org/json')
.then(response => {
console.log(response.data);
})
.catch(error => {
console.log(error);
});
// or async await syntax:
var resp = await axios.get('http://httpbin.org/json');
console.log(resp.data);
// to make requests concurrently Promise.all function can be used:
const results = await Promise.all([
axios.get('http://httpbin.org/html'),
axios.get('http://httpbin.org/html'),
axios.get('http://httpbin.org/html'),
])
// axios also supports other type of requests like POST and even automatically serialize them:
await axios.post('http://httpbin.org/post', {'query': 'hello world'});
// or formdata
const data = {name: 'John Doe', email: 'johndoe@example.com'};
await axios.post('https://jsonplaceholder.typicode.com/users',
querystring.stringify(data),
{
headers: {
'Content-Type': 'application/x-www-form-urlencoded'
}
}
);
// default values like headers can be configured globally
axios.defaults.headers.common['User-Agent'] = 'webscraping.fyi';
// or for session instance:
const instance = axios.create({
headers: {"User-Agent": "webscraping.fyi"},
})
const superagent = require('superagent');
// superagent supports both Promises and async/await
superagent.get('https://httpbin.org/get')
.then(res => console.log(res.text))
.catch(err => console.error(err));
const response = superagent.get('https://httpbin.org/get')
// post requests:
superagent.post('https://httpbin.org/post').send({ name: 'John Doe' })
// setting proxy
superagent.get('https://httpbin.org/ip').proxy('http://proxy.example.com:8080')
// settings headers and proxies
superagent.get('https://httpbin.org/headers').set('Cookie', 'myCookie=123').set('X-My-Header', 'myValue')