axios
axios is a popular JavaScript library that allows you to make HTTP requests from a Node.js environment. It is a promise-based library that works in both the browser and Node.js. It is similar to the Fetch API, but with a more powerful feature set and better browser compatibility.
One of the main benefits of using axios is that it automatically transforms the response data into a JSON object, making it easy to work with.
Axios is known for user-friendly API and support for asynchronous async/await syntax making it very accessible in web scraping.
Example Use
```javascript // axios can be used with promises: axios.get('http://httpbin.org/json') .then(response => { console.log(response.data); }) .catch(error => { console.log(error); });
// or async await syntax: var resp = await axios.get('http://httpbin.org/json'); console.log(resp.data);
// to make requests concurrently Promise.all function can be used: const results = await Promise.all([ axios.get('http://httpbin.org/html'), axios.get('http://httpbin.org/html'), axios.get('http://httpbin.org/html'), ])
// axios also supports other type of requests like POST and even automatically serialize them: await axios.post('http://httpbin.org/post', {'query': 'hello world'}); // or formdata const data = {name: 'John Doe', email: 'johndoe@example.com'};
await axios.post('https://jsonplaceholder.typicode.com/users', querystring.stringify(data), { headers: { 'Content-Type': 'application/x-www-form-urlencoded' } } );
// default values like headers can be configured globally axios.defaults.headers.common['User-Agent'] = 'webscraping.fyi'; // or for session instance: const instance = axios.create({ headers: {"User-Agent": "webscraping.fyi"}, })
```