Skip to content

rvest

1,476 1 21 MIT
1.0.4 (20 Aug 2022) Nov 22 2014 611.4 thousand (month)

rvest is a popular R library for web scraping and parsing HTML and XML documents. It is built on top of the xml2 and httr libraries and provides a simple and consistent API for interacting with web pages.

One of the main advantages of using rvest is its simplicity and ease of use. It provides a number of functions that make it easy to extract information from web pages, even for those who are not familiar with web scraping. The html_nodes and html_node functions allow you to select elements from an HTML document using CSS selectors, similar to how you would select elements in JavaScript.

rvest also provides functions for interacting with forms, including html_form, set_values, and submit_form functions. These functions make it easy to navigate through forms and submit data to the server, which can be useful when scraping sites that require authentication or when interacting with dynamic web pages.

rvest also provides functions for parsing XML documents. It includes xml_nodes and xml_node functions, which also use CSS selectors to select elements from an XML document, as well as xml_attrs and xml_attr functions to extract attributes from elements.

Another advantage of rvest is that it provides a way to handle cookies, so you can keep the session alive while scraping a website, and also you can handle redirections with handle_redirects

Example Use


library("rvest")

# Rvest can use basic HTTP client to download remote HTML:
tree <- read_html("http://webscraping.fyi/lib/r/rvest")
# or read from string:
tree <- read_html('
<div class="products">
  <a href="/product/1">Cat Food</a>
  <a href="/product/2">Dog Food</a>
</div>
')

# to parse HTML trees with rvest we use r pipes (the %>% symbol) and html_element function:
# we can use css selectors:
print(tree %>% html_element(".products>a") %>% html_text())
# "[1] "\nCat Food\nDog Food\n""

# or XPath:
print(tree %>% html_element(xpath="//div[@class='products']/a") %>% html_text())
# "[1] "\nCat Food\nDog Food\n""

# Additionally rvest offers many quality of life functions:
# html_text2 - removes trailing and leading spaces and joins values
print(tree %>% html_element("div") %>% html_text2())
# "[1] "Cat Food Dog Food""

# html_attr - selects element's attribute:
print(tree %>% html_element("div") %>% html_attr('class'))
# "products"

Alternatives / Similar


152 2.2.4 (3 years ago) Dec 22 2019 compare
981 1.4.7 (1 year, 4 days ago) May 06 2012 compare
218 1.3.6 (10 months ago) Apr 20 2015 compare
101 1.4.2 (11 months ago) Nov 09 2016 compare

Other Languages

51,415 2.31.0 (11 months ago) Feb 14 2011 compare
22,276 v2.1.0 (3 years ago) May 14 2018 compare
8,650 3.3.2 (5 months ago) Dec 28 2012 compare
104,139 1.6.8 (4 days ago) Aug 29 2014 compare
3,558 7.1.2 (10 months ago) Jul 03 2013 compare
1,074 1.3.0 (7 months ago) Feb 09 2011 compare
4,296 9.1.0 (4 months ago) Aug 28 2011 compare
14,607 3.9.5 (25 days ago) Jul 26 2019 compare
- 4.12.3 (3 months ago) Jul 26 2019 compare
2,580 5.2.1 (a month ago) Dec 13 2022 compare
got
13,956 14.2.1 (2 months ago) Mar 27 2014 compare
7,533 v1.3.4 (4 years ago) Feb 15 2020 compare
12,422 0.27.0 (2 months ago) Jul 26 2019 compare
5,389 0.13.0 (2 years ago) Jul 30 2007 compare
16,538 9.0.2 (12 days ago) Aug 22 2011 compare
27,832 1.0.0-rc.12 (6 months ago) Oct 08 2011 compare
1,617 3.3.1 (4 months ago) Dec 11 2011 compare
2,488 2024-05-09 (3 days ago) Jun 06 2019 compare
1,095 1.1 (3 years ago) Jul 30 2007 compare
281 1.2.0 (1 year, 6 months ago) Apr 14 2012 compare
645 2024-05-11 (16 hours ago) Feb 09 2017 compare
5,665 2.9.0 (4 months ago) Dec 19 2009 compare
6,108 1.16.4 (a month ago) Jul 25 2009 compare
698 2.8.3 (7 years ago) Jul 25 2009 compare
2,272 2.0.0 (1 year, 4 months ago) Dec 05 2008 compare
1,845 6.0.11 (5 months ago) Jun 15 2007 compare
1,154 0.110.0 (a month ago) Oct 31 2009 compare
5,761 0.22.0 (12 days ago) Jul 25 2009 compare
1,073 1.9.1 (a month ago) Jul 26 2019 compare
51,043 2.11.1 (2 months ago) Jul 26 2019 compare
1,041 7.45.3 (2 months ago) Feb 25 2003 compare
13,599 0.10.0 (5 years ago) Feb 25 2018 compare
4,063 1.4.1 (5 months ago) Oct 06 2009 compare
22,795 7.8.0 (8 months ago) Nov 14 2011 compare
663 0.4.12 (5 months ago) Jun 03 2007 compare
1,219 1.1.7 (3 years ago) Oct 25 2009 compare
383 18.1.0 (4 months ago) Aug 06 2011 compare
1,885 v7.1.0-BETA1 (9 days ago) Apr 28 2019 compare
877 0.11.0 (3 years ago) Sep 25 2013 compare
1,487 2.9.0 (a month ago) Jun 01 2013 compare
3,909 v7.1.0-BETA1 (23 days ago) Sep 26 2011 compare
607 1.2.1 (1 year, 10 months ago) Jun 09 2011 compare
583 23.11.0 (6 months ago) Dec 28 2012 compare
974 0.3.21 (2 months ago) Mar 01 2018 compare
2,857 1.4.3 (7 months ago) Sep 04 2013 compare
677 Start (6 years ago) Feb 20 2018 compare
5,630 v0.18.0 (1 year, 1 month ago) Aug 06 2019 compare
2,026 (2 years ago) Nov 20 2016 compare
13,596 v1.9.2 (12 days ago) Aug 29 2016 compare
701 v1.3.1 (a month ago) Feb 07 2019 compare
req
4,023 v3.43.4 (4 days ago) Jun 16 2023 compare
9,408 v2.13.1 (a day ago) Oct 28 2018 compare
2,129 v1.2.5 (2 years ago) Apr 29 2017 compare
655 (a month ago) Jun 08 2019 compare
506 1.1.4 (4 years ago) Apr 20 2010 compare
6,623 1.5.0 (23 hours ago) Sep 10 2012 compare
177 1.2.3 (3 months ago) Jul 30 2007 compare
3,560 v2.0.11 (a month ago) Oct 06 2013 compare
2,891 v2.1.1 (5 months ago) Jul 17 2018 compare
1,921 1.2.1 (2 years ago) Nov 11 2011 compare
732 1.1 (3 years ago) Dec 28 2012 compare
1,736 1.0.0 (10 days ago) Apr 14 2012 compare
2,065 v4.4.10 (5 months ago) Oct 26 2013 compare
5,964 1.1.14 (1 year, 9 months ago) Jul 26 2019 compare
3,012 1.5.0 (2 months ago) Sep 30 2018 compare
1,304 3.0.0 (1 year, 8 months ago) Dec 27 2011 compare
792 0.7.1 (3 months ago) Jul 25 2009 compare
22 0.6.0 (1 year, 16 days ago) Jul 24 2014 compare
3,219 0.9.13 (9 months ago) Jul 04 2017 compare
235 1.33.0 (2 months ago) Feb 05 2023 compare
1,733 0.8.5 (1 year, 8 months ago) Oct 17 2018 compare
2,988 0.12.0 (6 years ago) Mar 20 2015 compare
10,534 1.1.9 (5 years ago) Aug 24 2018 compare
1,321 v3.2.0 (a month ago) Dec 27 2021 compare
493 3.0.0 (a month ago) May 04 2020 compare
197 1.0.0-beta8.4 (10 months ago) Apr 18 2019 compare
1,324 v0.7.2 (5 months ago) Mar 16 2013 compare
412 0.1.3 (9 months ago) Feb 20 2022 compare
298 v1.7.2 (a month ago) Apr 18 2022 compare