Most of the data on the web is in large scale as HTML. It is often not available in a form that is useful for analysis, such as hierarchical or tree-based:
<html>
<head>
<title>Looks like a tittle</title>
</head>
<body>
<p align="center">What's up ?</p>
</body>
</html>
rvest is a very useful R library that helps you collect information from web pages. It is designed to work with magrittr, inspired by libraries such as BeatifulSoup.
To start the web scraping process, you first need to master the R bases. In this section, we will perform web scraping step by step, using the rvest R package written by Hadley Wickham.
For more information about the rvesr package, visit the following URLs.CRAN Page: https://cran.r-project.org/web/packages/rvest/index.html rvest on github: https://github.com/hadley/rvest.
Make sure this package is installed. If you do not have this package right now, you can use the following code to install it: install.packages('rvest').
Let's take a look at some important functions in rvest:
| Function | Description |
| read_html() | Create an html document from a URL, a file on a disk, or a string containing HTML. |
| html_nodes(doc, "table td") | Select parts of a document using css selectors. |
| html_nodes(doc, xpath = "//table//td") |
Select parts of a document using xpath selectors. |
| html_tag() | Extract components with the name of the tag. |
| html_text() | Extract text from html document. |
| html_attr() | Get a single html attribute. |
| html_attrs() | Get all HTML attributes. |
| xml() | Working with XML files. |
| xml_node() | Extract XML components. |
| html_table() | Parse HTML tables into a data frame. |
| html_form() set_values() submit_form() |
Extract, modify, and submit forms. |
| guess_encoding() , repair_encoding() |
Detect and repair problems regarding encoding. |