If you look on the Internet for stock market data, you will quickly find yourself inundated with sources providing stock quotes and financial data. An important but often overlooked factor when acquiring data is the efficiency of getting the data. All else being equal, you don't want to spend hours piecing together a dataset that you could have acquired in far less time. Taking this into consideration, we will try to obtain the largest amount of data from the least number of sources. This not only helps to keep the data as consistent as possible, but it also improves the repeatability of the analysis and the reproducibility of the results.
The first piece of data we want to obtain is a snapshot of the stocks we want to analyze. One of the best ways to do this is to download data from one of the many stock screener applications that exist. Our favorite screener to download stock data from belongs to http://finviz.com.
Let's acquire the stock market data we will use for this chapter with the help of the following steps:
- First, let's pull up FINVIZ.com's stock screener available at http://finviz.com/screener.ashx:
As you can see, the site has multiple fields that can be filtered. If you click on the All tab, you can see all of fields that can be displayed.
- For this project, we want to export all the fields for all the companies in the screener. You can either customize the screener by checking 69 checkboxes, as of the time of writing, or you can use the following URL to make all the fields show up automatically:
You should now see the screener with all the available fields.
- If you scroll all the way to the bottom right of the screen, there should be an export link. Click on this link and save the CSV file as
finviz.csv. - Finally, we will launch RStudio, read the
finviz.csvfile from the path where we saved it, and assign it to a data frame, as follows:finviz <- read.csv("path/finviz.csv") - After going through steps 1 to 4 for the first time (and some clever reading of URLs from our browser), we can replace the previous lines of code with the following two commands:
url_to_open <- 'http://finviz.com/export.ashx?v=152&c=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68' finviz <- read.csv(url(url_to_open))
Tip
Note the structure of the URL in step 2; it contains a comma-separated list of the checkboxes we wish to select. You can programmatically generate this URL to easily select whichever combination of companies' data you want to download.
If you want to avoid typing the numbers 0 through 68, you can use a combination of the sprintf and paste commands to accomplish the same thing:
url_to_open <- sprintf("http://finviz.com/export.ashx?v=152&c=%s", paste(0:68, collapse = ","))