Functions for Reading Data

If you use R’s read.csv function instead, the spaces before columns B and C will be included as part of the data, and the text columns will be interpreted as factors.

The first line in the file will be interpreted as a header, naming the columns, and the remaining three lines as data rows.

When reading the file, read_csv will infer that columns A and D are numbers and columns B and C are strings.

If the filename is a URL (i.e., has prefix http://, https://, ftp://, or ftps://), the file will automatically be downloaded.

This is rarely useful in a data analysis project, but you can use it to create examples or for debugging.

File Headers

The autogenerated column names all start with X and are followed by the number the columns have from left to right in the file.

If there is a header line, but you want to rename the columns, you cannot just provide the names to read_csv using col_names. The first row will still be interpreted as data. This gives you data you do not want in the first row, and it also affects the inferred types of the columns.

For more options affecting how input files are interpreted, read the function documentation: ?read_csv.

Column Types

When read_csv parses a file, it infers the type of each column. This inference can be slow, or worse the inference can be incorrect. If you know a priori what the types should be, you can specify this using the col_types option. If you do this, then read_csv will not make a guess at the types. It will, however, replace values that it cannot parse as of the right type into NA.²

Function-based Column Type Specification

Most of the col_ functions do not take any arguments, but they are affected by the locale parameter the same way that the string specifications are.

For factors, date, time, and datetime types, however, you have more control over the format using the col_ functions. You can use arguments to these functions for specifying how read_csv should parse dates and how it should construct factors.

Parsing Time and Dates

The most complex types to read (or write) are dates and time (and datetime), just because these are written in many different ways. You can specify the format that dates and datetime are in using a string with codes that indicate how time information is represented.

As we saw earlier, you can set the date and time format using the locale() function . If you do not, the default codes will be %AD for dates and %AT for time (there is no locale() argument for datetime). These codes specify YMD and H:M/H:M:S formats, respectively, but are more relaxed in matching the patterns. The date parse, for example, will allow different separators. For dates, both “1975-02-15” and “1975/02/15” will be read as February the 15th 1975, and for time, both “18:00” and “6:00 pm” will be six o’clock in the evening.

In the following text, I give a few examples. I will use the functions parse_date, parse_time, and parse_datetime rather than read_csv with column type specifications. These functions are used by read_csv when you specify a date, time, or datetime column type, but using read_csv for the examples would be unnecessarily verbose. Each takes a vector string representation of dates and time. For more examples, you can read the function documentation ?col_datetime.

Here, I have specified that the input only includes hours and not minutes. If we want hours (and not minutes) in 24-hour clocks, we need to use %H rather than %R.

This is February 1975; remember that the %y code assumes that numbers above 68 are in the twentieth century.

Datetimes can be parsed using combinations of date and time strings. With these, you also want to consider time zones. You can ignore those for dates and time, but unless you are sure that you will never have to consider time zones, you should not rely on the default time zone (which is UTC).⁹

You can either specify that time zones are relative to UTC with %z or location-based, with %Z if the time zone is given in the input, or you can use locale() if it is the same for all the input.

If you specify a time zone based on a location, R will automatically adjust for daylight saving time, but if you use dates relative to UTC, you will not—UTC does not have daylight savings. Central European Time (CET) is “+0100” and with daylight saving time “+0200”. US Pacific Time is “-0800”, but with daylight saving time, it is “-0900”. When you switch back and forth between daylight savings is determined by your location.

The output looks like a string, but the object classes are not character, which, among other things, is why the comparison works.

Space-separated Columns

The preceding functions all read delimiter-separated columns. They expect a single character to separate one column from the next. If the argument trim_ws is true, they ignore whitespace. This argument is true by default for read_csv, read_csv2, and read_tsv, but false for read_delim.

The header names are separated by two spaces. The first data line has spaces before the first line since the string is indented the way it is. Between columns, there are also two spaces . For the second data line, we have several spaces before the first value, once again, but this time only single space between the columns. If we used a delimiter character to specify that we wanted a space to separate columns, we had to have exactly the same number of spaces between each column.

The read_table function instead reads the data as fixed-width columns. It uses the whitespace in the file to figure out the width of the columns. After this, each line will be split into characters that match the width of the columns and assigned to those columns.

Here, the header C is at the position that should separate columns C and D, and these columns are therefore merged.

where the x in the first data line sits between two all-space columns. If you need more specialized fixed-width files, you might want to consider the read_fwf function. See its documentation for details: ?read_fwf.

The read_table and read_table2 functions take many of the same arguments as the delimiter-based parser, so you can, for example, specify column types and set the locale in the same way as the preceding data.

Not part of the main Tidyverse, the packages loaded when you load the package tidyverse, is readxl. Its read_excel function does exactly what it says on the tin; it reads Excel spreadsheets into R. Its interface is similar to the functions in readr. Where the interface differs is in Excel specific options such as which sheet to read. Such options are clearly only needed when reading Excel files.

Functions for Writing Data

Writing data to a file is more straightforward than reading data because we have the data in the correct types and we do not need to deal with different formats. With readr’s writing functions, we have fewer options to format our output—for example, we cannot give the functions a locale() and we cannot specify date and time formatting, but we can use different functions to specify delimiters and time will be output in ISO 8601 which is what the reading functions will use as default.

The functions are write_delim(), write_csv(), write_csv2(), and write_tsv(), and for formats that Excel can read, write_excel_csv() and write_excel_csv2(). The difference between write_csv() and write_excel_csv() and between write_csv2() and write_excel_csv2() is that the Excel functions include a UTF-8 byte order mark so Excel knows that the file is UTF-8 encoded.

The first argument to these functions is the data we want to write and the second is the path to the file we want to write to. If this file has suffix .gz, .bz2, or .xz, the output is automatically compressed.

I will not list all the arguments for these functions here, but you can read the documentation for them from the R console. The argument you are most likely to use is col_names which, if true, means that the function will write the column names as the first line in the output, and if false, will not. If you use write_delim(), you might also want to specify the delimiter character using the delim argument . By default it is a single space; if you write to a file using write_delim() with the default options, you get the data in a format that you can read using read_table2().

The delimiter characters and the decimal points for write_csv(), write_csv2(), and write_tsv are the same as for the corresponding read functions.