1.3 Reading Tabular Data with the readr
Package
The learning objectives for this section are to:
- Read tabular data into R and read in web data via web scraping tools and APIs
The readr
package is the primary means by which we will read tablular data, most notably, comma-separated-value (CSV) files. The readr
package has a few functions in it for reading and writing tabular data—we will focus on the read_csv
function. The readr
package is available on CRAN and the code for the package is maintained on GitHub.
The importance of the read_csv
function is perhaps better understood from an historical perspective. R’s built in read.csv
function similarly reads CSV files, but the read_csv
function in readr
builds on that by removing some of the quirks and “gotchas” of read.csv
as well as dramatically optimizing the speed with which it can read data into R. The read_csv
function also adds some nice user-oriented features like a progress meter and a compact method for specifying column types.
The only required argument to read_csv
is a character string specifying the path to the file to read. A typical call to read_csv
will look as follows.
library(readr)
<- read_csv("data/team_standings.csv")
teams
── Column specification ────────────────────────────────────────────────────────cols(
Standing = col_double(),
Team = col_character()
)
teams# A tibble: 32 x 2
Standing Team <dbl> <chr>
1 1 Spain
2 2 Netherlands
3 3 Germany
4 4 Uruguay
5 5 Argentina
6 6 Brazil
7 7 Ghana
8 8 Paraguay
9 9 Japan
10 10 Chile
# … with 22 more rows
By default, read_csv
will open a CSV file and read it in line-by-line. It will also (by default), read in the first few rows of the table in order to figure out the type of each column (i.e. integer, character, etc.). In the code example above, you can see that read_csv
has correctly assigned an integer class to the “Standing” variable in the input data and a character class to the “Team” variable. From the read_csv
help page:
If [the argument for
col_types
is] ‘NULL,’ all column types will be imputed from the first 1000 rows on the input. This is convenient (and fast), but not robust. If the imputation fails, you’ll need to supply the correct types yourself.
You can also specify the type of each column with the col_types
argument. In general, it’s a good idea to specify the column types explicitly. This rules out any possible guessing errors on the part of read_csv
. Also, specifying the column types explicitly provides a useful safety check in case anything about the dataset should change without you knowing about it.
<- read_csv("data/team_standings.csv", col_types = "cc") teams
Note that the col_types
argument accepts a compact representation. Here "cc"
indicates that the first column is character
and the second column is character
(there are only two columns). Using the col_types
argument is useful because often it is not easy to automatically figure out the type of a column by looking at a few rows (especially if a column has many missing values).
The read_csv
function will also read compressed files automatically. There is no need to decompress the file first or use the gzfile
connection function. The following call reads a gzip-compressed CSV file containing download logs from the RStudio CRAN mirror.
<- read_csv("data/2016-07-19.csv.gz", n_max = 10)
logs
── Column specification ────────────────────────────────────────────────────────cols(
date = col_date(format = ""),
time = col_time(format = ""),
size = col_double(),
r_version = col_character(),
r_arch = col_character(),
r_os = col_character(),
package = col_character(),
version = col_character(),
country = col_character(),
ip_id = col_double()
)
Note that the message (“Parse with column specification …”) printed after the call indicates that read_csv
may have had some difficulty identifying the type of each column. This can be solved by using the col_types
argument.
<- read_csv("data/2016-07-20.csv.gz", col_types = "ccicccccci", n_max = 10)
logs
logs# A tibble: 10 x 10
date time size r_version r_arch r_os package version country ip_id<chr> <chr> <int> <chr> <chr> <chr> <chr> <chr> <chr> <int>
1 2016-0… 06:04… 144723 3.3.1 i386 mingw… gtools 3.5.0 US 1
2 2016-0… 06:04… 2049711 3.3.0 i386 mingw… rmarkdo… 1.0 DK 2
3 2016-0… 06:04… 26252 <NA> <NA> <NA> R.metho… 1.7.1 AU 3
4 2016-0… 06:04… 556091 2.13.1 x86_64 mingw… tibble 1.1 CN 4
5 2016-0… 06:03… 313363 2.13.1 x86_64 mingw… iterato… 1.0.5 US 5
6 2016-0… 06:03… 378892 3.3.1 x86_64 mingw… foreach 1.3.2 US 5
7 2016-0… 06:04… 41228 3.3.1 x86_64 linux… moments 0.14 US 6
8 2016-0… 06:04… 403177 <NA> <NA> <NA> R.oo 1.20.0 AU 3
9 2016-0… 06:04… 525 3.3.0 x86_64 linux… rgl 0.95.1… KR 7
10 2016-0… 06:04… 755720 3.2.5 x86_64 mingw… geosphe… 1.5-5 US 8
You can specify the column type in a more detailed fashion by using the various col_*
functions. For example, in the log data above, the first column is actually a date, so it might make more sense to read it in as a Date variable. If we wanted to just read in that first column, we could do
<- read_csv("data/2016-07-20.csv.gz",
logdates col_types = cols_only(date = col_date()),
n_max = 10)
logdates# A tibble: 10 x 1
date <date>
1 2016-07-20
2 2016-07-20
3 2016-07-20
4 2016-07-20
5 2016-07-20
6 2016-07-20
7 2016-07-20
8 2016-07-20
9 2016-07-20
10 2016-07-20
Now the date
column is stored as a Date
object which can be used for relevant date-related computations (for example, see the lubridate
package).
A> The read_csv
function has a progress
option that defaults to TRUE. This options provides a nice progress meter while the CSV file is being read. However, if you are using read_csv
in a function, or perhaps embedding it in a loop, it’s probably best to set progress = FALSE
.
The readr
package includes a variety of functions in the read_*
family that allow you to read in data from different formats of flat files. The following table gives a guide to several functions in the read_*
family.
readr function |
Use |
---|---|
read_csv |
Reads comma-separated file |
read_csv2 |
Reads semicolon-separated file |
read_tsv |
Reads tab-separated file |
read_delim |
General function for reading delimited files |
read_fwf |
Reads fixed width files |
read_log |
Reads log files |