API is an acronym for application programming interfaces and you can imagine it as a user interface application, but for software instead of humans. Saying it in another way, it could be seen as a tool for your programming. When you need it to insert a nail, you call a hammer API, and if you need it to remove a nail, then you call a plier API. Spark has its own toolbox API for R, which you can access here: https://spark.apache.org/docs/2.2.0/api/R/index.html.

Following our explanation, extensions are customized R packages created to provide an interface to any Spark package, or the Spark toolbox. There are many extensions already available, but you also can create your own extension to call any of these Spark APIs. One extension example is the rsparkling package viewed in the previous section; it is the rsparkling package that provides an interface to the H2O machine learning algorithms.

To access the Spark API available for R, you can call these functions: spark_context(), java_context(), hive_context(), and spark_session(). You must use your spark_connect object as an argument. To see the methods available, please check this site: http://spark.apache.org/docs/latest/api/scala/#package. Select one of the four functions and check the value member methods. To call them, use the invoke(method = "any_value_member") function. Let's create a simple and workable example:

library(sparklyr)
show_version <- function(sc) {
  spark_context(sc) %>% 
    invoke("version") 
}

sc <- spark_connect(master = "local")
show_version(sc)

The preceding code creates a function called show_version, and requires only an object (the Spark spark_connect object). The function calls the spark_context function to provide the "version"  method from the Spark API. If you are not connected to a Spark cluster, remove the hashtag to start one, then call our function to see what your Spark version is. There are more examples presented in the sparklyr documentation, which you can see at https://spark.rstudio.com/extensions/.