Now that you have a good grasp of the basics of the Go programming language, it’s the time to put things into action! The easiest way to do that is through web application programming interfaces (APIs). An API is a set of specifications used to communicate between two programs. APIs utilize web technologies, especially Hypertext Transfer Protocol (HTTP), to exchange information between clients and servers. Go provides extensive support for HTTP. This chapter includes Go recipes for working with the net/http package, which contains implementations of the HTTP client and server.
Go and HTTP Calls
This section looks into how to perform HTTP calls in Go through an example. Say you want to post some metrics of an application to the server containing a timestamp, CPU load, and memory. Before diving into Go code to achieve this goal, let’s first look at what an HTTP response looks like. The easiest way to do this is to use the curl command from the command prompt followed by an URL, as illustrated in Figure 7-1.
A structure of H T T P response. It contains the curl command, date, content type, content length, connection, server, and origin. The origin is " 182.185.176.197".
Figure 7-1
Structure of an HTTP response
The HTTP response against the curl command includes information. The first line is the status line indicating which HTTP protocol the server is running, the status code, and an optional status explanation. This is followed by a list of headers, such as Date, Content-Type, and so on. The body of the response is enclosed in the curly braces {}.
The HTTP request looks the same, except for the first line. In the first line, you have the GET /path and then the protocol. For example, GET /ip HTTP/1.1 HOST:httpbin.org, followed by the headers, the empty line, and the request body. You’re going to use httpbin.org to emulate the request to the server and a response.
Now that you know what an HTTP request and response looks like, let’s dive into the recipe for this example of posting an application’s metrics to a server.
In Listing 7-1, the postMetric() function takes a Metric type variable as input and returns any errors. In the function, you use json.Marshal() to marshal the metric into byte slices. Then you create a Context variable called ctx to set a timeout on the request. In this example, we set a timeout of three seconds and deferred the cancel() of the request. A constant stores the URL of the website that you want to generate an HTTP request for. The built-in NewRequestWithContext() function creates a new HTTP request with context. The NewRequestWithContext() function accepts the four parameters: context, method (POST in this case), URL, and request body. Note that the body should be a io.Reader. That’s why you wrap the data in the bytes.NewReader() function.
On top of the data, you set the header of Content-Type to the application/JSON. The http.DefaultClient.Do() function calls the server and checks if any error is returned. Apart from the error, you also check the StatusCode of the request to make sure that its status is OK. After passing the validations, you can parse the HTTP response. The defer keyword ensures that the body of the response is closed when the function exits. You should never blindly read everything from the network, so you should define a maximum size, in this case 1MB. Then you use the io.LimitReader() on the body to make sure only the defined size is read. You define an anonymous structure called reply to store the JSON reply, which is the metric. In Go, an anonymous structure is a structure that does not have a name. They are useful for creating one-time usable structures. To decode the reply, json.NewDecoder() is used. In the end, you log the reply.
Some websites require authentication in the HTTP protocol. There are several methods for authentication, such as Basic, Bearer, Digest, and so on. There are also several formats of authentication, like Auth2, SAML, OICD, and so on. Go supports Basic authentication; for other authentication methods such as Auth0, you need to get a token and then must set the authorization HTTP header.
In the Go recipe shown in Listing 7-2, you are going to use the httpbin.org website to demonstrate Basic authentication. The authRequest() function takes the URL, user, and password as input. These parameters are then used to create a new HTTP request using the http.NewRequest() function. You pass nil for the body since this is a GET request. The SetBasicAuth() built-in function sets the basic authentication with the user and the password supplied to the function. To make a call to the server, use the http.DefaultClient.Do() function. Then you perform validations to check if the returned response has an OK status.
if err := authRequest(user, url, passwd); err != nil {
log.Fatal(err)
}
fmt.Println("OK")
}
Listing 7-2
Go Recipe on Authentication and Writing an HTTP Server in Go
/*Output:
OK
*/
If you were to use the httpbin.org website to generate HTTP requests and responses, the results you’d get are illustrated in Figure 7-2.
A structure of an H T T P response includes a response body and response headers. The response body contains authenticated users. Response headers contain the date, content type and length, connection, and server.
Figure 7-2
HTTP response structure when using httpbin.org
Let’s look at another example, shown in Listing 7-3, this time of an HTTP server that accepts metrics. Here, a metric is a JSON object that has host time, CPU, and memory. In the Go recipe, you define a struct called Metric to match the definitions in the JSON object, that is, the struct has member fields called time, host, CPU, and memory. The handleMetrics() function handles the HTTP requests. It takes input in the w variable, which is of type http.ResponseWriter, used to send the response back to the client, and r of type http.Request. Within the function, you first check to see that the request is a POST request. If it’s not, you issue an error. To limit the amount of data to be read, you set a maximum size, maxSize, in megabytes; this is 1MB in this case. You add the metric to the database and print out that the metric was added. To send the response back to the user, you set the Header of the response first. Now you create a response, which in this case is a map from a string to the empty interfaces with the id that you got from the server. You use json.NewEncoder to encode the response into W, sending it back to the client if there is an error. You cannot change the HTTP status code, so you just log the error.
package main
import (
"encoding/json"
"gorilla/mux"
"io"
"log"
"net/http"
"time"
)
// DB is a database
type DataBase struct{}
// Add adds a metric to the database
func (db *DataBase) Add(m Metric) string {
return "success"
}
type Metric struct {
Time time.Time `json:"time"`
Host string `json:"host"`
CPU float64 `json:"cpu"` //CPU load
Memory float64 `json:"memory"` //MB
}
func handleMetric(w http.ResponseWriter, r *http.Request) {
if r.Method != "POST" {
http.Error(w, "This Method is Not Allowed", http.StatusMethodNotAllowed)
return
}
var db *DataBase
defer r.Body.Close()
var m Metric
const maxSize = 1 << 20 //MB
dec := json.NewDecoder(io.LimitReader(r.Body, maxSize))
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal(err)
}
}
Listing 7-3
Go Recipe on Authentication and Writing HTTP Server That Accepts Metrics
Output:
1.
Open a terminal from the folder where your source code resides and run the file using the go run filename.go command.
2.
Open another terminal and run the following command to send the metric to the server:
A code shows a curl command.
3.
Upon success, you will get the following output on the first terminal running the server:
A code illustrates a go run code.
REST with gorilla/mux
The built-in HTTP server included in the net/http package is great, but it’s also very simple. At times it is possible that you can get by with just using the built-in HTTP server; however, adding external dependencies to a project is a bigger risk than you might think. Refer to the research paper at https://research.swtch.com/deps, which is recommended reading on software dependency problems. It explains why dependencies are risky and why you should avoid them as much as possible. Even though the net/http package provides several functionalities to accomplish different tasks related to the HTTP protocol, in some cases, especially when you’re writing complex APIs, it’s nice to have routers with more features. The net/http package performs poorly when it comes to complex request routing, for example, when splitting up a request URL into single parameters. The gorilla/mux package comes in handy in this situation due to its capabilities of creating routes with named parameters, domain restrictions, and GET/POST handlers.
The gorilla/muxpackage contains the implementations of a request router and dispatcher that can be used to match incoming requests to their respective handlers. Here, mux is the abbreviation for HTTP request multiplexer. The mux.Router present in the gorilla/mux package has the capability of matching any incoming request(s) against a list of registered routes, then calling the respective handler function for the route based on the matching URL or any other condition(s). This section includes an example based on using mux routers.
Say you have a bookstore and the user is requesting a book. In response, the user gets information about the book with the book title, author, and ISBN. When they call it, they’ll use /books/author/ISBN. In Listing 7-4, you start by defining a struct named Book that holds the information about the book. The handler function, handleGetBook(), takes in a ResponseWriter and the request. The gorilla/mux package contains the mux.Vars(r) function, which takes an object of http.Request as an input argument and returns a map containing the segments as the output. To extract the vars with mux.Vars() on the request and get the ISBN from the path, the getBook() function gets the book from the database. If the book is not present in the database, you log and return an error for the client. If the book is in the database, you can set the content-type to application/json and use json.Encoder to encode it back to the client.
if err := http.ListenAndServe(":8080", nil); err != nil {
log.Fatal(err)
}
}
Listing 7-4
Go Recipe on REST Using the gorilla/mux Package
Output:
1.
Open a terminal and enter the gorun command with the file containing this listing. For example, if your filename is code-7.4.go then you would run go run code-7.4.go.
2.
Open another terminal and run the following command to get the book details from the server:
A code illustrates a curl localhost.
Note that to successfully run the code snippets, if you encounter any errors related to importing packages, open the terminal and run these commands:
cd [the dir of your source code]
go mod init
go get github.com/gorilla/mux
Afterward, restart your IDE. For more information, run go help mod.
Hands-on Challenge
For this challenge, you must write key-value pairs in the memory of the HTTP database and its client. To do this, the list of supported functions include
Get() gets the key from the memory
Set() sets the key to the data as received from the standard input
List() lists all the stored keys
You are required to implement code for the client and server sides.
For the client, you are supposed to use the flag package to perform command-line flag parsing. Also, use the switch statement to perform the appropriate function based on the input you received. For the server code, use the gorilla /mux package; the db variable database will be an in-memory map. Also make dbLock of type sync.RWMutex guard the database from deadlocks. You also have handler functions for Get, Set, and List key requests. To perform routing for the server code, you must create a router in main() and use http.Handle() to handle the routing.
Solution
Listings 7-5 and 7-6 show one possible solution of this hands-on challenge. Let’s discuss the highlights of the code.
if err := http.ListenAndServe(addr, nil); err != nil {
log.Fatal(err)
}
}
Listing 7-5
Server-Side Code for the Hands-On Challenge
As shown in Listing 7-5, in the handleSet()function, you first retrieve the keys from the vars and defer the closing of the response body. Then you use LimitReader to limit how much you are going to write. Using ioutil.ReadAll(), you read until the limit. If there is an error, you log it. You look at the database using the built-in Lock() function and defer the unlocking until the function exits. The key is set using db[key] = data. The response is then stored in the resp variable. It contains the key and the size of the data that was sent. The content-type header is set to application-json.
The handleGet()function is also similar. When the Lock() function is used to lock a resource, only one goroutine can read/write at a time by acquiring the lock. Whereas, in the case of RLock(), multiple goroutines can read (not write) at a time by acquiring the lock. In this handler, the comma,ok idiom checks if the desired key is present in the database.
In the handleList()function, you first acquire the lock on the database. Then you create a slice for the names of the keys, iterate over the database, (i.e., the map), and append the keys and write them out as JSON.
Listing 7-6 defines a constant that holds the apiBase. It also defines the list(), set(), and get() functions. The list()function reads the keys and prints them one by one. The set()function sets a value. The get()function retrieves a specific key from the database.
In main(), you create a new router and then route the functions. Here, the Get and Set functions have key as part of the vars in the path; whereas List is just /kv.
Output:
1.
Open a terminal and run the server-side code by using the go run filename.go command. For example, if your filename is code-7.5-server.go, you would use command go run code-7.5-server.go.
2.
Open another terminal and run the following commands. Remember to wait at least one minute after sending a request. After sending the set request and waiting for one minute, press Ctrl+C. Then send the list request to make sure your key was set, as shown in Figure 7-3.
A code illustrates go run code-7.5.
Figure 7-3
Output for running the hands-on solution
Summary
This chapter provided Go recipes to give you hands-on experience handling HTTP calls in Go. Go provides a built-in package called net/http for handling HTTP calls. This chapter included recipes on using the net/http package to send and receive HTTP requests and responses, authenticating and building HTTP servers, and handling REST using the gorilla/mux package.
A killer feature of the Go language that sets it apart from other languages is its built-in support for concurrency. This built-in support makes Go the best choice for programming several different types of applications. The next chapter explains how to build concurrent programs using the Go programming language.
