In the third and final approach of uploading a picture, we will look at how to allow users to upload an image from their local hard drive to use as their profile picture when chatting. The file will then be served to the browsers via a URL. We will need a way to associate a file with a particular user to ensure that we associate the right picture with the corresponding messages.
In order to uniquely identify our users, we are going to copy Gravatar's approach by hashing their e-mail address and using the resulting string as an identifier. We will store the user ID in the cookie along with the rest of the user-specific data. This will actually have the added benefit of removing the inefficiency associated with continuous hashing from GravatarAuth.
In auth.go, replace the code that creates the authCookieValue object with the following code:
m := md5.New()
io.WriteString(m, strings.ToLower(user.Email()))
userId := fmt.Sprintf("%x", m.Sum(nil))
authCookieValue := objx.New(map[string]interface{}{
"userid": userId,
"name": user.Name(),
"avatar_url": user.AvatarURL(),
"email": user.Email(),
}).MustBase64()
Here, we have hashed the e-mail address and stored the resulting value in the userid field at the point at which the user logs in. From now on, we can use this value in our Gravatar code instead of hashing the e-mail address for every message. To do this, first, we update the test by removing the following line from avatar_test.go:
client.userData = map[string]interface{}{"email": "[email protected]"}
We then replace the preceding line with this line:
client.userData = map[string]interface{}{"userid": "0bc83cb571cd1c50ba6f3e8a78ef1346"}
We no longer need to set the email field since it is not used; instead, we just have to set an appropriate value to the new userid field. However, if you run go test in a terminal, you will see this test fail.
To make the test pass, in avatar.go, update the GetAvatarURL method for the GravatarAuth type:
func(GravatarAvatar) GetAvatarURL(c *client) (string, error) {
if userid, ok := c.userData["userid"]; ok {
if useridStr, ok := userid.(string); ok {
return "//www.gravatar.com/avatar/" + useridStr, nil
}
}
return "", ErrNoAvatarURL
}
This won't change the behavior, but it allows us to make an unexpected optimization, which is a great example of why you shouldn't optimize code too early the inefficiencies that you spot early on may not last long enough to warrant the effort required to fix them.
If our users are to upload a file as their avatar, they need a way to browse their local hard drive and submit the file to the server. We facilitate this by adding a new template-driven page. In the chat/templates folder, create a file called upload.html:
<html>
<head>
<title>Upload</title>
<link rel="stylesheet"
href="//netdna.bootstrapcdn.com/bootstrap/3.6.6/css/bootstrap.min.css">
</head>
<body>
<div class="container">
<div class="page-header">
<h1>Upload picture</h1>
</div>
<form role="form" action="/uploader" enctype="multipart/form-data"
method="post">
<input type="hidden" name="userid" value="{{.UserData.userid}}" />
<div class="form-group">
<label for="avatarFile">Select file</label>
<input type="file" name="avatarFile" />
</div>
<input type="submit" value="Upload" class="btn" />
</form>
</div>
</body>
</html>
We used Bootstrap again to make our page look nice and also to make it fit in with the other pages. However, the key point to note here is the HTML form that will provide the user interface required to upload files. The action points to /uploader, the handler for which we have yet to implement, and the enctype attribute must be multipart/form-data so that the browser can transmit binary data over HTTP. Then, there is an input element of the type file, which will contain a reference to the file we want to upload. Also, note that we have included the userid value from the UserData map as a hidden input this will tell us which user is uploading a file. It is important that the name attributes be correct, as this is how we will refer to the data when we implement our handler on the server.
Let's now map the new template to the /upload path in main.go:
http.Handle("/upload", &templateHandler{filename: "upload.html"})
When the user clicks on Upload after selecting a file, the browser will send the data for the file as well as the user ID to /uploader, but right now, that data doesn't actually go anywhere. We will implement a new HandlerFunc interface that is capable of receiving the file, reading the bytes that are streamed through the connection, and saving it as a new file on the server. In the chat folder, let's create a new folder called avatars this is where we will save the avatar image files.
Next, create a new file called upload.go and insert the following code make sure that you add the appropriate package name and imports (which are ioutils, net/http, io, and path):
func uploaderHandler(w http.ResponseWriter, req *http.Request) {
userId := req.FormValue("userid")
file, header, err := req.FormFile("avatarFile")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
data, err := ioutil.ReadAll(file)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
filename := path.Join("avatars", userId+path.Ext(header.Filename))
err = ioutil.WriteFile(filename, data, 0777)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
io.WriteString(w, "Successful")
}
Here, first uploaderHandler uses the FormValue method in http.Request to get the user ID that we placed in the hidden input in our HTML form. Then, it gets an io.Reader type capable of reading the uploaded bytes by calling req.FormFile, which returns three arguments. The first argument represents the file itself with the multipart.File interface type, which is also io.Reader. The second is a multipart.FileHeader object that contains the metadata about the file, such as the filename. And finally, the third argument is an error that we hope will have a nil value.
What do we mean when we say that the multipart.File interface type is also io.Reader? Well, a quick glance at the documentation at http://golang.org/pkg/mime/multipart/#File makes it clear that the type is actually just a wrapper interface for a few other more general interfaces. This means that a multipart.File type can be passed to methods that require io.Reader, since any object that implements multipart.File must, therefore, implement io.Reader.
Tip
Embedding standard library interfaces, such as the wrapper, to describe new concepts is a great way to make sure your code works in as many contexts as possible. Similarly, you should try to write code that uses the simplest interface type you can find, ideally from the standard library. For example, if you wrote a method that needed you to read the contents of a file, you could ask the user to provide an argument of the type multipart.File. However, if you ask for io.Reader instead, your code will become significantly more flexible because any type that has the appropriate Read method can be passed in, which includes user-defined types as well.
The ioutil.ReadAll method will just keep reading from the specified io.Reader interface until all of the bytes have been received, so this is where we actually receive the stream of bytes from the client. We then use path.Join and path.Ext to build a new filename using userid and copy the extension from the original filename that we can get from multipart.FileHeader.
We then use the ioutil.WriteFile method to create a new file in the avatars folder. We use userid in the filename to associate the image with the correct user, much in the same way as Gravatar does. The 0777 value specifies that the new file we create should have complete file permissions, which is a good default setting if you're not sure what other permissions should be set.
If an error occurs at any stage, our code will write it out to the response along with a 500 status code (since we specify http.StatusInternalServerError), which will help us debug it, or it will write Successful if everything went well.
In order to map this new handler function to /uploader, we need to head back to main.go and add the following line to func main:
http.HandleFunc("/uploader", uploaderHandler)
Now build and run the application and remember to log out and log back in again in order to give our code a chance to upload the auth cookie:
go build -o chat ./chat -host=:8080
Open http://localhost:8080/upload and click on Choose File, and then select a file from your hard drive and click on Upload. Navigate to your chat/avatars folder and you will notice that the file was indeed uploaded and renamed to the value of your userid field.
Now that we have a place to keep our user's avatar images on the server, we need a way to make them accessible to the browser. We do this using the net/http package's built-in file server. In main.go, add the following code:
http.Handle("/avatars/",
http.StripPrefix("/avatars/",
http.FileServer(http.Dir("./avatars"))))
This is actually a single line of code that has been broken up to improve readability. The http.Handle call should feel familiar, as we are specifying that we want to map the /avatars/ path with the specified handler this is where things get interesting. Both http.StripPrefix and http.FileServer return http.Handler, and they make use of the wrapping pattern we learned about in the previous chapter. The StripPrefix function takes http.Handler in, modifies the path by removing the specified prefix, and passes the functionality onto an inner handler. In our case, the inner handler is an http.FileServer handler that will simply serve static files, provide index listings, and generate the 404 Not Found error if it cannot find the file. The http.Dir function allows us to specify which folder we want to expose publicly.
Let's build the program and run it before opening http://localhost:8080/avatars/ in a browser. You'll notice that the file server has generated a listing of the files inside our avatars folder. Clicking on a file will either download the file, or in the case of an image, simply display it. If you haven't done this already, go to http://localhost:8080/upload and upload a picture, and then head back to the listing page and click on it to see it in the browser.
The final step in making filesystem avatars work is writing an implementation of our Avatar interface that generates URLs that point to the filesystem endpoint we created in the previous section.
Let's add a test function to our avatar_test.go file:
func TestFileSystemAvatar(t *testing.T) {
filename := filepath.Join("avatars", "abc.jpg")
ioutil.WriteFile(filename, []byte{}, 0777)
defer os.Remove(filename)
var fileSystemAvatar FileSystemAvatar
client := new(client)
client.userData = map[string]interface{}{"userid": "abc"}
url, err := fileSystemAvatar.GetAvatarURL(client)
if err != nil {
t.Error("FileSystemAvatar.GetAvatarURL should not return an error")
}
if url != "/avatars/abc.jpg" {
t.Errorf("FileSystemAvatar.GetAvatarURL wrongly returned %s", url)
}
}
This test is similar to, but slightly more involved than, the GravatarAvatar test because we are also creating a test file in our avatars folder and deleting it afterwards.
The rest of the test is simple: we set a userid field in client.userData and call GetAvatarURL to ensure we get the right value back. Of course, running this test will fail, so let's go and add the following code in order to make it pass in avatar.go:
type FileSystemAvatar struct{}
var UseFileSystemAvatar FileSystemAvatar
func (FileSystemAvatar) GetAvatarURL(c *client) (string, error) {
if userid, ok := c.userData["userid"]; ok {
if useridStr, ok := userid.(string); ok {
return "/avatars/" + useridStr + ".jpg", nil
}
}
return "", ErrNoAvatarURL
}
As you can see here, in order to generate the correct URL, we simply get the userid value and build the final string by adding the appropriate segments together. You may have noticed that we have hardcoded the file extension to .jpg, which means that the initial version of our chat application will only support JPEGs.
Let's look at our new code in action by updating main.go to use our new Avatar implementation:
r := newRoom(UseFileSystemAvatar)
Now build and run the application as usual and go to http://localhost:8080/upload and use a web form to upload a JPEG image to use as your profile picture. To make sure it's working correctly, choose a unique image that isn't your Gravatar picture or the image from the auth service. Once you see the successful message after clicking on Upload, go to http://localhost:8080/chat and post a message. You will notice that the application has indeed used the profile picture that you uploaded.
To change your profile picture, go back to the /upload page and upload a different picture, and then jump back to the /chat page and post more messages.
To support different file types, we have to make our GetAvatarURL method for the FileSystemAvatar type a little smarter.
Instead of just blindly building the string, we will use the very important ioutil.ReadDir method to get a listing of the files. The listing also includes directories so we will use the IsDir method to determine whether we should skip it or not.
We will then check whether each file matches the userid field (remember that we named our files in this way) by a call to path.Match. If the filename matches the userid field, then we have found the file for that user and we return the path. If anything goes wrong or if we can't find the file, we return the ErrNoAvatarURL error as usual.
Update the appropriate method in avatar.go with the following code:
func (FileSystemAvatar) GetAvatarURL(c *client) (string, error) {
if userid, ok := c.userData["userid"]; ok {
if useridStr, ok := userid.(string); ok {
files, err := ioutil.ReadDir("avatars")
if err != nil {
return "", ErrNoAvatarURL
}
for _, file := range files {
if file.IsDir() {
continue
}
if match, _ := path.Match(useridStr+"*", file.Name());
match {
return "/avatars/" + file.Name(), nil
}
}
}
}
return "", ErrNoAvatarURL
}
Delete all the files in the avatar folder to prevent confusion and rebuild the program. This time, upload an image of a different type and note that our application has no difficulty handling it.
When we look back at how our Avatar type is used, you will notice that every time someone sends a message, the application makes a call to GetAvatarURL. In our latest implementation, each time the method is called, we iterate over all the files in the avatars folder. For a particularly chatty user, this could mean that we end up iterating over and over again many times a minute. This is an obvious waste of resources and would, at some point very soon, become a scaling problem.
Instead of getting the avatar URL for every message, we should get it only once when the user first logs in and cache it in the auth cookie. Unfortunately, our Avatar interface type requires that we pass in a client object to the GetAvatarURL method and we do not have such an object at the point at which we are authenticating the user.
Tip
So did we make a mistake when we designed our Avatar interface? While this is a natural conclusion to come to, in fact we did the right thing. We designed the solution with the best information we had available at the time and therefore had a working chat application much sooner than if we'd tried to design for every possible future case. Software evolves and almost always changes during the development process and will definitely change throughout the lifetime of the code.
We have concluded that our GetAvatarURL method depends on a type that is not available to us at the point we need it, so what would be a good alternative? We could pass each required field as a separate argument, but this would make our interface brittle, since as soon as an Avatar implementation needs a new piece of information, we'd have to change the method signature. Instead, we will create a new type that will encapsulate the information our Avatar implementations need while conceptually remaining decoupled from our specific case.
In auth.go, add the following code to the top of the page (underneath the package keyword, of course):
import gomniauthcommon "github.com/stretchr/gomniauth/common"
type ChatUser interface {
UniqueID() string
AvatarURL() string
}
type chatUser struct {
gomniauthcommon.User
uniqueID string
}
func (u chatUser) UniqueID() string {
return u.uniqueID
}
Here, the import statement imported the common package from Gomniauth and, at the same time, gave it a specific name through which it will be accessed: gomniauthcommon. This isn't entirely necessary since we have no package name conflicts. However, it makes the code easier to understand.
In the preceding code snippet, we also defined a new interface type called ChatUser, which exposes the information needed in order for our Avatar implementations to generate the correct URLs. Then, we defined an actual implementation called chatUser (notice the lowercase starting letter) that implements the interface. It also makes use of a very interesting feature in Go: type embedding. We actually embedded the gomniauth/common.User interface type, which means that our struct interface implements the interface automatically.
You may have noticed that we only actually implemented one of the two required methods to satisfy our ChatUser interface. We got away with this because the Gomniauth User interface happens to define the same AvatarURL method. In practice, when we instantiate our chatUser struct provided we set an appropriate value for the implied Gomniauth User field our object implements both Gomniauth's User interface and our own ChatUser interface at the same time.
Before we can use our new type, we must update the Avatar interface and appropriate implementations to make use of it. As we will follow TDD practices, we are going to first make these changes in our test file, see the compiler errors when we try to build our code, and see failing tests once we fix those errors before finally making the tests pass.
Open avatar_test.go and replace TestAuthAvatar with the following code:
func TestAuthAvatar(t *testing.T) {
var authAvatar AuthAvatar
testUser := &gomniauthtest.TestUser{}
testUser.On("AvatarURL").Return("", ErrNoAvatarURL)
testChatUser := &chatUser{User: testUser}
url, err := authAvatar.GetAvatarURL(testChatUser)
if err != ErrNoAvatarURL {
t.Error("AuthAvatar.GetAvatarURL should return ErrNoAvatarURL
when no value present")
}
testUrl := "http://url-to-gravatar/"
testUser = &gomniauthtest.TestUser{}
testChatUser.User = testUser
testUser.On("AvatarURL").Return(testUrl, nil)
url, err = authAvatar.GetAvatarURL(testChatUser)
if err != nil {
t.Error("AuthAvatar.GetAvatarURL should return no error
when value present")
}
if url != testUrl {
t.Error("AuthAvatar.GetAvatarURL should return correct URL")
}
}
Using our new interface before we have defined it is a good way to check the sanity of our thinking, which is another advantage of practicing TDD. In this new test, we create TestUser provided by Gomniauth and embed it into a chatUser type. We then pass the new chatUser type into our GetAvatarURL calls and make the same assertions about output as we always have done.
Tip
Gomniauth's TestUser type is interesting as it makes use of the Testify package's mocking capabilities. Refer to https://github.com/stretchr/testify for more information.
The On and Return methods allow us to tell TestUser what to do when specific methods are called. In the first case, we tell the AvatarURL method to return the error, and in the second case, we ask it to return the testUrl value, which simulates the two possible outcomes we are covering in this test.
Updating the other two tests is much simpler because they rely only on the UniqueID method, the value of which we can control directly.
Replace the other two tests in avatar_test.go with the following code:
func TestGravatarAvatar(t *testing.T) {
var gravatarAvatar GravatarAvatar
user := &chatUser{uniqueID: "abc"}
url, err := gravatarAvatar.GetAvatarURL(user)
if err != nil {
t.Error("GravatarAvatar.GetAvatarURL should not return an error")
}
if url != "//www.gravatar.com/avatar/abc" {
t.Errorf("GravatarAvatar.GetAvatarURL wrongly returned %s", url)
}
}
func TestFileSystemAvatar(t *testing.T) {
// make a test avatar file
filename := path.Join("avatars", "abc.jpg")
ioutil.WriteFile(filename, []byte{}, 0777)
defer func() { os.Remove(filename) }()
var fileSystemAvatar FileSystemAvatar
user := &chatUser{uniqueID: "abc"}
url, err := fileSystemAvatar.GetAvatarURL(user)
if err != nil {
t.Error("FileSystemAvatar.GetAvatarURL should not return an error")
}
if url != "/avatars/abc.jpg" {
t.Errorf("FileSystemAvatar.GetAvatarURL wrongly returned %s", url)
}
}
Of course, this test code won't even compile because we are yet to update our Avatar interface. In avatar.go, update the GetAvatarURL signature in the Avatar interface type to take a ChatUser type rather than a client type:
GetAvatarURL(ChatUser) (string, error)
Trying to build this will reveal that we now have broken implementations because all the GetAvatarURL methods are still asking for a client object.
Changing an interface like the one we have is a good way to automatically find the parts of our code that have been affected because they will cause compiler errors. Of course, if we were writing a package that other people would use, we would have to be far stricter about changing the interfaces like this, but we haven't released our v1 yet, so it's fine.
We are now going to update the three implementation signatures to satisfy the new interface and change the method bodies to make use of the new type. Replace the implementation for FileSystemAvatar with the following:
func (FileSystemAvatar) GetAvatarURL(u ChatUser) (string, error) {
if files, err := ioutil.ReadDir("avatars"); err == nil {
for _, file := range files {
if file.IsDir() {
continue
}
if match, _ := path.Match(u.UniqueID()+"*", file.Name());
match {
return "/avatars/" + file.Name(), nil
}
}
}
return "", ErrNoAvatarURL
}
The key change here is that we no longer access the userData field on the client, and just call UniqueID directly on the ChatUser interface instead.
Next, we update the AuthAvatar implementation with the following code:
func (AuthAvatar) GetAvatarURL(u ChatUser) (string, error) {
url := u.AvatarURL()
if len(url) == 0 {
return "", ErrNoAvatarURL
}
return url, nil
}
Our new design proves to be much simpler, it's always a good thing if we can reduce the amount of code required. The preceding code makes a call to get the AvatarURL value, and provided it isn't empty, we return it; otherwise, we return the ErrNoAvatarURL error.
Tip
Note how the expected flow of the code is indented to one level, while error cases are nested inside if blocks. While you can't stick to this practice 100% of the time, it's a worthwhile endeavor. Being able to quickly scan the code (when reading it) to see the normal flow of execution down a single column allows you to understand the code much quicker. Compare this to code that has lots of if...else nested blocks, which takes a lot more unpicking to understand.
Finally, update the GravatarAvatar implementation:
func (GravatarAvatar) GetAvatarURL(u ChatUser) (string, error) {
return "//www.gravatar.com/avatar/" + u.UniqueID(), nil
}
So far, we have assigned the Avatar implementation to the room type, which enables us to use different avatars for different rooms. However, this has exposed an issue: when our users sign in, there is no concept of which room they are headed to so we cannot know which Avatar implementation to use. Because our application only supports a single room, we are going to look at another approach to select implementations: the use of global variables.
A global variable is simply a variable that is defined outside any type definition and is accessible from every part of the package (and from outside the package if it's exported). For a simple configuration, such as which type of Avatar implementation to use, global variables are an easy and simple solution. Underneath the import statements in main.go, add the following line:
// set the active Avatar implementation var avatars Avatar = UseFileSystemAvatar
This defines avatars as a global variable that we can use when we need to get the avatar URL for a particular user.
We need to change the code that calls GetAvatarURL for every message to just access the value that we put into the userData cache (via the auth cookie). Change the line where msg.AvatarURL is assigned, as follows:
if avatarUrl, ok := c.userData["avatar_url"]; ok {
msg.AvatarURL = avatarUrl.(string)
}
Find the code inside loginHandler in auth.go where we call provider.GetUser and replace it, down to where we set the authCookieValue object, with the following code:
user, err := provider.GetUser(creds)
if err != nil {
log.Fatalln("Error when trying to get user from", provider, "-", err)
}
chatUser := &chatUser{User: user}
m := md5.New()
io.WriteString(m, strings.ToLower(user.Email()))
chatUser.uniqueID = fmt.Sprintf("%x", m.Sum(nil))
avatarURL, err := avatars.GetAvatarURL(chatUser)
if err != nil {
log.Fatalln("Error when trying to GetAvatarURL", "-", err)
}
Here, we created a new chatUser variable while setting the User field (which represents the embedded interface) to the User value returned from Gomniauth. We then saved the userid MD5 hash to the uniqueID field.
The call to avatars.GetAvatarURL is where all of our hard work has paid off, as we now get the avatar URL for the user far earlier in the process. Update the authCookieValue line in auth.go to cache the avatar URL in the cookie and remove the e-mail address since it is no longer required:
authCookieValue := objx.New(map[string]interface{}{
"userid": chatUser.uniqueID,
"name": user.Name(),
"avatar_url": avatarURL,
}).MustBase64()
However expensive the work the Avatar implementation needs to do, such as iterating over files on the filesystem, it is mitigated by the fact that the implementation only does so when the user first logs in and not every time they send a message.
Finally, we get to snip away at some of the fat that has accumulated during our refactoring process.
Since we no longer store the Avatar implementation in room, let's remove the field and all references to it from the type. In room.go, delete the avatar Avatar definition from the room struct and update the newRoom method:
func newRoom() *room {
return &room{
forward: make(chan *message),
join: make(chan *client),
leave: make(chan *client),
clients: make(map[*client]bool),
tracer: trace.Off(),
}
}
In main.go, remove the parameter passed into the newRoom function call since we are using our global variable instead of this one.
After this exercise, the end user experience remains unchanged. Usually when refactoring the code, it is the internals that are modified while the public-facing interface remains stable and unchanged. As you go, remember to re-run the unit tests to make sure you don't break anything as you evolve the code.