Our first approach to providing a better experience when an internet connection is slow or unreliable is to implement a caching mechanism for our remote resources. With this in place, a single online run of the application would be sufficient to defend against connectivity problems as it'll populate the cache data. The bonus is that, on repeated runs of the application, it'll be much faster to load these resources.

Building on the previous image example, we implement a new function, cacheStream(), which we'll call instead of readStream(). A helper function called cacheFileName() establishes a file location to use for the cache based on a url parameter. Every time we request a URL using this function, it'll attempt to load a cached copy from that location; if it's present, then io.ReadCloser to this location will be returned directly. If the cache file isn't present, then we use the original readStream() function to download the content into the cache file and then return a stream to the cache file as before:

func cacheFileName(u string) string {
   id, _ := url.Parse(u)
   file := filepath.Base(id.Path)
   return path.Join("/tmp/", fmt.Sprintf("%s:%s", id.Hostname(), file))
}

func cacheStream(url string) io.ReadCloser {
   cacheFile := cacheFileName(url)
   if _, err := os.Stat(cacheFile); !os.IsNotExist(err) {
      fmt.Println("Found cached file at", cacheFile)
      file, _ := os.Open(cacheFile)
      return file
   }

   fmt.Println("No cache found, downloading")
   stream := readStream(url)
   writer, _ := os.Create(cacheFile)
   io.Copy(writer, stream)
   stream.Close()
   writer.Close()

   fmt.Println("Saved to", cacheFile)
   stream, _ = os.Open(cacheFile)
   return stream
}

This implementation is just an illustration of how this can be done; if it were to be used in a production application, you would need to use a better cache location and handle potential thread issues.

The benefit of the stream-based approach is that we can use it for assets other than images. Just like the image example, we can update our JSON code to use cacheStream() instead of readStream(), and our data will be downloaded once and then read from the local file by the cache code:

By caching the JSON, our application can function if the network fails

These examples should help to work with remote resources in your application, but are relatively simple examples. How do we work with more complicated cloud services?