The gob package (https://golang.org/pkg/encoding/gob) provides an encoding format that can be used to convert complex Go data types into binary. Gob is self-describing, meaning each encoded data item is accompanied by a type description. The encoding process involves streaming the gob-encoded data to an io.Writer so it can be written to a resource for future consumption.

The following snippet shows an example code that encodes variable books, a slice of the Book type with nested values, into the gob format. The encoder writes its generated binary data to an os.Writer instance, in this case the file variable of the *os.File type:

type Name struct { 
   First, Last string 
} 
 
type Book struct { 
   Title       string 
   PageCount   int 
   ISBN        string 
   Authors     []Name 
   Publisher   string 
   PublishDate time.Time 
} 
 
func main() { 
   books := []Book{ 
         Book{ 
               Title:       "Leaning Go", 
               PageCount:   375, 
               ISBN:        "9781784395438", 
               Authors:     []Name{{"Vladimir", "Vivien"}}, 
               Publisher:   "Packt", 
               PublishDate: time.Date( 
                     2016, time.July, 
                     0, 0, 0, 0, 0, time.UTC, 
               ), 
         }, 
         Book{ 
               Title:       "The Go Programming Language", 
               PageCount:   380, 
               ISBN:        "9780134190440", 
               Authors:     []Name{ 
                     {"Alan", "Donavan"}, 
                     {"Brian", "Kernighan"}, 
               }, 
               Publisher:   "Addison-Wesley", 
               PublishDate: time.Date( 
                     2015, time.October, 
                     26, 0, 0, 0, 0, time.UTC, 
               ), 
         }, 
         ... 
   } 
 
   // serialize data structure to file 
   file, err := os.Create("book.dat") 
   if err != nil { 
         fmt.Println(err) 
         return 
   } 
   enc := gob.NewEncoder(file) 
   if err := enc.Encode(books); err != nil { 
         fmt.Println(err) 
   } 
} 

golang.fyi/ch10/gob0.go

Although the previous example is lengthy, it is mostly made of the definition of the nested data structure assigned to variable books. The last half-dozen or more lines are where the encoding takes place. The gob encoder is created with enc := gob.NewEncoder(file). Encoding the data is done by simply calling enc.Encode(books) which streams the encoded data to the provide file.

The decoding process does the reverse by streaming the gob-encoded binary data using an io.Reader and automatically reconstructing it as a strongly-typed Go value. The following code snippet decodes the gob data that was encoded and stored in the books.data file in the previous example. The decoder reads the data from an io.Reader, in this instance the variable file of the *os.File type:

type Name struct { 
   First, Last string 
} 
 
type Book struct { 
   Title       string 
   PageCount   int 
   ISBN        string 
   Authors     []Name 
   Publisher   string 
   PublishDate time.Time 
} 
 
func main() { 
   file, err := os.Open("book.dat") 
   if err != nil { 
         fmt.Println(err) 
         return 
   } 
 
   var books []Book 
   dec := gob.NewDecoder(file) 
   if err := dec.Decode(&books); err != nil { 
         fmt.Println(err) 
         return 
   } 
} 

golang.fyi/ch10/gob1.go

Decoding a previously encoded gob data is done by creating a decoder using dec := gob.NewDecoder(file). The next step is to declare the variable that will store the decoded data. In our example, the books variable, of the []Book type, is declared as the destination of the decoded data. The actual decoding is done by invoking dec.Decode(&books). Notice the Decode() method takes the address of its target variable as an argument. Once decoded, the books variable will contain the reconstituted data structure streamed from the file.

The encoding package also comes with a json encoder sub-package (https://golang.org/pkg/encoding/json/) to support JSON-formatted data. This greatly broadens the number of languages with which Go programs can exchange complex data structures. JSON encoding works similarly as the encoder and decoder from the gob package. The difference is that the generated data takes the form of a clear text JSON-encoded format instead of a binary. The following code updates the previous example to encode the data as JSON:

type Name struct { 
   First, Last string 
} 
 
type Book struct { 
   Title       string 
   PageCount   int 
   ISBN        string 
   Authors     []Name 
   Publisher   string 
   PublishDate time.Time 
} 
 
func main() { 
   books := []Book{ 
         Book{ 
               Title:       "Leaning Go", 
               PageCount:   375, 
               ISBN:        "9781784395438", 
               Authors:     []Name{{"Vladimir", "Vivien"}}, 
               Publisher:   "Packt", 
               PublishDate: time.Date( 
                     2016, time.July, 
                     0, 0, 0, 0, 0, time.UTC), 
         }, 
         ... 
   } 
 
   file, err := os.Create("book.dat") 
   if err != nil { 
         fmt.Println(err) 
         return 
   } 
   enc := json.NewEncoder(file) 
   if err := enc.Encode(books); err != nil { 
         fmt.Println(err) 
   } 
} 

golang.fyi/ch10/json0.go

The code is exactly the same as before. It uses the same slice of nested structs assigned to the books variable. The only difference is the encoder is created with enc := json.NewEncoder(file) which creates a JSON encoder that will use the file variable as its io.Writer destination. When enc.Encode(books) is executed, the content of the variable books is serialized as JSON to the local file books.dat, shown in the following code (formatted for readability):

[     
      {
            "Title":"Leaning Go",
            "PageCount":375,
            "ISBN":"9781784395438",
            "Authors":[{"First":"Vladimir","Last":"Vivien"}],
            "Publisher":"Packt",
            "PublishDate":"2016-06-30T00:00:00Z"
      },
      {
            "Title":"The Go Programming Language",
            "PageCount":380,
            "ISBN":"9780134190440",
            "Authors":[
                  {"First":"Alan","Last":"Donavan"},
                      {"First":"Brian","Last":"Kernighan"}
            ],
            "Publisher":"Addison-Wesley",
            "PublishDate":"2015-10-26T00:00:00Z"
      },
      ...
]

File books.dat (formatted)

The generated JSON-encoded content uses the name of the struct fields as the name for the JSON object keys by default. This behavior can be controlled using struct tags (see the section, Controlling JSON mapping with struct tags).

Consuming the JSON-encoded data in Go is done using a JSON decoder that streams its source from an io.Reader. The following snippet decodes the JSON-encoded data, generated in the previous example, stored in the file book.dat. Note that the data structure (not shown in the following code) is the same as before:

func main() { 
   file, err := os.Open("book.dat") 
   if err != nil { 
         fmt.Println(err) 
         return 
   } 
 
   var books []Book 
   dec := json.NewDecoder(file) 
   if err := dec.Decode(&books); err != nil { 
         fmt.Println(err) 
         return 
   } 
} 

golang.fyi/ch10/json1.go

The data in the books.dat file is stored as an array of JSON objects. Therefore, the code must declare a variable capable of storing an indexed collection of nested struct values. In the previous example, the books variable, of the type []Book is declared as the destination of the decoded data. The actual decoding is done by invoking dec.Decode(&books). Notice the Decode() method takes the address of its target variable as an argument. Once decoded, the books variable will contain the reconstituted data structure streamed from the file.

By default, the name of a struct field is used as the key for the generated JSON object. This can be controlled using struct type tags to specify how JSON object key names are mapped during encoding and decoding of the data. For instance, the following code snippet declares struct fields with the json: tag prefix to specify how object keys are to be encoded and decoded:

type Book struct { 
   Title       string      `json:"book_title"` 
   PageCount   int         `json:"pages,string"` 
   ISBN        string      `json:"-"` 
   Authors     []Name      `json:"auths,omniempty"` 
   Publisher   string      `json:",omniempty"` 
   PublishDate time.Time   `json:"pub_date"` 
} 

golang.fyi/ch10/json2.go

The tags and their meaning are summarized in the following table:

When the previous struct is encoded, it produces the following JSON output in the books.dat file (formatted for readability):

... 
{ 
   "book_title":"The Go Programming Language", 
   "pages":"380", 
   "auths":[ 
         {"First":"Alan","Last":"Donavan"}, 
         {"First":"Brian","Last":"Kernighan"} 
   ], 
   "Publisher":"Addison-Wesley", 
   "pub_date":"2015-10-26T00:00:00Z" 
} 
... 

Notice the JSON object keys are titled as specified in the struct tags. The object key "pages" (mapped to the struct field, PageCount) is encoded as a string. Finally, the struct field, ISBN, is omitted, as annotated in the struct tag.

The JSON package uses two interfaces, Marshaler and Unmarshaler, to hook into encoding and decoding events respectively. When the encoder encounters a value whose type implements json.Marshaler, it delegates serialization of the value to the method MarshalJSON defined in the Marshaller interface. This is exemplified in the following abbreviated code snippet where the type Name is updated to implement json.Marshaller as shown:

type Name struct { 
   First, Last string 
} 
func (n *Name) MarshalJSON() ([]byte, error) { 
   return []byte( 
         fmt.Sprintf(""%s, %s"", n.Last, n.First) 
   ), nil 
} 
 
type Book struct { 
   Title       string 
   PageCount   int 
   ISBN        string 
   Authors     []Name 
   Publisher   string 
   PublishDate time.Time 
} 
func main(){ 
   books := []Book{ 
         Book{ 
               Title:       "Leaning Go", 
               PageCount:   375, 
               ISBN:        "9781784395438", 
               Authors:     []Name{{"Vladimir", "Vivien"}}, 
               Publisher:   "Packt", 
               PublishDate: time.Date( 
                     2016, time.July, 
                     0, 0, 0, 0, 0, time.UTC), 
         }, 
         ... 
   } 
   ... 
   enc := json.NewEncoder(file) 
   if err := enc.Encode(books); err != nil { 
         fmt.Println(err) 
   } 
} 

golang.fyi/ch10/json3.go

In the previous example, values of the Name type are serialized as a JSON string (instead of an object as earlier). The serialization is handled by the method Name.MarshallJSON which returns an array of bytes that contains the last and first name separated by a comma. The preceding code generates the following JSON output:

    [
          ...
          {
                "Title":"Leaning Go",
                "PageCount":375,
                "ISBN":"9781784395438",
                "Authors":["Vivien, Vladimir"],
                "Publisher":"Packt",
                "PublishDate":"2016-06-30T00:00:00Z"
          },
          ...
    ]

For the inverse, when a decoder encounters a piece of JSON text that maps to a type that implements json.Unmarshaler, it delegates the decoding to the type's UnmarshalJSON method. For instance, the following shows the abbreviated code snippet that implements json.Unmarshaler to handle the JSON output for the Name type:

type Name struct { 
   First, Last string 
} 
 
func (n *Name) UnmarshalJSON(data []byte) error { 
   var name string 
   err := json.Unmarshal(data, &name) 
   if err != nil { 
         fmt.Println(err) 
         return err 
   } 
   parts := strings.Split(name, ", ") 
   n.Last, n.First = parts[0], parts[1] 
   return nil 
} 

golang.fyi/ch10/json4.go

The Name type is an implementation of json.Unmarshaler. When the decoder encounters a JSON object with the key "Authors", it uses the method Name.Unmarshaler to reconstitute the Go struct Name type from the JSON string.