JSON

Working with JSON in GoMedium

Encoding

• Marshal traverses the value v recursively.

  • If an encountered value implements the Marshaler interface and is not a nil pointer, Marshal calls its MarshalJSON method to produce JSON.

  • If no MarshalJSON method is present but the value implements encoding.TextMarshaler instead, Marshal calls its MarshalText method and encodes the result as a JSON string.

  • Array and slice values encode as JSON arrays, except that []byte encodes as a base64-encoded string, and a nil slice encodes as the null JSON value.

• Data types like func, chan, complex64/128 can not be converted to JSON values. Trying to marshal an object which contains these data types will throw an json.UnsupportedTypeError error except when the field is unconditionally omitted using a tag/

• If a value is a pointer, then the value of the pointer is used for the marshaling. If a value is an interface, then the concrete value of the interface is used for the marshaling.

• Sometimes, we do not want to encode a field value as it is but to provide a custom value for marshaling. This can be achieved by implementing json.Marshaler or encoding.TextMarshaler interface.

• Nil slices in Go will be encoded to the null JSON value, whereas an empty (but not nil) slice will be encoded to an empty JSON array

• For example, when the encoding/json package marshals a map into JSON, it reorders the data alphabetically by keys, regardless of the insertion order.

In these benchmarks we can see that json.MarshalIndent() takes 65% longer to run and uses around 30% more memory than json.Marshal(), as well as making two more heap allocations. Those figures will change depending on what you’re encoding, but in my experience they’re fairly indicative of the performance impact.

Behind the scenes json.MarshalIndent() works by calling json.Marshal() as normal, then running the JSON through the standalone json.Indent() function to add the whitespace. There’s also a reverse function available, json.Compact(), which you can use to remove whitespace from JSON.

Decoding

• We can use json.Valid function to check if JSON is valid. This function returns true if JSON data is valid or false otherwise.

• If a field in JSON does not contain the value of the data type declared in the structure, Unmarshal will not coerce that value to an appropriate data type of the field and instead, it will return an error.

• Когда мы декодируем JSON в структуру, то это похоже на merge двух объектов в JS. Если в структуре уже есть свои значения полей, то они и останутся. Если такое же поле есть в JSON то оно перезапишет значение в структуре.

Полный список правил:

• The fields that are unexported in the struct or missing from the JSON are not unmarshalled.

• If a field value in the JSON is null and its corresponding field type’s zero-value is nil (like interface, map, pointer, or slice) then the value is replaced with nil, else that field is ignored for unmarshalling and retains its original value.

• If Unmarshal encounters an array type and array values in the JSON are more than the array can hold, then extra values are discarded. If array values in JSON are less than the length of the array, then the remaining array elements are set to their zero-values. The array type should be compatible with the values in the JSON.

• If Unmarshal encounters a slice type, then the slice in the struct is set to 0 length and elements from the JSON array are appended one at a time. If the JSON contains an empty array, then Unmarshal replaces the slice in the struct with an empty slice. The slice type should be compatible with the values in the JSON.

• If Unmarshal encounters a map type and the map’s value in the struct is nil, then a new map is created and object values in the JSON are appended. If the map value is non-nil, then the original value of the map is reused and new entries are appended. The map type should be compatible with the values in the JSON.

• If Unmarshal encounters a pointer field and the value of that field in the JSON is null then that field is set to nil pointer value. If the field in the JSON is not null then new memory is allocated for the pointer in case the pointer is nil or the old value of the pointer is reused.

При декодировании JSON-а в map есть свой список правил преобразований:

1. A JSON string value is stored as string.

2. A JSON number value(int or float) is stored as float64.

3. A JSON boolean value is stored as bool.

4. A JSON null value is stored as nil value.

5. A JSON array value is stored as a slice of type []interface{}.

6. A JSON object value is stored as a map of type map[string]interface{}.

As we know, a valid JSON format can be an object (like in the example above) or an array. Since Unmarshal is capable of allocating memory for a pointer as well as it can create containers to hold decoded JSON data on its own, we can store complex JSON data without defining a container-type.

Using json.Unmarshal() requires about 80% more memory than json.Decoder, as well as being a tiny bit slower.

The Decode() method could potentially return the following five types of error:

Error-resistant JSON reading:

func readJSON(w http.ResponseWriter, r *http.Request, dst interface{}) error { // Decode the request body into the target destination.
	err := json.NewDecoder(r.Body).Decode(dst)
	if err != nil {
		// If there is an error during decoding, start the triage...
		var syntaxError *json.SyntaxError
		var unmarshalTypeError *json.UnmarshalTypeError
		var invalidUnmarshalError *json.InvalidUnmarshalError
		switch {
		// Use the errors.As() function to check whether the error has the type
		// *json.SyntaxError. If it does, then return a plain-english error message // which includes the location of the problem.
		case errors.As(err, &syntaxError):
			return fmt.Errorf("body contains badly-formed JSON (at character %d)", syntaxError.Offset)
		// In some circumstances Decode() may also return an io.ErrUnexpectedEOF error // for syntax errors in the JSON. So we check for this using errors.Is() and // return a generic error message. There is an open issue regarding this at
		// https://github.com/golang/go/issues/25956.
		case errors.Is(err, io.ErrUnexpectedEOF):
			return errors.New("body contains badly-formed JSON")
		// Likewise, catch any *json.UnmarshalTypeError errors. These occur when the
		// JSON value is the wrong type for the target destination. If the error relates // to a specific field, then we include that in our error message to make it
		// easier for the client to debug.
		case errors.As(err, &unmarshalTypeError):
			if unmarshalTypeError.Field != "" {
				return fmt.Errorf("body contains incorrect JSON type for field %q", unmarshalTypeError.Field)
			}
			return fmt.Errorf("body contains incorrect JSON type (at character %d)", unmarshalTypeError.Offset)
		// An io.EOF error will be returned by Decode() if the request body is empty. We // check for this with errors.Is() and return a plain-english error message
		// instead.
		case errors.Is(err, io.EOF):
			return errors.New("body must not be empty")
		// A json.InvalidUnmarshalError error will be returned if we pass a non-nil
		// pointer to Decode(). We catch this and panic, rather than returning an error // to our handler. At the end of this chapter we'll talk about panicking
		// versus returning errors, and discuss why it's an appropriate thing to do in // this specific situation.
		case errors.As(err, &invalidUnmarshalError):
			panic(err)
		// For anything else, return the error message as-is.
		default:
			return err
		}
	}
	return nil
}

Go’s json.Decoder provides a DisallowUnknownFields() setting that we can use to generate an error when during decoding there is unknown field.

Partial JSON decoding

If you have a lot of JSON input to process and only need a small part of it, it’s often possible to leverage the json.RawMessage type to help deal with this.

// Let's say that the only thing we're interested in is processing the "genres" array in // the following JSON object
js := `{"title": "Top Gun", "genres": ["action", "romance"], "year": 1986}`
// Decode the JSON object to a map[string]json.RawMessage type. The json.RawMessage // values in the map will retain their original, un-decoded, JSON values.
var m map[string]json.RawMessage
err := json.NewDecoder(strings.NewReader(js)).Decode(&m)
if err != nil {
	log.Fatal(err)
}
// We can then access the JSON "genres" value from the map and decode it as normal using // the json.Unmarshal() function.
var genres []string
err = json.Unmarshal(m["genres"], &genres)
if err != nil {
	log.Fatal(err)
}
fmt.Printf("genres: %v\n", genres)

Using Pointers

If you have an application that receives a few distinct message types, you might define "receiver" structure like

type IncomingMessage struct {
    Cmd *Command
    Msg *Message
}

and the sending party can populate the Cmd field and/or the Msg field of the top-level JSON object, depending on the type of message they want to communicate.

Unmarshal, when decoding the JSON into an IncomingMessage struct, will only allocate the data structures present in the JSON data. To know which messages to process, the programmer need simply test that either Cmd or Msg is not nil.

The string struct tag directive

A final, less-frequently-used, struct tag directive is string. You can use this on individual struct fields to force the data to be represented as a string in the JSON output.

Note that the string directive will only work on struct fields which have int*, uint*, float* or bool types. For any other type of struct field it will have no effect.