Handling shapechanging json in go.

digjson

Handling shapechanging json in go.

Purpose

There quite a few great libraries for Go when handling JSON, however, handling json that might change shape isn't something those packages handle well.

What do you mean by json that changes shape

Working with APIs that emit JSON is usually easy, however, in some cases, a single endpoint might return a dictionary one case, an array in another. This is very hard to deal with in a typed language like Go. digjson attempts to make working with these "changing shapes" a bit easier.

Goals

Turn a json path into a reasonable value that you can work with.

Non-Goals

Performance beyond the standard libraries json.decode (I'll make it as fast as I can with the feature set).

Is this actually a problem?

For years, I never saw anything like this, but in the course of the last 6 months I've run into two APIs (one public and one private) that have this behavior. There are some other packages that solve some of the issues, but I wanted something that fit my use case exactly.

I still don't understand, can you give me an example

Absolutely, the tests show some examples, but lets use something simple.

To start, the JSON looks like this:

jsondata = { "user": { "username: "user1" } }
var u string
was_found, err := Dig(jsondata, "user.username", &u)

The value "user1" is now in u.

Dig also works with structs:

jsondata = { "user": { "username: "user1" } }
type User struct {
    Username string `json:"username"`
}

var u User
was_found, err := Dig(jsondata, "user", &u)

So far, this isn't doing much for you compared to the normal JsonDecoder other than some nice json path access, lets look at three examples I've run into in the real world.

jsondata_with_obj = { "user": { "username": "user1" } }
jsondata_with_nil = { "user": null }
jsondata_with_array = { "user": [{ "username": "user1"}, {"username": "user2"} ] }

This is very hard to deal with, because during the decode, you don't know what the type will be, You can choose to decode to map[string]interface{} but this is very cumbersome with deep Json Documents.

Dig assumes you want the datatype you're giving it, if it's a list, and there is only a object, it will wrap it in a list (one item), if there is a null, you'll get back a empty list. It's easier if i just show you.

type User struct {
    Username string `json:"username"`
}

var users []User
was_found, err := Dig(jsondata_with_obj, "user", &users)

// users == [ User{ Username: "user1" } ]

was_found, err := Dig(jsondata_with_nil, "user", &users)

// users == []

was_found, err := Dig(jsondata_with_array, "user", &users)


// users == [ User{ Username: "user1" }, User{ Username: "user2"} ]

This seems super terrible

Yes

Why would anyone ever build an API like this?

I believe the instances where I'm experiencing this are related to a platform that returns XML and someone has written a XML to JSON converter. In these cases, it's ... understandable that things work the way they do, but it's still very painful to consume the data.

Can I change the separator to '.' if it doesn't work for me?

Not yet, but that is a good idea. (PR welcome)

Does it implicitly convert "123" (string) to 123 (int) if I give it a struct field with the type int.

Yes, there is also DigStrict that will not do that and return an error. I recommend checking out the tests for a clearer description here.

Are there tests?

Some, but I'm sure we could write more (PR Welcome).

Is it obvious how this works?

GoLang has a reflect module that is very powerful, but it is pretty weird to use if you're not familiar with how Go works, I encourage you to read the code!

Why are there so many questions in this README?

Good point :)

Owner
Similar Resources

Automatically generate Go (golang) struct definitions from example JSON

gojson gojson generates go struct definitions from json or yaml documents. Example $ curl -s https://api.github.com/repos/chimeracoder/gojson | gojson

Jan 1, 2023

A JSON diff utility

JayDiff A JSON diff utility. Install Downloading the compiled binary Download the latest version of the binary: releases extract the archive and place

Dec 11, 2022

Fast and flexible JSON encoder for Go

Fast and flexible JSON encoder for Go

Jettison Jettison is a fast and flexible JSON encoder for the Go programming language, inspired by bet365/jingo, with a richer features set, aiming at

Dec 21, 2022

Create go type representation from json

json2go Package json2go provides utilities for creating go type representation from json inputs. Json2go can be used in various ways: CLI tool Web pag

Dec 26, 2022

Console JSON formatter with query feature

Console JSON formatter with query feature

Console JSON formatter with query feature. Install: $ go get github.com/miolini/jsonf Usage: Usage of jsonf: -c=true: colorize output -d=false: de

Dec 4, 2022

Fluent API to make it easier to create Json objects.

Jsongo Fluent API to make it easier to create Json objects. Install go get github.com/ricardolonga/jsongo Usage To create this: { "name":"Ricar

Nov 7, 2022

Arbitrary transformations of JSON in Golang

kazaam Description Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang. This functionality provides

Dec 18, 2022

Parsing JSON is a hassle in golang

GoJSON Parsing JSON is a hassle in golang. This package will allow you to parse and search elements in a json without structs. Install gojson go get g

Nov 12, 2021

A JSON stream parser for Go

pjson A JSON stream parser for Go Example The example below prints all string values from a JSON document. package

Oct 3, 2022
Related tags
Get JSON values quickly - JSON parser for Go
Get JSON values quickly - JSON parser for Go

get json values quickly GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line

Dec 28, 2022
JSON diff library for Go based on RFC6902 (JSON Patch)

jsondiff jsondiff is a Go package for computing the diff between two JSON documents as a series of RFC6902 (JSON Patch) operations, which is particula

Dec 4, 2022
Fast JSON encoder/decoder compatible with encoding/json for Go
Fast JSON encoder/decoder compatible with encoding/json for Go

Fast JSON encoder/decoder compatible with encoding/json for Go

Jan 6, 2023
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

Jun 26, 2022
Json-go - CLI to convert JSON to go and vice versa
Json-go - CLI to convert JSON to go and vice versa

Json To Go Struct CLI Install Go version 1.17 go install github.com/samit22/js

Jul 29, 2022
JSON Spanner - A Go package that provides a fast and simple way to filter or transform a json document

JSON SPANNER JSON Spanner is a Go package that provides a fast and simple way to

Sep 14, 2022
Abstract JSON for golang with JSONPath support

Abstract JSON Abstract JSON is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its struct

Jan 5, 2023
Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection

fastjson - fast JSON parser and validator for Go Features Fast. As usual, up to 15x faster than the standard encoding/json. See benchmarks. Parses arb

Jan 5, 2023
Small utility to create JSON objects
Small utility to create JSON objects

gjo Small utility to create JSON objects. This was inspired by jpmens/jo. Support OS Mac Linux Windows Requirements Go 1.1.14~ Git Installtion Build $

Dec 8, 2022
JSON query in Golang

gojq JSON query in Golang. Install go get -u github.com/elgs/gojq This library serves three purposes: makes parsing JSON configuration file much easie

Dec 28, 2022