Quick and easy expression matching for JSON schemas used in requests and responses

schema GoDoc GoReport

schema makes it easier to check if map/array structures match a certain schema. Great for testing JSON API's or validating the format of incoming requests and providing error messages to the api user. Also see trial for simple test assertions.

The initial version was built by gfrey and jgroenveld. It was inspired by chancancode/json_expressions

Coverage: https://gocover.io/github.com/jgroeneveld/schema

Example

example_test.go

func TestJSON(t *testing.T) {
    reader := getJSONResponse()

    err := schema.MatchJSON(
        schema.Map{
            "id":       schema.IsInteger,
            "name":     "Max Mustermann",
            "age":      42,
            "height":   schema.IsFloat,
            "footsize": schema.IsPresent,
            "address": schema.Map{
                "street": schema.IsString,
                "zip":    schema.IsString,
            },
            "tags": schema.ArrayIncluding("red"),
        },
        reader,
    )

    if err != nil {
        t.Fatal(err)
    }
}

JSON Input

{
    "id": 12,
    "name": "Hans Meier",
    "age": 42,
    "height": 1.91,
    "address": {
        "street": 12
    },
    "tags": ["blue", "green"]
}

err.Error() Output

"address": Missing keys: "zip"
"address.street": is no string but float64
"name": "Hans Meier" != "Max Mustermann"
"tags": red:string(0) not included
Missing keys: "footsize"

Entry Points

schema.Match(schema.Matcher, interface{}) error
schema.MatchJSON(schema.Matcher, io.Reader) error

Matchers

"ConcreteValue"
    Any concrete value like: "Name", 12, true, false, nil

IsPresent
    Is the value given (empty string counts as given).
    This is essentially a wildcard in map values or array elements.

Types
    - IsString
    - IsInt
    - IsFloat
    - IsBool
    - IsTime(format)

Map{"key":Matcher, ...}
    Matches maps where all given keys and values have to match. 
    No extra or missing keys allowed.

MapIncluding{"key":Matcher, ...}
    Matches maps but only checks the given keys and values and ignores extra ones.

Array(Matcher...)
    Matches all array elements in order.

ArrayUnordered(Matcher...)
    Matches all array elements but order is ignored.

ArrayIncluding(Matcher...)
    Reports elements that can not be matched.

ArrayEach(Matcher)
    Each element of the array has to match the given matcher.
    
Capture(name)
    Can be used once or more to capture values and to make sure a value stays the same 
    if it occurs multiple times in a schema. See [capture_test.go](capture_test.go).
    Can also be used to use the captured value in future operations (e.g. multiple requests with the same id).
    
StringEnum(...values)

How to write matchers

To use custom or more specialized matchers, the schema.Matcher interface needs to be implemented. Either via struct or by using schema.MatcherFunc

To report errors, schema.SelfError(message) needs to be used if the data itself is the problem.

schema.Error.Add(field, message) if a subelement of the data is the problem (see Map and Array).

var IsTime = schema.MatcherFunc("IsTime",
    func(data interface{}) *schema.Error {
        s, ok := data.(string)
        if !ok {
            return schema.SelfError("is no valid time: not a string")
        }

        _, err := time.Parse(time.RFC3339, s)
        if err != nil {
            return schema.SelfError("is no valid time: " + err.Error())
        }
        return nil
    },
)

To be more generic with regard to the time format the following pattern can be used:

func IsTime (format string) schema.Matcher
	return schema.MatcherFunc("IsTime",
		func(data interface{}) *schema.Error {
			s, ok := data.(string)
			if !ok {
				return schema.SelfError("is no valid time: not a string")
			}

			_, err := time.Parse(format, s)
			if err != nil {
				return schema.SelfError("is no valid time: " + err.Error())
			}
			return nil
		},
	)
}

Ideas

  • write Optional(Matcher) that matches if key is missing or given matcher is satisfied
  • write Combine(...Matcher) that matches if all given matchers are satisfied

Issues

Numbers

JSON does not differ between integers and floats, ie. there are only numbers. This is why the go JSON library will always return a float64 value if no type was specified (unmarshalling into an interface{} type). This requires some magic internally and can result in false positives. For very large or small integers errors could occur due to rounding errors. If something has no fractional value it is assumed to be equal to an integer (42.0 == 42).

Array Matchers

For arrays there are matcher variants for including and unordered. They take the following steps:

  • Order the given matchers, where concrete values are matched first, then all matchers except the most generic IsPresent, and finally all IsPresent matchers. This order guarantees that the most specific values are matched first.
  • For each of the ordered matchers, verify one of the remaining values matches.
  • Keep a log of all matched values.

This will work in most of the cases, but might fail for some weird nested structures where something like a backtracking approach would be required.

Owner
Jaap Groeneveld
Hi! I am a Multistack developer from Hamburg, Germany.
Jaap Groeneveld
Similar Resources

HTTP traffic mocking and testing made easy in Go ༼ʘ̚ل͜ʘ̚༽

gock Versatile HTTP mocking made easy in Go that works with any net/http based stdlib implementation. Heavily inspired by nock. There is also its Pyth

Jan 4, 2023

Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Dec 13, 2022

gostub is a library to make stubbing in unit tests easy

gostub gostub is a library to make stubbing in unit tests easy. Getting started Import the following package: github.com/prashantv/gostub Click here t

Dec 28, 2022

Database testing made easy in Go.

dbtest Database testing made easy in Go. Features Declarative Define the minimum test specification in a YAML-based DSL, then all tests can be generat

Oct 31, 2022

gomonkey is a library to make monkey patching in unit tests easy

gomonkey is a library to make monkey patching in unit tests easy, and the core idea of monkey patching comes from Bouke, you can read this blogpost for an explanation on how it works.

Jan 4, 2023

Go-interactions - Easy slash commands for Arikawa

go-interactions A library that aims to make dealing with discord's slash command

May 26, 2022

Library created for testing JSON against patterns.

Gomatch Library created for testing JSON against patterns. The goal was to be able to validate JSON focusing only on parts essential in given test cas

Oct 28, 2022

A Go test assertion library for verifying that two representations of JSON are semantically equal

A Go test assertion library for verifying that two representations of JSON are semantically equal

jsonassert is a Go test assertion library for verifying that two representations of JSON are semantically equal. Usage Create a new *jsonassert.Assert

Jan 4, 2023

ESME is a go library that allows you to mock a RESTful service by defining the configuration in json format

ESME is a go library that allows you to mock a RESTful service by defining the configuration in json format. This service can then simply be consumed by any client to get the expected response.

Mar 2, 2021
Trade Matching / Transaction System Load Testing Solution

Load Generation System for Trade Matching Systems Operation Users select one of the following options from thew Test Management Portal: Generate a new

Feb 25, 2022
Quick and dirty test to compare MySQL perf between ARM64 & Rosetta MySQL on M1Pro mac

Quick and dirty test to compare MySQL perf between ARM64 & Rosetta MySQL on M1Pro mac

Nov 5, 2021
Sample code for a quick demo of go 1.18's fuzzing

Fuzzing in Go 1.18 What is it? "Fuzzing is a type of automated testing which continuously manipulates inputs to a program to find bugs. Go fuzzing use

Feb 11, 2022
a benchmarking&stressing tool that can send raw HTTP requests

reqstress reqstress is a benchmarking&stressing tool that can send raw HTTP requests. It's written in Go and uses fasthttp library instead of Go's def

Dec 18, 2022
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Dec 12, 2022
:exclamation:Basic Assertion Library used along side native go testing, with building blocks for custom assertions

Package assert Package assert is a Basic Assertion library used along side native go testing Installation Use go get. go get github.com/go-playground/

Jan 6, 2023
A lightweight load balancer used to create big Selenium clusters
A lightweight load balancer used to create big Selenium clusters

Go Grid Router Go Grid Router (aka Ggr) is a lightweight active load balancer used to create scalable and highly-available Selenium clusters. Articles

Dec 28, 2022
A simple `fs.FS` implementation to be used inside tests.

testfs A simple fs.FS which is contained in a test (using testing.TB's TempDir()) and with a few helper methods. PS: This lib only works on Go 1.16+.

Mar 3, 2022
Vault mock - Mock of Hashicorp Vault used for unit testing

vault_mock Mock of Hashicorp Vault used for unit testing Notice This is a person

Jan 19, 2022
Terminal application used for API testing
Terminal application used for API testing

Easily create, manage and execute http requests from the terminal.

Dec 20, 2022