Marshmallow provides a flexible and performant JSON unmarshalling in Go. It specializes in dealing with unstructured struct - when some fields are known and some aren't, with zero performance overhead nor extra coding needed.

Marshmallow

Marshmallow Campfire

CodeQL Status Run Tests Dependency Review Go Report Card Manual Code Coverage Go Reference Licence Latest Release

marshmallow-gopher

Marshmallow package provides a simple API to perform flexible and performant JSON unmarshalling in Go.

Marshmallow specializes in dealing with unstructured struct - when some fields are known and some aren't, with zero performance overhead nor extra coding needed. While unmarshalling, marshmallow allows fully retaining the original data and access it via a typed struct and a dynamic map.

Contents

Install

go get -u github.com/perimeterx/marshmallow

Usage

package main

import (
	"fmt"
	"github.com/perimeterx/marshmallow"
)

func main() {
	marshmallow.EnableCache() // this is used to boost performance, read more below
	v := struct {
		Foo string `json:"foo"`
		Boo []int  `json:"boo"`
	}{}
	result, err := marshmallow.Unmarshal([]byte(`{"foo":"bar","boo":[1,2,3],"goo":12.6}`), &v)
	fmt.Printf("v=%+v, result=%+v, err=%v", v, result, err)
	// Output: v={Foo:bar Boo:[1 2 3]}, result=map[boo:[1 2 3] foo:bar goo:12.6], err=<nil>
}

Performance Benchmark And Alternatives

Marshmallow performs best when dealing with mixed data - when some fields are known and some are unknown. More info below. Other solutions are available for this kind of use case, each solution is explained and documented in the link below. The full benchmark test can be found here.

Benchmark Iterations Time/Iteration Bytes Allocated Allocations
unmarshall twice 228693 5164 ns/op 1640 B/op 51 allocs/op
raw map 232236 5116 ns/op 2296 B/op 53 allocs/op
go codec 388442 3077 ns/op 2512 B/op 37 allocs/op
marshmallow 626168 1853 ns/op 608 B/op 18 allocs/op
marshmallow without populating struct 678616 1751 ns/op 608 B/op 18 allocs/op

marshmallow performance comparison

Marshmallow provides the best performance (up to X3 faster) while not requiring any extra coding. In fact, marshmallow performs as fast as normal json.Unmarshal call, however, such a call causes loss of data for all the fields that did not match the given struct. With marshmallow you never lose any data.

Benchmark Iterations Time/Iteration Bytes Allocated Allocations
marshmallow 626168 1853 ns/op 608 B/op 18 allocs/op
native library 652106 1845 ns/op 304 B/op 11 allocs/op
marshmallow without populating struct 678616 1751 ns/op 608 B/op 18 allocs/op

When Should I Use Marshmallow

Marshmallow is best suited for use cases where you are interested in all the input data, but you have predetermined information only about a subset of it. For instance, if you plan to reference two specific fields from the data, then iterate all the data and apply some generic logic. How does it look with the native library:

func isAllowedToDrive(data []byte) (bool, error) {
	result := make(map[string]interface{})
	err := json.Unmarshal(data, &result)
	if err != nil {
		return false, err
	}

	age, ok := result["age"]
	if !ok {
		return false, nil
	}
	a, ok := age.(float64)
	if !ok {
		return false, nil
	}
	if a < 17 {
		return false, nil
	}

	hasDriversLicense, ok := result["has_drivers_license"]
	if !ok {
		return false, nil
	}
	h, ok := hasDriversLicense.(bool)
	if !ok {
		return false, nil
	}
	if !h {
		return false, nil
	}

	for key := range result {
		if strings.Contains(key, "prior_conviction") {
			return false, nil
		}
	}

	return true, nil
}

And with marshmallow:

func isAllowedToDrive(data []byte) (bool, error) {
	v := struct {
		Age               int  `json:"age"`
		HasDriversLicense bool `json:"has_drivers_license"`
	}{}
	result, err := marshmallow.Unmarshal(data, &v)
	if err != nil {
		return false, err
	}

	if v.Age < 17 || !v.HasDriversLicense {
		return false, nil
	}

	for key := range result {
		if strings.Contains(key, "prior_conviction") {
			return false, nil
		}
	}

	return true, nil
}

API

Marshmallow exposes two main API functions - Unmarshal and UnmarshalFromJSONMap. Each of them can operate in three possible modes, and allow setting skipPopulateStruct mode.

Marshmallow also supports caching of refection information using EnableCache and EnableCustomCache.

Marshmallow Logo

Similar Resources

Gron transforms JSON into discrete assignments to make it easier to grep

gron Make JSON greppable! gron transforms JSON into discrete assignments to make it easier to grep for what you want and see the absolute 'path' to it

Nov 11, 2021

Gron - Gron transforms JSON into discrete assignments to make it easier to grep

gron Make JSON greppable! gron transforms JSON into discrete assignments to make

Jan 6, 2022

A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Cron.go This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually design

May 4, 2022

Run Jobs on a schedule, supports fixed interval, timely, and cron-expression timers; Instrument your processes and expose metrics for each job.

A simple process manager that allows you to specify a Schedule that execute a Job based on a Timer. Schedule manage the state of this job allowing you to start/stop/restart in concurrent safe way. Schedule also instrument this Job and gather metrics and optionally expose them via uber-go/tally scope.

Mar 28, 2022

Lightweight, fast and dependency-free Cron expression parser (due checker) for Golang (tested on v1.13 and above)

adhocore/gronx gronx is Golang cron expression parser ported from adhocore/cron-expr. Zero dependency. Very fast because it bails early in case a segm

Jul 4, 2022

clockwork - Simple and intuitive job scheduling library in Go.

clockwork - Simple and intuitive job scheduling library in Go.

clockwork A simple and intuitive scheduling library in Go. Inspired by python's schedule and ruby's clockwork libraries. Example use package main imp

Mar 3, 2022

Easy and fluent Go cron scheduling

goCron: A Golang Job Scheduling Package. goCron is a Golang job scheduling package which lets you run Go functions periodically at pre-determined inte

Jun 26, 2022

A programmable, observable and distributed job orchestration system.

A programmable, observable and distributed job orchestration system.

📖 Overview Odin is a programmable, observable and distributed job orchestration system which allows for the scheduling, management and unattended bac

Jun 23, 2022

Efficient and reliable background processing for Go

CurlyQ CurlyQ provides a simple, easy-to-use interface for performing background processing in Go. It supports scheduled jobs, job deduplication, and

Mar 3, 2022
Comments
  • Unmarshaling doesn't work for composed types

    Unmarshaling doesn't work for composed types

    Hey, I was trying to use the lib to unmarshal a composed struct, and expected it to work like the json package and to fill the nested struct as well. It looks like the issue is in mapStructFields that doesn't handle composed structs so it skips the nested values.

    package main
    
    import (
    	"fmt"
    	"github.com/perimeterx/marshmallow"
    )
    
    type A struct {
    	Foo string `json:"foo"`
    	Boo []int  `json:"boo"`
    }
    
    type B struct {
    	A
    }
    
    func main() {
    	marshmallow.EnableCache() // this is used to boost performance, read more below
    	a := A{}
    	result, err := marshmallow.Unmarshal([]byte(`{"foo":"bar","boo":[1,2,3],"goo":12.6}`), &a)
    	fmt.Printf("a=%+v, result=%+v, err=%v", a, result, err)
    	// Output: a={Foo:bar Boo:[1 2 3]}, result=map[boo:[1 2 3] foo:bar goo:12.6], err=<nil>
    
    	b := B{}
    	result, err = marshmallow.Unmarshal([]byte(`{"foo":"bar","boo":[1,2,3],"goo":12.6}`), &b)
    	fmt.Printf("b=%+v, result=%+v, err=%v", b, result, err)
    	// Output: b={A:{Foo: Boo:[]}}, result=map[boo:[1 2 3] foo:bar goo:12.6], err=<nil>
    }
    
    
A persistent and flexible background jobs library for go.

Jobs Development Status Jobs is no longer being actively developed. I will still try my best to respond to issues and pull requests, but in general yo

Jun 7, 2022
RESTful-JSON-API - RESTful-JSON-API using Go

RESTful-JSON-API using Go This basic REST-API principle establishes a one-to-one

Feb 15, 2022
A zero-dependencies and lightweight go library for job scheduling

A zero-dependencies and lightweight go library for job scheduling.

Jan 3, 2022
Simple, zero-dependency scheduling library for Go

go-quartz Simple, zero-dependency scheduling library for Go. About Inspired by the Quartz Java scheduler. Library building blocks Job interface. Any t

Jun 30, 2022
Scheduler - Scheduler package is a zero-dependency scheduling library for Go

Scheduler Scheduler package is a zero-dependency scheduling library for Go Insta

Jan 14, 2022
high performance distributed task scheduling system, Support multi protocol scheduling tasks
 high performance distributed task scheduling system, Support multi protocol scheduling tasks

high performance distributed task scheduling system, Support multi protocol scheduling tasks

Jun 14, 2022
Job worker service that provides an API to run arbitrary Linux processes.
Job worker service that provides an API to run arbitrary Linux processes.

Job Scheduler Summary Prototype job worker service that provides an API to run arbitrary Linux processes. Overview Library The library (Worker) is a r

May 26, 2022
This package provides the way to get the previous timestamp or the next timestamp that satisfies the cron expression.

Cron expression parser Given a cron expression, you can get the previous timestamp or the next timestamp that satisfies the cron expression. I have us

May 3, 2022
Redisq is a queue-over-redis that provides simple way to works with queues stored in Redis.

Redisq is a queue-over-redis that provides simple way to works with queues stored in Redis.

Feb 10, 2022
Feb 14, 2022