Reflectionless data binding for Go's net/http (not actively maintained)

binding is reflectionless data binding for Go

GoDoc

binding

Reflectionless data binding for Go's net/http

Features

  • HTTP request data binding
  • Data validation (custom and built-in)
  • Error handling

Benefits

  • Moves data binding, validation, and error handling out of your application's handler
  • Reads Content-Type to deserialize form, multipart form, and JSON data from requests
  • No middleware: just a function call
  • Usable in any setting where net/http is present (Negroni, gocraft/web, std lib, etc.)
  • No reflection

Usage example

package main

import (
	"fmt"
	"net/http"

	"github.com/mholt/binding"
)

// First define a type to hold the data
// (If the data comes from JSON, see: http://mholt.github.io/json-to-go)
type ContactForm struct {
	User struct {
		ID int
	}
	Email   string
	Message string
}

// Then provide a field mapping (pointer receiver is vital)
func (cf *ContactForm) FieldMap(req *http.Request) binding.FieldMap {
	return binding.FieldMap{
		&cf.User.ID: "user_id",
		&cf.Email:   "email",
		&cf.Message: binding.Field{
			Form:     "message",
			Required: true,
		},
	}
}

// Now your handlers can stay clean and simple
func handler(resp http.ResponseWriter, req *http.Request) {
	contactForm := new(ContactForm)
	if errs := binding.Bind(req, contactForm); errs != nil {
		http.Error(resp, errs.Error(), http.StatusBadRequest)
		return
	}
	fmt.Fprintf(resp, "From:    %d\n", contactForm.User.ID)
	fmt.Fprintf(resp, "Message: %s\n", contactForm.Message)
}

func main() {
	http.HandleFunc("/contact", handler)
	http.ListenAndServe(":3000", nil)
}

Multipart/form-data usage example

package main

import (
	"bytes"
	"fmt"
	"github.com/mholt/binding"
	"io"
	"log"
	"mime/multipart"
	"net/http"
)

// We expect a multipart/form-data upload with
// a file field named 'data'
type MultipartForm struct {
	Data *multipart.FileHeader `json:"data"`
}

func (f *MultipartForm) FieldMap(req *http.Request) binding.FieldMap {
	return binding.FieldMap{
		&f.Data: "data",
	}
}

// Handlers are still clean and simple
func handler(resp http.ResponseWriter, req *http.Request) {
	multipartForm := new(MultipartForm)
	if errs := binding.Bind(req, multipartForm); errs != nil {
		http.Error(resp, errs.Error(), http.StatusBadRequest)
		return
	}

	// To access the file data you need to Open the file
	// handler and read the bytes out.
	var fh io.ReadCloser
	var err error
	if fh, err = multipartForm.Data.Open(); err != nil {
		http.Error(resp,
			fmt.Sprint("Error opening Mime::Data %v", err),
			http.StatusInternalServerError)
		return
	}
	defer fh.Close()
	dataBytes := bytes.Buffer{}
	var size int64
	if size, err = dataBytes.ReadFrom(fh); err != nil {
		http.Error(resp,
			fmt.Sprint("Error reading Mime::Data %v", err),
			http.StatusInternalServerError)
		return
	}
	// Now you have the attachment in databytes.
	// Maximum size is default is 10MB.
	log.Printf("Read %v bytes with filename %s",
		size, multipartForm.Data.Filename)
}

func main() {
	http.HandleFunc("/upload", handler)
	http.ListenAndServe(":3000", nil)
}

You can test from CLI using the excellent httpie client

http -f POST localhost:3000/upload data@myupload

Custom data validation

You may optionally have your type implement the binding.Validator interface to perform your own data validation. The .Validate() method is called after the struct is populated.

func (cf ContactForm) Validate(req *http.Request) error {
	if cf.Message == "Go needs generics" {
		return binding.Errors{
			binding.NewError([]string{"message"}, "ComplaintError", "Go has generics. They're called interfaces.")
		}
	}
	return nil
}

Binding custom types

For types you've defined, you can bind form data to it by implementing the Binder interface. Here's a contrived example:

type MyBinder map[string]string

func (t MyBinder) Bind(fieldName string, strVals []string) error {
	t["formData"] = strVals[0]
	return nil
}

If you can't add a method to the type, you can still specify a Binder func in the field spec. Here's a contrived example that binds an integer (not necessary, but you get the idea):

func (t *MyType) FieldMap(req *http.Request) binding.FieldMap {
	return binding.FieldMap{
		"a-key": binding.Field{
			Form: "number",
			Binder: func(fieldName string, formVals []string) error {
				val, err := strconv.Atoi(formVals[0])
				if err != nil {
					return binding.Errors{binding.NewError([]string{fieldName}, binding.DeserializationError, err.Error())}
				}
				t.SomeNumber = val
				return nil
			},
		},
	}
}

The Errors type has a convenience method, Add, which you can use to append to the slice if you prefer.

Supported types (forms)

The following types are supported in form deserialization by default. (JSON requests are delegated to encoding/json.)

  • uint, *uint, []uint, uint8, *uint8, []uint8, uint16, *uint16, []uint16, uint32, *uint32, []uint32, uint64, *uint64, []uint64
  • int, *int, []int, int8, *int8, []int8, int16, *int16, []int16, int32, *int32, []int32, int64, *int64, []int64
  • float32, *float32, []float32, float64, *float64, []float64
  • bool, *bool, []bool
  • string, *string, []string
  • time.Time, *time.Time, []time.Time
  • *multipart.FileHeader, []*multipart.FileHeader
Owner
Matt Holt
M.S. Computer Science. Author of the Caddy Web Server, CertMagic, Papa Parse, JSON/curl-to-Go, Timeliner, Relica, and more...
Matt Holt
Comments
  • Test json binding

    Test json binding

    Implement a simple test to verify that binding of JSON requests are handled by the Json function. To accomplish this, I had to rename the existing Json method to DefaultJsonBinder and introduce a function variable, Json, whose default value is DefaultJsonBinder.

    I'm happy to do this another way, or for the PR to not be merged at all, but since I did this work while getting started on the #9, I figured I'd submit it. Since it's ultimately completely orthogonal to #9, I didn't want to mix the two concerns.

  • json arrays are completely broken

    json arrays are completely broken

    The merge of #7 broke the binding of JSON arrays to slices.

    trying to bind json arrays to slices results in a single element slice where the single element's value is the zero value for the type. (e.g. {"arr": [1,2,3]} gives a slice with one element whose value 0).

    I've created a simple to demonstrate the problem at https://github.com/bhcleek/binding/tree/json-array-fail. Note that the test passes on commit 495a5e4.

  • dynamic (runtime) FieldMap()

    dynamic (runtime) FieldMap()

    use case: I'm using a 3rd party service that posts a resource to my application using application/x-www-form-urlencoded with camelcase names and as application/json with snakecase names depending (form encoded when the resource is created on their system and json when the resource is modified). I'm not advocating this is a smart thing to do, but it exists in the wild none-the-less.

    Currently, I used struct composition to use the same struct, but I have a very duplicated FieldMap. I noticed Validate takes *http.Request as a param, but FieldMap does not. If FieldMap also passed the http.Request that would solve my issue, but break the api.

    Another idea would be to add a BindFn method so passing any func()binding.FieldMap will satisfy and then the caller would be responsible for also calling validate.

    Thoughts?

  • binding.Field.Required=true is ignored on JSON input

    binding.Field.Required=true is ignored on JSON input

    binding.Field.Required is ignored if the corresponding type implements the Binder interface. Bind is not run for absent input either.

    Example:

    type EmailAddress string
    
    func (e *EmailAddress) Bind(strVals []string, errs binding.Errors) binding.Errors {
        // [snip]
        *e = EmailAddress(strVals[0])
        return errs
    }
    
    type SignupForm struct {
        Email       EmailAddress
        InviteCode1 string
    }
    
    func (cf *SignupForm) FieldMap() binding.FieldMap {
        return binding.FieldMap{
            &cf.Email: binding.Field{
                Form:     "email",
                Required: true,
            },
            &cf.InviteCode1: binding.Field{
                Form:     "invite-code-1",
                Required: true,
            },
        }
    }
    
    // ----
    func signupHandler(resp http.ResponseWriter, req *http.Request) {
        myForm := new(SignupForm)
        errs := binding.Bind(req, myForm)
        if errs.Handle(resp) {
            return
        }
    }
    

    Test:

    curl --data "message=Yadda" http://127.0.0.1/signup
    

    Expected:

    [{"fieldNames":["email"],"classification":"RequiredError","message":"Required"},
    {"fieldNames":["invite-code-1"],"classification":"RequiredError","message":"Required"}]
    

    Actual result:

    [{"fieldNames":["invite-code-1"],"classification":"RequiredError","message":"Required"}]
    
  • 400 instead of 422

    400 instead of 422

    I would prefer to use 400 Bad Request instead of 422 Unprocessable Entity when a validation of a type fails.

    The 422 is an obscure code and 400 is generally used when you supplied the parameters incorrectly which is the case of &number=notanumber.

  • Return error interface instead of Errors

    Return error interface instead of Errors

    Would you be interested in a pull request to make the API more idiomatic. The following code highlights the issue with returning the Errors type and how it goes against the grain and can cause hard to find runtime bugs. Assume binding.Bind(c.Request, m) is succesful

    var err error
    err = binding.Bind(c.Request, m)
    log.Println(err == nil)  // false
    
    err := binding.Bind(c.Request, m)
    log.Println(err == nil)  // true
    

    Both example compile, however because the first example is using an interface and Bind does not explicitly return nil on Empty errors it is using the result of Errors.String

  • Add type interface{} target as argument to Field.Binder?

    Add type interface{} target as argument to Field.Binder?

    As incentive to delve into this topic I've drafted up a Gist to illustrate the current state of Binding's usage in the wild. I will introduce some ideas to Binding referring to that example: https://gist.github.com/wmark/73ee2abe30e13af93a18#file-mailgun-go-L62-L70

    In order to create a library (or to add it to Binding itself) we have to add another argument to Field.Binder, by which the function can access formVals[]' destination. (Please note the suggested functions in the comments.)

    Or, do you think using closures would suffice here and in other use-cases?

  • Customise error message

    Customise error message

    When you have a property that is marked as required, the error message is "Required", is there a way to add a custom message, maybe a property on the binding.Field struct?

    Happy to do a PR if that helps so it checks if the new ErrorMessage property is set, if not it uses the default message

  • implement idiomatic errors

    implement idiomatic errors

    Consider this a proposal to fix #22. I believe these changes represent a more idiomatic approach, but I welcome any feedback. Obviously, this breaks compatibility with previous releases. Once we reach an agreement on changes, I'll update README.md before we merge.

    Return error types from exported functions

    Change the Binder function type, Binder.Bind, and Validator.Validate to have an error return value and modify their parameters to no longer include the slice of errors.

    Flatten field errors. Each error is associated with at most one field.

    Stop exporting Error. Replace it with a new interface type, FieldError, and rename Error to fieldError.

  • Introduce a FieldMapFromStruct() to avoid repetitions?

    Introduce a FieldMapFromStruct() to avoid repetitions?

    What can we add to avoid repetitions such as this? https://gist.github.com/wmark/73ee2abe30e13af93a18#file-mailgun-go-L3-L57

    A FieldMapFromStruct() could use reflection to generate the FieldMap from tags(?). Because this has to be done once only, for example during the module's init() I believe it wouldn't count towards the claim Binding was »reflectionless«.

  • Example doesn't bind the User.ID

    Example doesn't bind the User.ID

    I am posting the following JSON through using Postman

    { "user_id" : 1, "email" : "[email protected]", "message" : "hello" }
    

    But it is returning 0 in the response

    From:    0
    Message: hello
    
  • Don't convert non-binding.Errors.

    Don't convert non-binding.Errors.

    As a follow-on to 52f4dd9, continue the process of allowing errors that aren't binding.Errors to flow through without adaptation.

    The need for this arises from cases where validation is performed against a struct that contains field of a struct type that also implements Validator, when the validation for the inner type returns an error that isn't a binding.Errors. The existing code consumes such errors and turns them into not-particularly-helpful binding.Errors instances, inhibiting upstack processing of distinct error types. This does away with that interference.

  • autobindings to automatically create FieldMap functions for your structs

    autobindings to automatically create FieldMap functions for your structs

    Hey,

    First of all great work. So when I started using binding, one of the overheads I felt was writing FieldMap function.

    So leveraging the go generate power, I have created a tool called autobindings that automatically generates the field map function.

    Right now, it checks for the JSON tag, if it's available it uses the same mapping, if not it uses the field name for mapping.

    I would love if you could refer this library in your documentation :). And let me know your feedback.

    Thanks

  • Ability to control the HTTP error codes

    Ability to control the HTTP error codes

    Similar to https://github.com/mholt/binding/issues/31 I would like to return a different error code if someone does a HTTP POST without any form data. At the moment it defaults to 415 Unsupported Media Type which doesn't make sense to me.

  • Re-thinking the binding package

    Re-thinking the binding package

    Quite a few important discussions and pull requests have happened here which I think deserve some attention (all are important, but these are the relevant ones):

    • #17 Add type interface{} target as argument to Field.Binder?
    • #18 Introduce a FieldMapFromStruct() to avoid repetitions? and #19 dynamic (runtime) FieldMap()
    • #20 Added Map to support mux.Vars in Gorilla
    • #22 Return error interface instead of Errors
    • #24 Expose *Binder interface for more byte oriented payloads

    To me, two main themes prevail:

    • Reduce duplication
    • Use interfaces to make the package generic/idiomatic

    To make this happen, I'm willing to redesign the whole package and even break the existing API (it's not 1.0 yet anyway) to accomplish more useful, long-living goals. The philosophy behind the package still stands, though: no reflection, easy to use, idiomatic Go.

    So what is the purpose of binding? Currently, it's to populate an instance of a type with values and validate the result. I think that's a good overall purpose. It also happens to provide form and multipart deserializers and data validation, along with interfaces to facilitate all that. Also very useful. But it also employs third-party serializers like encoding/json for JSON payloads. And it could support others.

    Issue #24 raises the question of limiting binding to net/http. Why not open it to the more general io.Reader? We could have a wrapper function to make net/http convenient, but at its core, relying on io.Reader seems more useful. While we're at it, we could clean up and compartmentalize the error handling/validation APIs some more.

    These are some things I'm kicking around in my head. Discussion is encouraged, and I invite users of the package to add your feedback.

Related tags
Trims, sanitizes & scrubs data based on struct tags (go, golang)

Conform- keep user input in check (go, golang) Trim, sanitize, and modify struct string fields in place, based on tags. Update Jan 12, 2016 -- Now als

Dec 20, 2022
Data validation, cleaning and error collection for golang

GoForms - form data validation, cleaning and error reporting The goforms library is a proof-of-concept for a data validation, cleaning and error colle

Jan 14, 2022
Run Tor onion services on Kubernetes (actively maintained)
Run Tor onion services on Kubernetes (actively maintained)

tor-controller This project started as an exercise to update kragniz's https://github.com/kragniz/tor-controller version Important!! This project is n

Dec 15, 2022
Gos: Armed Golang 💪 ( solutions for go module, goproxy, cross compilation, etc.)
Gos: Armed Golang 💪 ( solutions for go module, goproxy, cross compilation, etc.)

The current gos is still an alpha version, welcome more heroes to comment and improve it ?? , you can add more commands to it, or modify something to make it perform better.

Sep 15, 2022
automated "fork" of gVisor that only contains the netstack bits so the go.mod is smaller. maintained by scripts, not humans.

netstack This is a "fork" of https://github.com/google/gvisor, extracting out just the "netstack" networking bits, which previously were self-containe

Nov 24, 2022
A utility library to make use of the X Go Binding easier. (Implements EWMH and ICCCM specs, key binding support, etc.)

xgbutil is a utility library designed to work with the X Go Binding. This project's main goal is to make various X related tasks easier. For example,

Dec 10, 2022
A utility library to make use of the X Go Binding easier. (Implements EWMH and ICCCM specs, key binding support, etc.)

xgbutil is a utility library designed to work with the X Go Binding. This project's main goal is to make various X related tasks easier. For example,

Dec 10, 2022
mass-binding-target is a command line tool for generating binding target list by search plot files from disk.

mass-binding-target mass-binding-target is a command line tool for generating binding target list by search plot files from disk. Build Go 1.13 or new

Nov 5, 2021
This is a small Go program, which can tell you what processes are actively using a set of files

winuse This is a small Go program, which can tell you what processes are actively using a set of files. It primarily exists to show off how one can bi

Jan 28, 2022
Data injection wrapper for values maintained in a GCP Secret Manager document.

injector The injector is a utility that retrieves a GCP Secret Manager document which contains an HJSON or JSON object with an embedded top-level envi

Jan 19, 2022
Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

Dec 24, 2019
Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces

lossy Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces. Its main usage is to test robustness of ap

Oct 14, 2022
[NO LONGER MAINTAINED} oauth 2 server implementation in Go

hero hero is a feature rich oauth 2 server implementation in Go. Features User account management Client management oauth 2 rfc 6749 compliant Configu

Nov 18, 2022
A fork of the simple WireGuard VPN server GUI community maintained
A fork of the simple WireGuard VPN server GUI community maintained

Subspace - A simple WireGuard VPN server GUI Subspace - A simple WireGuard VPN server GUI Slack Screenshots Features Contributing Setup 1. Get a serve

Dec 25, 2022
Native ZooKeeper client for Go. This project is no longer maintained. Please use https://github.com/go-zookeeper/zk instead.

Native Go Zookeeper Client Library License 3-clause BSD. See LICENSE file. This Repository is No Longer Maintained Please use https://github.com/go-zo

Dec 19, 2022
An AMQP 0-9-1 Go client maintained by the RabbitMQ team. Originally by @streadway: `streadway/amqp`

Go RabbitMQ Client Library This is a Go AMQP 0.9.1 client maintained by the RabbitMQ core team. It was originally developed by Sean Treadway. Differen

Jan 1, 2023
Token-list - The community maintained Solana token registry

Please note: This repository is being rebuilt to accept the new volume of token

Feb 2, 2022
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http

fasthttp Fast HTTP implementation for Go. Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than

Jan 5, 2023
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http

fasthttp Fast HTTP implementation for Go. Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than

Jan 2, 2023
fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client.

fhttp The f stands for flex. fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client. T

Jan 1, 2023