Package gorilla/schema fills a struct with form values.

schema

GoDoc CircleCI Sourcegraph

Package gorilla/schema converts structs to and from form values.

Example

Here's a quick example: we parse POST form values and then decode them into a struct:

// Set a Decoder instance as a package global, because it caches
// meta-data about structs, and an instance can be shared safely.
var decoder = schema.NewDecoder()

type Person struct {
    Name  string
    Phone string
}

func MyHandler(w http.ResponseWriter, r *http.Request) {
    err := r.ParseForm()
    if err != nil {
        // Handle error
    }

    var person Person

    // r.PostForm is a map of our POST form values
    err = decoder.Decode(&person, r.PostForm)
    if err != nil {
        // Handle error
    }

    // Do something with person.Name or person.Phone
}

Conversely, contents of a struct can be encoded into form values. Here's a variant of the previous example using the Encoder:

var encoder = schema.NewEncoder()

func MyHttpRequest() {
    person := Person{"Jane Doe", "555-5555"}
    form := url.Values{}

    err := encoder.Encode(person, form)

    if err != nil {
        // Handle error
    }

    // Use form values, for example, with an http client
    client := new(http.Client)
    res, err := client.PostForm("http://my-api.test", form)
}

To define custom names for fields, use a struct tag "schema". To not populate certain fields, use a dash for the name and it will be ignored:

type Person struct {
    Name  string `schema:"name,required"`  // custom name, must be supplied
    Phone string `schema:"phone"`          // custom name
    Admin bool   `schema:"-"`              // this field is never set
}

The supported field types in the struct are:

  • bool
  • float variants (float32, float64)
  • int variants (int, int8, int16, int32, int64)
  • string
  • uint variants (uint, uint8, uint16, uint32, uint64)
  • struct
  • a pointer to one of the above types
  • a slice or a pointer to a slice of one of the above types

Unsupported types are simply ignored, however custom types can be registered to be converted.

More examples are available on the Gorilla website: https://www.gorillatoolkit.org/pkg/schema

License

BSD licensed. See the LICENSE file for details.

Owner
Gorilla Web Toolkit
Gorilla is a web toolkit for the Go programming language that provides useful, composable packages for writing HTTP-based applications.
Gorilla Web Toolkit
Comments
  • Add Encoder

    Add Encoder

    Getting the ball rolling on this.

    As for custom converters, I have coded up a separate branch that uses the existing cache to achieve this. It has the added benefit of unifying the way conversion functions are selected between the decoder and encoder. However, it changes the signature of Converter. Calling for opinions.

    Resolves #44

  • Deliberately set empty field

    Deliberately set empty field

    I need a way to explicitly set a field value to empty and not have it be ignored by gorilla/schema when processed. It is important to not simply wipe out all missing fields, but only make empty the fields that were provided and somehow marked as intentionally empty.

    Use case: struct that is pre-populated from a database entry that is then passed into schema.Decoder to apply matched fields from a web form.

    Possibly if the fields exist in the incoming web form and are empty, they can be wiped out while leaving alone the fields in the struct that weren't referenced in the web form. Such a thing would need testing that I haven't myself done.

    I've created a very hacky patch[1] to allow deliberately unsetting a field value when empty, instead of just ignoring it. In an application I'm building, I pull in a record from the database and then use schema to merge changes. When the data coming in from a web form is empty, I intend for the value to be wiped out, not just ignored. But because of how structs work, it's really not easy to tell the difference between absent field versus a deliberately empty value. In this very hacky patch, if it hits a string value of "[[[ERASE]]]" it will set the field value to "". It's not good enough and is bound to have loads of gotchas. Maybe there an alternative way to do this that I've missed; if so, sorry for the wishlist request.

    [1] https://github.com/scjalliance/schema/commit/fdf73b8a723833e7bf577c94ae36f0e0c05bfc35 (don't actually use this commit)

  • ⚠️ The Gorilla Toolkit is Looking for a New Maintainer

    ⚠️ The Gorilla Toolkit is Looking for a New Maintainer

    The Gorilla Toolkit is looking for a new maintainer (or maintainers, plural). As the last standing maintainer of the project, I no longer have time to fully dedicate to maintaining the libraries here, and

    The major libraries - mux (https://github.com/gorilla/mux), schema (https://github.com/gorilla/schema), handlers (https://github.com/gorilla/handlers), and sessions (https://github.com/gorilla/sessions), are all reasonably mature libraries, but ongoing stewardship around bug triage, feature enhancements, and potential "version 2.0s" are all possibilities.

    • Have a demonstrated history of OSS contributions. This is important, as you need to be trustworthy: no maintainer is better than an adversarial maintainer!
    • Ideally, you actively contribute for 3-6 months, I merge after you review, and you gain the commit bit on the relevant repos after that period and/or active engagement on your part.
    • I transition you to admin of the project.

    Note: I don't expect this to be quick or easy - the websocket library, with 16k stars & 15k unique clones per week, has been looking for a new maintainer 3.5+ years, and has yet to have anyone reliably stick. If I don't have any luck finding new maintainer(s) in the next 6 months or so, it's likely I'll mark these projects as in maintenance mode only and archive the repos.

    Please keep the replies on-topic.

    Cross-posted from https://github.com/gorilla/mux/issues/659

  • Encode() will panic if struct contains field of struct pointer

    Encode() will panic if struct contains field of struct pointer

    Seems like this package doesn't allow struct to contain *struct field. Is there any plan to support this?

    Here the code that triggers panic:

    image

    Here is the panic message:

    --- FAIL: TestSlices (0.00s)
    exit status 2
    FAIL	example.com/x/vendor/github.com/gorilla/schema	0.010s
    panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    	panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation, code=0x1 addr=0x0 pc=0x111e298]
    
    goroutine 6 [running]:
    testing.tRunner.func1(0xc420074750)
    	/usr/local/Cellar/go/1.8.1/libexec/src/testing/testing.go:622 +0x29d
    panic(,0x1145d40, 0x1216840)
    	/usr/local/Cellar/go/1.8.1/libexec/src/runtime/panic.go:,489 +0x2cf
    example.com/x/vendor/github.com/gorilla/schema.typeEncoder.func1(0x1131640, 0xc42001ea70, 0x1b6, 0x5, 0x1131640)
    	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder.go:117 +0xb8
    example.com/x/vendor/github.com/gorilla/schema.(*Encoder).encode(0xc4200132b0, 0x1135740, 0xc42001ea00, 0x16, 0xc42000eb40, 0xc420013280, 0xc4200132c0)
    	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder.go:63 +0x433
    ,example.com/x/vendor/github.com/gorilla/schema.(*Encoder).Encode(0xc4200132b0, 0x1135740, 0xc42001ea00, 0xc42000eb40,, 0xc42000eb40, 0xc420022000)
    	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder.go:,29 +0x6b
    example.com/x/vendor/github.com/gorilla/schema.TestSlices(,0xc420074750)
    	/Users/kurt/gopath/src/example.com/x/vendor/github.com/gorilla/schema/encoder_test.go:161, +0x2ec
    testing.tRunner(0xc420074750,, 0x11764e0)
    	/usr/local/Cellar/go/1.8.1/libexec/src/testing/testing.go:657, +0x96
    created by testing.(*T).Run,
    	/usr/local/Cellar/go/1.8.1/libexec/src/testing/testing.go:697 +0x2ca
    Error: Tests failed.
    
  • Proposal: Add Type to ConversionError

    Proposal: Add Type to ConversionError

    I'd like to response graceful error messages to user for given type of invalid field. See new version of ConversionError below. I'd like to implement this. Refer to link for more information: https://github.com/ajg/form/issues/5

    e.g.

    type ConversionError struct {
        Key   string
        Type reflect.Type
        Index int    
        Err   error  
    }
    
  • Use last item from array when mapping to a non array/slice

    Use last item from array when mapping to a non array/slice

    When multiple values are posted from a form, it makes sense to use the last defined value, which would be from the last defined element.

    This allow overriding the value, which is specially useful when dealing with checkboxes, to have a default value submitted when the checkbox is not selected.

    <input name="foo" type="hidden" value="0">
    <input name="foo" type="checkbox" value="1">
    
  • Specifying multiple parameters for a string type silently coerces data

    Specifying multiple parameters for a string type silently coerces data

    For instance, if I GET /path?key=banana&key=pear, then use schema to parse the url into a type fruit struct { key string }, I'll get either 'banana' or 'pear' (undefined behavior).

    I would prefer to get a type error, as I was expecting a string but got an array of string.

    Thoughts?

  • Allow 'json' tag to be used as a field alias

    Allow 'json' tag to be used as a field alias

    Right now this package requires the 'schema' tag to be set on each structure. In my particular workflow it's possible for the web client to submit a structure via JSON or via the forms collection. Ideally schema would be able to use the 'schema' tag or the 'json' tag (preferring the first if they both exist) to decode an incoming request.

  • Map underscore-style form fields to CamelCase struct fields

    Map underscore-style form fields to CamelCase struct fields

    I am porting a large application (JSON API) from Perl to Go now. All protocol's fields are in "underscore" notation but in the server-side code I use structs with "CamelCase" notation. It is very annoying to write aliases for each field, that's why I've made a small patch whitch adds a new decoder's option named "UnderscoreToCamelCase". When checked, it automatically makes underscore-style aliases for CamelCase fields with empty tag. P.S. Sorry for my English, it is not my native language.

  • [bug] []string is not decoded correctly

    [bug] []string is not decoded correctly

    Describe the bug

    A clear and concise description of what the bug is.

    Decoding a parameter into a string slice does not split on commas as it's supposed to, instead it creates a slice of one string, which is the input.

    Versions Go version: go version go1.15 windows/amd64 package version: 1.2.0 & versions before

    Steps to Reproduce

    See code below

    Expected behavior

    The value test should decode into an array of length 4.

    Code Snippets

    package main
    
    import (
    	"encoding/json"
    	"fmt"
    
    	"github.com/gorilla/schema"
    )
    
    func main() {
    
    	src := map[string][]string{
    		"test": {"this,is,a,test"},
    	}
    
    	var out struct {
    		Test []string `schema:"test"`
    	}
    
    	if err := schema.NewDecoder().Decode(&out, src); err != nil {
    		panic(err)
    	}
    
    	j, _ := json.Marshal(out)
    	fmt.Println(string(j))
    	fmt.Println(len(out.Test))
    }
    

    This yields

    {"Test":["this,is,a,test"]}
    1
    

    I have marshalled to JSON as go omits quotes from it's prints which masks the problem

    []bool, []int and other non-strings work fine

    A small bit of debugging shows it's getting into this condition https://github.com/gorilla/schema/blob/master/decoder.go#L280 and not the last else block which it's supposed to for this particular case.

  • [Reopen] Support custom converter from struct field names to form fields (callback)

    [Reopen] Support custom converter from struct field names to form fields (callback)

    Fixes #54, #55

    I reopen the pull request #55 that adds custom field name mapper. I changed the followings, but others are same to the original pull request.

    1. Merge master branch and resolve conflicts.
    2. Rename type and function names according to the following comments.
      • https://github.com/gorilla/schema/pull/55#discussion_r53386801
      • https://github.com/gorilla/schema/pull/55#discussion_r53386803
  • [question] Why can't Encoder check and call fmt.Stringer.String() on element automatically?

    [question] Why can't Encoder check and call fmt.Stringer.String() on element automatically?

    Why can't Encoder directly determine whether an element implements an interface (such as fmt.Stringer or encoding.TextMarshaler) and actively call its interface to implement Encode()?

    After all, Decode already does that (using encoding.TextUnmrshaler interface).

  • [bug] schema: converter not found for Page, when the struct name is same with field name.

    [bug] schema: converter not found for Page, when the struct name is same with field name.

    Steps to Reproduce

    playground

    //go:build ignore
    
    package main
    
    import (
    	"log"
    
    	"github.com/gorilla/schema"
    )
    
    type Page struct {
    	Page     int `schema:"page"`
    	PageSize int `schema:"pageSize"`
    }
    
    type PageAlias struct {
    	Page     int `schema:"page"`
    	PageSize int `schema:"pageSize"`
    }
    
    type Outer struct {
    	Page
    }
    
    type OuterAlias struct {
    	PageAlias
    }
    
    func main() {
    	decoder := schema.NewDecoder()
    
    	{
    		pp := Outer{}
    		if err := decoder.Decode(&pp, map[string][]string{
    			"page": {"1"},
    		}); err != nil {
    			log.Printf("decode failed with Outer: %v\n", err)
    		} else {
    			log.Printf("decode result with Outer: %+v\n", pp)
    		}
    	}
    
    	{
    		pp := OuterAlias{}
    		if err := decoder.Decode(&pp, map[string][]string{
    			"page": {"1"},
    		}); err != nil {
    			log.Printf("decode failed with OuterAlias: %v\n", err)
    		} else {
    			log.Printf("decode result with OuterAlias: %+v\n", pp)
    		}
    	}
    }
    

    Expected behavior

    What output or behaviour were you expecting instead?

    Parse form to Outer struct successfully.

  • feat: allow configuring `NewDecoder` via callbacks

    feat: allow configuring `NewDecoder` via callbacks

    Since GH-59 was rejected on the basis that it wasn't useful enough to break the public API. I took a stab at an alternate solution that doesn't break the public API. While still allowing one to configure the decoder when defining it as a package global (as recommended by the README).

    var decoder = NewDecoder(func(d *Decoder) {
    	d.IgnoreUnknownKeys(true)
    })
    

    Also partially solves GH-93.

  • fix misspell

    fix misspell

    ~Fixes #~

    Summary of Changes

    1. fixed misspell ommited -> omitted

    PS: Make sure your PR includes/updates tests! If you need help with this part, just ask!

  • [question] Error converting into []byte or []uint8

    [question] Error converting into []byte or []uint8

    Describe the problem you're having

    Schema fails to convert into []uint8 despite it being listed as supported. Anyone aware of why this might be the case? Or if Im misunderstanding the scenario.

    I have a server that accepts a multipart/form-data form data, of which one of the fields is a []byte, but when trying to decode into it I get the error:

    schema: error converting value for index 0 of "<value>"
    

    Versions

    Go version: go version go1.18.1 darwin/arm64

    Gorilla version: github.com/gorilla/schema v1.2.0

    "Show me the code!"

    Minimal test to show this failing:

    	req, err := http.NewRequest(http.MethodGet, "todo", nil)
    	req.ParseForm()
    	req.Form.Set("test1", "anyvalue")
    
    	type demo struct {
    		Test1 []byte // I've tried with pointer to byte slice and uint8 slice
    	}
    
    	var d demo
    	err = formDecoder.Decode(&d, req.Form)
    	fmt.Printf("%v", err)
    

    Go playground example: https://go.dev/play/p/e_62qC5zkik

  • Not checking for nil might lead to nil pointer dereference

    Not checking for nil might lead to nil pointer dereference

    https://github.com/gorilla/schema/blob/8285576f31afd6804df356a38883f4fa05014373/encoder.go#L167-L173

    The f should be checked for nil before it is used in closure.

Schema management CLI for MySQL
Schema management CLI for MySQL

Skeema is a tool for managing MySQL tables and schema changes in a declarative fashion using pure SQL. It provides a CLI tool allowing you to: Export

Dec 27, 2022
SQL schema migration tool for Go.

sql-migrate SQL Schema migration tool for Go. Based on gorp and goose. Using modl? Check out modl-migrate. Features Usable as a CLI tool or as a libra

Jan 2, 2023
Dbmate is a database migration tool, to keep your database schema in sync across multiple developers and your production servers.

Dbmate is a database migration tool, to keep your database schema in sync across multiple developers and your production servers. It is a stand

Jan 1, 2023
GitHub's Online Schema Migrations for MySQL
GitHub's Online Schema Migrations for MySQL

gh-ost GitHub's online schema migration for MySQL gh-ost is a triggerless online schema migration solution for MySQL. It is testable and provides paus

Apr 3, 2020
A Go package to help write migrations with go-pg/pg.

go-pg-migrations A Go package to help write migrations with go-pg/pg. Usage Installation Because go-pg now has Go modules support, go-pg-migrations al

Oct 16, 2022
golang csrf react example, using gorilla/mux and gorilla/mux

Demo REST backend Gorilla csrf middleware and Js frontend Use gorilla/mux and gorilla/csrf How to run open goland IDE, run middleware_test.go by click

Feb 2, 2022
Mantil-template-form-to-dynamodb - Receive form data and write it to a DynamoDB table
Mantil-template-form-to-dynamodb - Receive form data and write it to a DynamoDB table

This template is an example of serverless integration between Google Forms and DynamoDB

Jan 17, 2022
Package gorilla/securecookie encodes and decodes authenticated and optionally encrypted cookie values for Go web applications.

securecookie securecookie encodes and decodes authenticated and optionally encrypted cookie values. Secure cookies can't be forged, because their valu

Dec 26, 2022
Go generator to copy values from type to type and fields from struct to struct. Copier without reflection.

Copygen is a command-line code generator that generates type-to-type and field-to-field struct code without adding any reflection or dependenc

Dec 29, 2022
Dec 28, 2022
☄ The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct.
☄ The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct.

Gormat - Cross platform gopher tool The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct. 中文说明 Features Data

Dec 20, 2022
A Prometheus metrics exporter for AWS that fills in gaps CloudWatch doesn't cover

YAAE (Yet Another AWS Exporter) A Prometheus metrics exporter for AWS that fills in gaps CloudWatch doesn't cover About This exporter is meant to expo

Dec 10, 2022
A tool to compare if terraform provider migration schema snapshot is equal to schema defined in resource code

migration schema comparer for Terraform When develop Terraform provider sometimes we need do some state migration(not schema migration) via StateUpgra

Nov 18, 2021
Schema - JSON Schema rules plugin

This plugin allows to configure JSON Schema validations rules ensuring user-submitted records adhere to a pre-defined data schema.

Feb 16, 2022
:steam_locomotive: Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values. Dual Array and Full map support.

Package form Package form Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values. It has the following features: Supports map of

Dec 26, 2022
Package gorilla/sessions provides cookie and filesystem sessions and infrastructure for custom session backends.

sessions gorilla/sessions provides cookie and filesystem sessions and infrastructure for custom session backends. The key features are: Simple API: us

Dec 28, 2022
A Form Encoding & Decoding Package for Go

form A Form Encoding & Decoding Package for Go, written by Alvaro J. Genial. Synopsis This library is designed to allow seamless, high-fidelity encodi

Nov 23, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Dec 15, 2022
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

Pagser Pagser inspired by page parser。 Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and str

Dec 13, 2022
Match regex group into go struct using struct tags and automatic parsing

regroup Simple library to match regex expression named groups into go struct using struct tags and automatic parsing Installing go get github.com/oris

Nov 5, 2022