Implementation for checking that required fields got filled in map -> struct decoding direction.

A basic test has been added as well.

type Required struct {
	RequiredBar string `mapstructure:"bar,required"`
	Value       string `mapstructure:"foo"`
}

Fixes: #7

Can a tag be created of this so that dependent packages can pull a specific version rather than having to use current sources? That would be much appreciated!

While this shows a 0.1, I made that version up. You can see in the ebuild it pulls the master.zip file rather than a tagged version.

https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-go/mapstructure/mapstructure-0.1.ebuild

Noticed that if encoding a struct into a map, time.Time entries are passed into empty maps, which doesn't seam very useful.

I could add a hook to encode it into a string instance, but because hooks run as pre-processors, it's not possible to preserve it as a time.Time.

Example code:

package main

import (
	"fmt"
	"time"

	"github.com/mitchellh/mapstructure"
)

func PayloadEncoder(target *map[string]interface{}) (*mapstructure.Decoder, error) {
	return mapstructure.NewDecoder(&mapstructure.DecoderConfig{
		//DecodeHook: mapstructure.ComposeDecodeHookFunc(
		// <- no hook can be added here to solve this.
		//),
		Result: target,
	})
}

type MyStruct struct {
	CreatedAt time.Time
}

func main() {
	p1 := MyStruct{
		CreatedAt: time.Now(),
	}
	m1 := make(map[string]interface{})
	d1, err := PayloadEncoder(&m1)
	fmt.Println("\nerr:", err, "\ndec:", d1, "\nmap:", m1)
	err2 := d1.Decode(p1)
	fmt.Printf("\nerr: %v\noutput: %#v\n", err2, m1)
}

Output:

err: <nil> 
dec: &{0xc0001be000} 
map: map[]

err: <nil>
output: map[string]interface {}{"CreatedAt":map[string]interface {}{}}

I have specific case that my struct have []int32 , but values for it goes from url via fields=xxx&fields=yyy so input map have map['fields']=[]string{'xxx','yyy'}

to decode it i need to get map with names and ints. How ca i do that via this package? P.S. Now i have handmade stuff that uses reflection and get values via proto.EnumValueMap via passing to it slice element type name

This feature adds the ability to decode structs to maps (essentially allowing the converse of the conventional use of this library). I've tried to be as complete in the test cases that I could add.

One thing this doesn't do (yet) is map structs recursively. I'm not sure if this library should even do that, the containers for those inner structs can't really be supplied elegantly. If this was to be done - and i'd be happy to do it if that's the case, all structs would be initialized and decoded to map[string]interface{}. What do you think @mitchellh?

Closes #53

It would be nice if the import can handle strings with a timestamp or a date time format as string. Something like that: "2015-09-30T01:18:56.096224525+02:00"

mapstructure is a Go library for decoding generic map values to structures and vice versa

At first glance, there is no way to encode a struct to a map with this library ..?

Use below code to reproduce:

package main

import (
	"fmt"
	"github.com/mitchellh/mapstructure"
)

type Config struct {
	ClientConfig  clientConfig   `yaml:"client,omitempty"`
	ClientConfigs []clientConfig `yaml:"clients,omitempty"`
}

type clientConfig struct {
	URL string `"yaml:url,omitempty"`
}

func main() {

	input := map[string]interface{}{
		"client": map[string]string{
			"url": "http://198.168.181.61:3100/api/prom/push",
		},
	}

	var result Config
	err := mapstructure.Decode(input, &result)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%#v", result)

}

output:

main.Config{ClientConfig:main.clientConfig{URL:""}, ClientConfigs:[]main.clientConfig(nil)}

I have added in a case to handle array type because I need to deal with uuid.UUID type

	var err error
	dataKind := getKind(val)
	fmt.Println(name, val, dataKind)
	switch dataKind {
	case reflect.Bool:
		err = d.decodeBool(name, data, val)
	case reflect.Interface:
		err = d.decodeBasic(name, data, val)
	case reflect.String:
		err = d.decodeString(name, data, val)
	case reflect.Int:
		err = d.decodeInt(name, data, val)
	case reflect.Uint:
		err = d.decodeUint(name, data, val)
	case reflect.Float32:
		err = d.decodeFloat(name, data, val)
	case reflect.Struct:
		err = d.decodeStruct(name, data, val)
	case reflect.Map:
		err = d.decodeMap(name, data, val)
	case reflect.Ptr:
		err = d.decodePtr(name, data, val)
	case reflect.Slice:
		err = d.decodeSlice(name, data, val)
	case reflect.Array: // required to handle uuid.UUID type
		err = d.decodeArray(name, data, val)
	default:
		// If we reached this point then we weren't able to decode it
		return fmt.Errorf("%s: unsupported type: %s", name, dataKind)
	}

Would you accept this new feature?

Similar to #186, mapstructure is not handling structs assigned to interfaces:

var cc interface{}
cc = struct{ Foo, Bar string }{}

decoderConfig := &mapstructure.DecoderConfig{Result: &cc, ErrorUnused: true}
decoder, err := mapstructure.NewDecoder(decoderConfig)
if err != nil {
	log.Fatal(err)
}

conf := map[string]string{
	"foo": "bar",
	"bar": "baz",
}

if err := decoder.Decode(conf); err != nil {
	log.Fatal(err)
}

Will give

* '' has invalid keys: bar, foo

Would this be something worthwile to address with a PR or are there constraints that would prevent handling this case?

Using *reflect.StructField as map key results in a data race. I honestly do not understand why yet and can only reproduce this in the consul codebase with high concurrency but this patch fixes it.

The same problem also exists in hashicorp/hcl.

https://github.com/hashicorp/hcl/pull/213

If I'm not mistaken, the main.go below should output:

sub_config:
    first_children:
        - name: parent
          grand_children:
            - name: foo
            - name: bar

Instead, it is outputting:

sub_config:
    first_children:
        - name: parent
          children:
            - name: foo
            - name: bar

Notice that the second slice is intended to be named grand_children but is really being named children as it is actually named in the struct.

If I step through the mapstructure.Decode function, I can see the tag get read and build an appropriate fieldName as sub_config.first_children[0].grand_children, but that at some point gets shuffled back to children.

We can see in this screenshot the result of the Decode prior to rendering the yaml:

package main

import (
	"fmt"
	"os"

	"github.com/mitchellh/mapstructure"
	"gopkg.in/yaml.v3"
)

type Config struct {
	Subconfig SubSection `mapstructure:"sub_config"`
}

type SubSection struct {
	Children []Child `mapstructure:"first_children"`
}

type Child struct {
	Name     string
	Children []Grandchild `mapstructure:"grand_children"`
}

type Grandchild struct {
	Name string
}

func main() {
	config := &Config{
		Subconfig: SubSection{
			Children: []Child{
				{Name: "parent", Children: []Grandchild{
					{Name: "foo"},
					{Name: "bar"},
				}},
			},
		},
	}

	encodedData := map[string]interface{}{}
	err := mapstructure.Decode(config, &encodedData)
	if err != nil {
		err = fmt.Errorf("unable to encode configuration: %w", err)
		fmt.Println(err)
		os.Exit(1)
	}

	data, err := yaml.Marshal(encodedData)
	if err != nil {
		err = fmt.Errorf("unable to marshal to yaml: %w", err)
		fmt.Println(err)
		os.Exit(1)
	}

	fmt.Println(string(data))
}

module foo

go 1.19

require (
	github.com/mitchellh/mapstructure v1.5.0 // indirect
	gopkg.in/yaml.v3 v3.0.1 // indirect
)

Struct format

type (
	Specification struct {
		Environment
		Templates
	}

	Environment struct {
		Build   image.Image
		Runtime image.Image
	}

	Templates struct {
		ArtifactPathTemplate          string                      `mapstructure:"artifact-path"`
		StartCommandTemplate          *executable.TemplateCommand `mapstructure:"start-command"`
		ComponentInstallationTemplate *executable.TemplateCommand `mapstructure:"component-install"`
		ExecutableBinary              string                      `mapstructure:"executable-binary"`
	}
)

getting error error decoding 'Specification': unsupported entry[type= []map[string]interface {}, value= ] Anyone can help how to decode these value

Technically this package still supports Go 1.14 which is why I decided to add a guard for the new functions. I can easily get rid of it, but I'd recommend bumping the Go version in the go.mod file in that case.

Structs in slices were simply copyed to the target map. With this PR they can also be mapped, using the "deep" tag, or the global "Deep" option.

Partially solves #249

When decoding a struct containing a string pointer and a struct pointer into a map, the resulting map has a pointer to the string but not a pointer to the struct.

func TestDecode_structToMap_PointersConsistency(t *testing.T) {
	type SourceChild struct {
		String string `mapstructure:"string"`
	}

	type SourceParent struct {
		ChildPtr  *SourceChild `mapstructure:"child-ptr"`
		StringPtr *string      `mapstructure:"string-ptr"`
	}

	var target map[string]interface{}

	var value = "goodbye"

	source := SourceParent{
		ChildPtr: &SourceChild{
			String: "hello",
		},
		StringPtr: &value,
	}

	if err := Decode(source, &target); err != nil {
		t.Fatalf("got error: %s", err)
	}

	expected := map[string]interface{}{
		"child-ptr": map[string]interface{}{
			"string": "hello",
		},
		"string-ptr": "goodbye",
	}

	expectedPointers := map[string]interface{}{
		"child-ptr": &map[string]interface{}{
			"string": "hello",
		},
		"string-ptr": stringPtr("goodbye"),
	}

	if !reflect.DeepEqual(target, expected) && !reflect.DeepEqual(target, expectedPointers) {
		t.Fatalf("bad: \nexpected        : %#v\nexpectedPointers: %#v\nresult          : %#v", expected, expectedPointers, target)
	}
}

Produces:

=== RUN   TestDecode_structToMap_PointersConsistency
    mapstructure_test.go:2941: bad: 
        expected        : map[string]interface {}{"child-ptr":map[string]interface {}{"string":"hello"}, "string-ptr":"goodbye"}
        expectedPointers: map[string]interface {}{"child-ptr":(*map[string]interface {})(0xc0000a8058), "string-ptr":(*string)(0xc00008eec0)}
        result          : map[string]interface {}{"child-ptr":map[string]interface {}{"string":"hello"}, "string-ptr":(*string)(0xc00008ede0)}
--- FAIL: TestDecode_structToMap_PointersConsistency (0.00s)

mapstructure version https://github.com/mitchellh/mapstructure/commit/bf980b35cac4dfd34e05254ee5aba086504c3f96 go version go1.18.7 linux/amd64

I can't see a reason why structures and base types should behave differently. I would expect to either keep the pointers or remove them. Can you please explain to me if this is a design choice or not? My guess is that struct pointers where not supported before PR #271 which made it dereference struct pointers systematically.

Discussion

Should all pointers be represented in the target map or none?

From the perspective of my use case, the pointer representation in the map is not necessary.

Here are the benefits of the pointers usage in this context:

• Data representation : allows the representation of "no information". E.g. a nul pointer is different from an empty string. -> nul pointer can be represented as an unset key
• Memory and speed optimization : redundent data structures can be referenced and large data structure can be referenced so that they are not copied on return -> this can be usefull when the decoded structure contains big data that should not be copied

For these reasons, I would suggest to:

• Handle pointers of any type in the same way (e.g. struct pointers and string pointers)
• Create a global option tho choose whether to keep or not the pointer representation in the target map (in the other direction, it's the type of the struct data that defines if it's a pointer or not). Defaults to "remove pointers". Name suggestion : "PreserveReferences"(true/false)
• For retro-compatibility, add an option to keep the pointer representation on non-struct types. Eventually set this by default until next major. Name suggestion : "LegacyReferencesHandling" (true/false)
• Add a property tag option to keep pointer representation and not copy data but only the pointer value (used for optimiazation in case of a large data structure, for example a string, that should be referenced in the map instead of copied). Name suggestion: "nocopy"

As an alternative, the options could be merge in a "ReferencesHandling" option with values legacy/preserve/discard.

Please let me know what you think and if there are any thechnical constraints that would not allow this. I would be happy to contribute.