A library for diffing golang structures

Diff PkgGoDev Go Report Card Build Status

A library for diffing golang structures and values.

Utilizing field tags and reflection, it is able to compare two structures of the same type and create a changelog of all modified values. The produced changelog can easily be serialized to json.

NOTE: All active development now takes place on the v2 branch.

Installation

For version 2:

go get github.com/r3labs/diff/v2

Changelog Format

When diffing two structures using Diff, a changelog will be produced. Any detected changes will populate the changelog array with a Change type:

type Change struct {
	Type string      // The type of change detected; can be one of create, update or delete
	Path []string    // The path of the detected change; will contain any field name or array index that was part of the traversal
	From interface{} // The original value that was present in the "from" structure
	To   interface{} // The new value that was detected as a change in the "to" structure
}

Given the example below, we are diffing two slices where the third element has been removed:

from := []int{1, 2, 3, 4}
to := []int{1, 2, 4}

changelog, _ := diff.Diff(from, to)

The resultant changelog should contain one change:

Change{
    Type: "delete",
    Path: ["2"],
    From: 3,
    To:   nil,
}

Supported Types

A diffable value can be/contain any of the following types:

Type Supported
struct
slice
string
int
bool
map
pointer
custom types

Please see the docs for more supported types, options and features.

Tags

In order for struct fields to be compared, they must be tagged with a given name. All tag values are prefixed with diff. i.e. diff:"items".

Tag Usage
- Excludes a value from being diffed
identifier If you need to compare arrays by a matching identifier and not based on order, you can specify the identifier tag. If an identifiable element is found in both the from and to structures, they will be directly compared. i.e. diff:"name, identifier"
immutable Will omit this struct field from diffing. When using diff.StructValues() these values will be added to the returned changelog. It's use case is for when we have nothing to compare a struct to and want to show all of its relevant values.
nocreate The default patch action is to allocate instances in the target strut, map or slice should they not exist. Adding this flag will tell patch to skip elements that it would otherwise need to allocate. This is separate from immutable, which is also honored while patching.
omitunequal Patching is a 'best effort' operation, and will by default attempt to update the 'correct' member of the target even if the underlying value has already changed to something other than the value in the change log 'from'. This tag will selectively ignore values that are not a 100% match.

Usage

Basic Example

Diffing a basic set of values can be accomplished using the diff functions. Any items that specify a "diff" tag using a name will be compared.

import "github.com/r3labs/diff/v2"

type Order struct {
    ID    string `diff:"id"`
    Items []int  `diff:"items"`
}

func main() {
    a := Order{
        ID: "1234",
        Items: []int{1, 2, 3, 4},
    }

    b := Order{
        ID: "1234",
        Items: []int{1, 2, 4},
    }

    changelog, err := diff.Diff(a, b)
    ...
}

In this example, the output generated in the changelog will indicate that the third element with a value of '3' was removed from items. When marshalling the changelog to json, the output will look like:

[
    {
        "type": "delete",
        "path": ["items", "2"],
        "from": 3,
        "to": null
    }
]

Options and Configuration

Options can be set on the differ at call time which effect how diff acts when building the change log.

import "github.com/r3labs/diff/v2"

type Order struct {
    ID    string `diff:"id"`
    Items []int  `diff:"items"`
}

func main() {
    a := Order{
        ID: "1234",
        Items: []int{1, 2, 3, 4},
    }

    b := Order{
        ID: "1234",
        Items: []int{1, 2, 4},
    }

    changelog, err := diff.Diff(a, b, diff.DisableStructValues(), diff.AllowTypeMismatch(true))
    ...
}

You can also create a new instance of a differ that allows options to be set.

import "github.com/r3labs/diff/v2"

type Order struct {
    ID    string `diff:"id"`
    Items []int  `diff:"items"`
}

func main() {
    a := Order{
        ID: "1234",
        Items: []int{1, 2, 3, 4},
    }

    b := Order{
        ID: "1234",
        Items: []int{1, 2, 4},
    }

    d, err := diff.NewDiffer(diff.SliceOrdering(true))
    if err != nil {
        panic(err)
    }

    changelog, err := d.Diff(a, b)
    ...
}

Supported options are:

SliceOrdering ensures that the ordering of items in a slice is taken into account

DiscardComplexOrigin is a directive to diff to omit additional origin information about structs. This alters the behavior of patch and can lead to some pitfalls and non-intuitive behavior if used. On the other hand, it can significantly reduce the memory footprint of large complex diffs.

AllowTypeMismatch is a global directive to either allow (true) or not to allow (false) patch apply the changes if 'from' is not equal. This is effectively a global version of the omitunequal tag.

Filter provides a callback that allows you to determine which fields the differ descends into

DisableStructValues disables populating a separate change for each item in a struct, where the struct is being compared to a nil Value.

TagName sets the tag name to use when getting field names and options.

Patch and merge support

Diff additionally supports merge and patch. Similar in concept to text patching / merging the Patch function, given a change log and a target instance will make a best effort to apply the changes in the change log to the variable pointed to. The intention is that the target pointer is of the same type however, that doesn't necessarily have to be true. For example, two slices of differing structs may be similar enough to apply changes to in a polymorphic way, and patch will certainly try.

The patch function doesn't actually fail, and even if there are errors, it may succeed sufficiently for the task at hand. To accommodate this patch keeps track of each change log option it attempts to apply and reports the details of what happened for further scrutiny.

import "github.com/r3labs/diff/v2"

type Order struct {
    ID    string `diff:"id"`
    Items []int  `diff:"items"`
}

func main() {
    a := Order{
        ID: "1234",
        Items: []int{1, 2, 3, 4},
    }

    b := Order{
        ID: "1234",
        Items: []int{1, 2, 4},
    }

    c := Order{}
    changelog, err := diff.Diff(a, b)

    patchlog := diff.Patch(changelog, &c)
    //Note the lack of an error. Patch is best effort and uses flags to indicate actions taken
    //and keeps any errors encountered along the way for review
    fmt.Printf("Encountered %d errors while patching", patchlog.ErrorCount())
    ...
}

Instances of differ with options set can also be used when patching.

package main

import "github.com/r3labs/diff/v2"

type Order struct {
	ID    string `json:"id"`
	Items []int  `json:"items"`
}

func main() {
    a := Order{
        ID:    "1234",
        Items: []int{1, 2, 3, 4},
        }

    b := Order{
        ID:    "1234",
        Items: []int{1, 2, 4},
    }

    d, _ := diff.NewDiffer(diff.TagName("json"))

    changelog, _ := d.Diff(a, b)

    d.Patch(changelog, &a)
    // reflect.DeepEqual(a, b) == true
}

As a convenience, there is a Merge function that allows one to take three interfaces and perform all the tasks at the same time.

import "github.com/r3labs/diff/v2"

type Order struct {
    ID    string `diff:"id"`
    Items []int  `diff:"items"`
}

func main() {
    a := Order{
        ID: "1234",
        Items: []int{1, 2, 3, 4},
    }

    b := Order{
        ID: "1234",
        Items: []int{1, 2, 4},
    }

    c := Order{}
    patchlog, err := diff.Merge(a, b, &c)
    if err != nil {
        fmt.Printf("Error encountered while diffing a & b")
    }
    fmt.Printf("Encountered %d errors while patching", patchlog.ErrorCount())
    ...
}

Running Tests

make test

Contributing

Please read through our contributing guidelines. Included are directions for opening issues, coding standards, and notes on development.

Moreover, if your pull request contains patches or features, you must include relevant unit tests.

Versioning

For transparency into our release cycle and in striving to maintain backward compatibility, this project is maintained under the Semantic Versioning guidelines.

Copyright and License

Code and documentation copyright since 2015 r3labs.io authors.

Code released under the Mozilla Public License Version 2.0.

Owner
R3 Labs
R3 Labs GitHub organisation
R3 Labs
Comments
  • Q: Adding a single struct entry to a map produces multiple changes

    Q: Adding a single struct entry to a map produces multiple changes

    Not sure if this is expected, but if so, you could consider this more of a question.

    As the title says, when a map has an additional entry (entries are custom struct types), then instead of getting 1 Change when comparing the maps, I get N changes (where N=number of fields in the structs).

    I have the following types:

    type Product struct {
    	ID             int `diff:"id,identifier"`
    	Title          string
    	Description    string
    	BaseMaterials  map[MaterialID]bool
    	ExtraMaterials []MaterialID
    	Sizes          []SizeID
    	Weight         int16
    	Prices         map[SizeID]ProductPrice // also a struct type
    	Type           TypeID
    }
    
    type Catalog struct {
      Products map[int]Product `diff:"products"`
    }
    

    And I'm trying to compare two catalogs, of which the one has one key missing from its Catalog.Products map. The following example demonstrates:

    // assume we have catalog1 & catalog2, which are identical
    
    // remove a single entry (Product) from catalog2.Products
    delete(catalog2.Products, 2)
    
    // and compare
    changelog, _ := diff.Diff(catalog1, catalog2)
    

    At this point, the changelog contains N create diffs, where N is the number of fields of the Product. All diffs are of type "from nil to something", since in catalog2 the product is not present at all.

    What I actually need, is to get a single, compound diff with the new entry. Is that possible given the current implementation, or do I have to iterate over the changelog and manually inspect the diffs to combine them to a single one?

    Thanks in advance.

  • Modify slice diffing to detect reordered slice items

    Modify slice diffing to detect reordered slice items

    This changeset modifies slice comparison behavior to indicate that slices are changing when items in the slice are reordered. Previously, doing a comparison of two slices such as []int{1,2,3} and []int{1,3,2} would result in no changelog being generated.

    Because this changes the logic used to compare slices, some slice comparisons will now return different results.

  • Fix flagging change as immutable when finding a field that has

    Fix flagging change as immutable when finding a field that has "-" ta…

    …g before the actual field

    There is an error when flagging a field with a "-" tag in combination with patching, so that a patch to a field is not applied although it should be. Here's an example of the current behavior:

    package main
    
    import (
    	"fmt"
    
    	"github.com/r3labs/diff/v2"
    )
    
    type A struct {
    	F1 string `diff:"-"`
    	F2 string
    }
    
    type B struct {
    	F1 string
    	F2 string
    }
    
    func main() {
    	a1 := A{
    		F1: "test",
    		F2: "value1",
    	}
    	a2 := A{
    		F1: "test",
    		F2: "value2",
    	}
    
    	la, _ := diff.Diff(&a1, &a2)
    	diff.Patch(la, &a1)
    	fmt.Printf("%s\n", a1.F2) // should print "value2", but prints "value1"
    
    	b1 := B{
    		F1: "test",
    		F2: "value1",
    	}
    	b2 := B{
    		F1: "test",
    		F2: "value2",
    	}
    
    	lb, _ := diff.Diff(&b1, &b2)
    	diff.Patch(lb, &b1)
    	fmt.Printf("%s", b1.F2) // works as expected
    }
    

    The problem is: when the field tagged with "-" is found before the actual field to patch, the whole change is flagged "immutable", see patch_struct.go

    The PR fixes that behavior.

  • Patch against map doesn't work

    Patch against map doesn't work

    When I try to Patch map using code below

      testMap := map[string]interface{}{}
      testMap["firstName"] = "John"
      testMap["lastName"] = "Michael"
      testMap["createdBy"] = "TS"
      
      patchLog := diff.Patch(diff.Changelog{{
          Type: "update",
          Path: []string{"createdBy"},
          From: "TS",
          To:   "KS",
        }
      }, &testMap)
      Ω(patchLog[0].Errors).To(BeNil())
    

    I get the following error

    reflect.Set: value of type string is not assignable to type map[string]interface {}
    
  • Patch / Merge

    Patch / Merge

    My first attempt at a Patch / Merge implementation for Diff's change log format.

    Folded into diff's source tree. Features include:

    • Patch function which takes a change log and a pointer to a target
    • Merge function which takes origin and changed values as well as a pointer to a target
    • produces a PatchLog, similar to a ChangeLog in which what action was taken is described
    • Offers a couple of struct tag additions that affect how patch applies changes, these include:
      • omitunequal, which applies the strict rule that the target value must match the origin value or it will not get replaced
      • create, which allows for the creation of map and slice elements should they not already be present
    • Patch honors the "-" name in the diff tag's name portion. thus excluding said strut member from updates
    • Patch honors the diff struct tags 'name' and will apply the change to the target member or map appropriately.
    • There are examples in diff_examples-_test.go These help maintain the code bases unit test coverage percentage
    • There is a new error utility, which allows for nested link lists of errors in a special type.
    • The business of patching is fuzzy and errors don't necessarily mean the operation failed so patch is a 'best effort' operation and will fulfill all the changes in the change lot to the target as best it can. Audit / errors are contained in the PatchLog

    Test pass and code coverage remains above 80% Screen Shot 2020-08-17 at 1 45 28 PM

    A simple usage example: Screen Shot 2020-08-17 at 2 00 20 PM

  • Diffing struct with private field *big.Int fails

    Diffing struct with private field *big.Int fails

    I have problem when trying to diffing struct decimal from https://github.com/shopspring/decimal . looks like it fails because of private field *big.Int

    Example code:

    package main
    
    import (
    	"fmt"
    	"math/big"
    
    	"github.com/r3labs/diff"
    )
    
    type number struct {
    	value *big.Int
    	exp   int32
    }
    
    func (n *number) SetValue(x int64) {
    	n.value = big.NewInt(x)
    }
    
    func main() {
    	a := number{}
    	b := number{}
    	b.SetValue(111)
    
    	diffSliceOrder, _ := diff.NewDiffer(diff.SliceOrdering(true))
    	changelog, err := diffSliceOrder.Diff(a, b)
    
    	fmt.Println(err)
    	fmt.Println(changelog)
    }
    

    we will get a panic

    panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
    
    goroutine 1 [running]:
    reflect.valueInterface(0x525fc0, 0xc000046280, 0xb6, 0x1, 0x0, 0x0)
            c:/go/src/reflect/value.go:1014 +0x1c3
    reflect.Value.Interface(...)
            c:/go/src/reflect/value.go:1003
    github.com/r3labs/diff.(*Differ).diffPtr(0xc0000044e0, 0xc0000462b0, 0x1, 0x1, 0x525fc0, 0xc000046270, 0xb6, 0x525fc0, 0xc000046280, 0xb6, ...)
            E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff_pointer.go:33 +0x8b9
    github.com/r3labs/diff.(*Differ).diff(0xc0000044e0, 0xc0000462b0, 0x1, 0x1, 0x525fc0, 0xc000046270, 0xb6, 0x525fc0, 0xc000046280, 0xb6, ...)
            E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:128 +0x875
    github.com/r3labs/diff.(*Differ).diffStruct(0xc0000044e0, 0x620358, 0x0, 0x0, 0x50bce0, 0xc000046270, 0x99, 0x50bce0, 0xc000046280, 0x99, ...)
            E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff_struct.go:50 +0x460
    github.com/r3labs/diff.(*Differ).diff(0xc0000044e0, 0x620358, 0x0, 0x0, 0x50bce0, 0xc000046270, 0x99, 0x50bce0, 0xc000046280, 0x99, ...)
            E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:112 +0xdf5
    github.com/r3labs/diff.(*Differ).Diff(0xc0000044e0, 0x50bce0, 0xc000046270, 0x50bce0, 0xc000046280, 0x0, 0x461b54, 0x508c80, 0xc000046260, 0xc00008def0)
            E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:101 +0x163
    main.main()
            E:/Go Apps/raditzlawliet/xxx/xxx/experiment/differ.big.Int/main.go:25 +0x18a
    exit status 2
    
  • Diffing maps with pointer values fails

    Diffing maps with pointer values fails

    When diffing a map with pointer values, e.g.:

    struct1 := tmstruct{Bar: 1, Foo: "one"}
    struct2 := tmstruct{Bar: 2, Foo: "two"}
    
    map1 := map[string]*tmstruct{
      "one": &struct1,
    }
    map2 := map[string]*tmstruct{
    "one": &struct1,
    "two": &struct2,
    }
    

    ...this comes back as ErrTypeMismatch due to comparing nil to a pointer.

    I was able to fix this for my use case by modifying diff_pointer.go thusly:

    func (cl *Changelog) diffPtr(path []string, a, b reflect.Value) error {
    	if a.Kind() == reflect.Invalid {
    		cl.add(CREATE, path, nil, b.Interface())
    		return nil
    	}
    
    	if b.Kind() == reflect.Invalid {
    		cl.add(DELETE, path, a.Interface(), nil)
    		return nil
    	}
    	[...]
    

    However, this change breaks the tests mismatched-values-struct-nil and mismatched-values-nil-struct.

    Do you have thoughts on how this conflict could be resolved?

    For testing my change, I added these cases to the case list in TestDiff:

    		{
    			"map-string-pointer-create-test",
    			map[string]*tmstruct{"one": &struct1},
    			map[string]*tmstruct{"one": &struct1, "two": &struct2},
    			Changelog{
    				Change{Type: CREATE, Path: []string{"two"}, From: nil, To: &struct2},
    			},
    			nil,
    		},
    		{
    			"map-string-pointer-delete-test",
    			map[string]*tmstruct{"one": &struct1, "two": &struct2},
    			map[string]*tmstruct{"one": &struct1},
    			Changelog{
    				Change{Type: DELETE, Path: []string{"two"}, From: &struct2, To: nil},
    			},
    			nil,
    		},
    
  • Error with update change

    Error with update change

    When I have two slices with two changes (update and create) the changelogs returned is not what it is expected. The example is:

    I have two slices like these: a := []map[string]interface{}{ { "name": "name1", "type": []string{"null", "string"}, }, }

    b := []map[string]interface{}{ { "name": "name1", "type": []string{"null", "int"}, }, { "name": "name2", "type": []string{"null", "string"}, }, }

    changelog, _ := diff.Diff(a, b)

    The changelog is:

    [{create [0 type 1] <nil> int} {create [1 name] <nil> name2} {create [1 type] <nil> [null string]}]

    But the expected result is:

    [{update [0 type 1] string int} {create [1 name] <nil> name2} {create [1 type] <nil> [null string]}]

    I saw that in the diff_map.go file there is a function called swapChange that does the change from "update" to "create". Maybe here is the problem.

    Thank you very much.

  • Type lost in serialization

    Type lost in serialization

    Hi, this might not be a bug, but an issue I´ve ran into and need to work around. Consider the following

    type Test struct {
    	S *int `json:"s,omitempty"`
    }
    
    func TestPointerDiff(t *testing.T) {
    	val1 := 1
    	val2 := 2
    
    	t1 := Test{S: &val1}
    	t2 := Test{S: &val2}
    
    	changelog, err := diff.Diff(t1, t2)
    	assert.NoError(t, err)
    
    	js, err := json.Marshal(changelog)
    	assert.NoError(t, err)
    
    	assert.NoError(t, json.Unmarshal(js, &changelog))
    
    	patchLog := diff.Patch(changelog, &t1)
    	assert.False(t, patchLog.HasErrors())
    }
    

    This will fail, since the type of the int is lost when marshaling to JSON.

    Any ideas how I could work around this?

  • [v2.14.0] panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

    [v2.14.0] panic: reflect.Value.Interface: cannot return value obtained from unexported field or method

    When trying to use the github.com/deckarep/golang-set library, running a diff on the sets causes a panic from the reflection package.

    Here is a minimum reproducible example:

    package main
    
    import (
    	"github.com/deckarep/golang-set"
    	"github.com/r3labs/diff/v2"
    )
    
    func main() {
    	a := mapset.NewSet("a", "b", "c")
    	b := mapset.NewSet("a", "c")
    	_, _ = diff.Diff(a, b)
    }
    

    And an example error:

    panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
    
    goroutine 1 [running]:
    reflect.valueInterface({0x8517c0, 0xc000088350, 0x0}, 0xc0)
    	C:/Program Files/Go/src/reflect/value.go:1362 +0xd9
    reflect.Value.Interface(...)
    	C:/Program Files/Go/src/reflect/value.go:1351
    github.com/r3labs/diff/v2.(*Differ).diffMap(0x854a00, {0xc000088340, 0x1, 0x1}, {0x854a00, 0xc0000a63a0, 0x854a00}, {0x854a00, 0xc0000a63c0, 0x1b5})
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff_map.go:27 +0x59f
    github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0xc000088340, 0x1, 0x1}, {0x854a00, 0xc0000a63a0, 0xc0000ac058}, {0x854a00, 0xc0000a63c0, 0x1b5}, ...)
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff.go:182 +0xb30
    github.com/r3labs/diff/v2.(*Differ).diffStruct(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x857020, 0xc0000a63a0, 0x54}, {0x857020, 0xc0000a63c0, 0x199})
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff_struct.go:62 +0xad3
    github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x857020, 0xc0000a63a0, 0x92b2d0}, {0x857020, 0xc0000a63c0, 0x199}, ...)
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff.go:166 +0xf1e
    github.com/r3labs/diff/v2.(*Differ).diffPtr(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x863960, 0xc0000a63a0, 0x2e1cf6a7000}, {0x863960, 0xc0000a63c0, 0x16}, ...)
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff_pointer.go:45 +0x5bc
    github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x863960, 0xc0000a63a0, 0xc0000cfe01}, {0x863960, 0xc0000a63c0, 0x16}, ...)
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff.go:184 +0xabd
    github.com/r3labs/diff/v2.(*Differ).Diff(0xc0000e0000, {0x863960, 0xc0000a63a0}, {0x863960, 0xc0000a63c0})
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff.go:129 +0x1b3
    github.com/r3labs/diff/v2.Diff({0x863960, 0xc0000a63a0}, {0x863960, 0xc0000a63c0}, {0x0, 0x2e1cf630598, 0x60})
    	C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/[email protected]/diff.go:72 +0x75
    main.main()
    	C:/Users/Vilsol/go/src/awesomeProject/difftest/main.go:11 +0x11b
    

    I am using:

    • Go -> 1.17
    • github.com/r3labs/diff/v2 -> v2.14.0
    • github.com/deckarep/golang-set -> v1.7.1
  • Patch delete for nested map deletes parent

    Patch delete for nested map deletes parent

    When I try to Patch map using code below, it removes details as well as attributes:

    testMap := map[string]interface{}{}
    testMap["firstName"] = "John"
    testMap["lastName"] = "Michael"
    testMap["createdBy"] = "TS"
    testMap["details"] = map[string]interface{}{
      "status": "active",
      "attributes": map[string]interface{}{
    	  "attrA": "A",
    	  "attrB": "B",
      },
    }
    
    diff.Patch(diff.Changelog{{
      Type: "delete",
      Path: []string{"details", "attributes"},
      From: []interface{}{
        map[string]interface{}{
          "Key":   "attrA",
          "Value": "A",
        },
        map[string]interface{}{
          "Key":   "attrB",
          "Value": "B",
        },
      },
      To: nil,
      }
    }, &testMap)
    Ω(testMap["details"]).NotTo(BeNil())
    

    Result:

    Expected
        <nil>: nil
    not to be nil
    

    Expected to have at the end:

    {
      "firstName": "John",
      "lastName": "Michael",
      "createdBy": "TS",
      "details": {
        "status": "active"
      }
    }
    
  • Changelog Serialization - Interface issue

    Changelog Serialization - Interface issue

    Hi diff team,

    I am working on a project that is leveraging the library and there appears to be a problem with interfaces when serializing the change logs. The types get lost in translation.

    I have created a simplified reproduction of the issue at hand that exemplifies the issue. In the example there is a type that has a field which is an interface where we are adding different types that qualify for the interface. When we serialize the changelog, then de-serialize the changelog, then apply it as a patch, you will see the error. The important piece is that it does not recognize what type it was from before the change log was serialized.

    type Something interface {
    	Name() string
    }
    type A struct {
    	First  string
    	Second string
    }
    
    func (a A) Name() string {
    	return a.First + a.Second
    }
    
    type B struct {
    	First  string
    	Second string
    }
    
    func (b B) Name() string {
    	return b.First + b.Second
    }
    
    type Example struct {
    	This []Something
    }
    
    func TestChangeExample(t *testing.T) {
    	before := Example{This: []Something{A{First: "Joe", Second: "Shmo"}}}
    	after := Example{This: []Something{A{First: "Joe", Second: "Shmo"}, B{First: "Jane", Second: "Doe"}}}
    	differ, err := diff.NewDiffer(diff.ConvertCompatibleTypes())
    	if err != nil {
    		t.Fatal(err)
    	}
    	cl, err := differ.Diff(&before, &after)
    	if err != nil {
    		t.Fatal(err)
    	}
    	b, err := json.Marshal(&cl)
    	if err != nil {
    		t.Fatal(err)
    	}
    	var newCL diff.Changelog
    	err = json.Unmarshal(b, &newCL)
    	if err != nil {
    		t.Fatal(err)
    	}
    
    	pl := diff.Patch(newCL, &before)
    	for _, p := range pl {
    		if p.Errors != nil {
    			t.Fatal(p.Errors)
    		}
    	}
    	println("success?")
    }
    

    In the Goland IDE inspection and comparison to the before after shows that it does not recognize the same type: image

  • Patching slice back to empty slice returns patchLog.Applied() to be false

    Patching slice back to empty slice returns patchLog.Applied() to be false

    patchLog.Applied() is false when we run the following code snippet regardless the patch operation works as expected.

    package main
    
    import (
    	"fmt"
    
    	"github.com/r3labs/diff/v3"
    )
    
    type Order struct {
    	ID         string     `json:"id"`
    	OrderItems OrderItems `json:"orderItems"`
    }
    
    type OrderItems struct {
    	Items []string `json:"items"`
    }
    
    func main() {
    	a := Order{
    		ID: "1234",
    	}
    
    	b := Order{
    		ID:         "1234",
    		OrderItems: OrderItems{[]string{"1", "2", "4"}},
    	}
    
    	d, _ := diff.NewDiffer(diff.TagName("json"))
    
    	changelog, _ := d.Diff(b, a)
    
    	patchLog := d.Patch(changelog, &b)
    	fmt.Printf("patchlog applied : %t \nhas errors %t \nerror count %d \n\n", patchLog.Applied(), patchLog.HasErrors(), patchLog.ErrorCount())
    
    	fmt.Printf("values \n a: %#v \n b: %#v", a, b)
    }
    
  • Patch failing for byte slices when patching onto struct with nil destination

    Patch failing for byte slices when patching onto struct with nil destination

    I'm getting some unexpected behaviour when diffing and applying the changelog for byte slices. In this example I am comparing 2 structs that have a byte slice and patching the result onto an empty struct that has a nil byte slice. Is this supported behaviour?

    Go Playground

    package main
    
    import (
    	"fmt"
    
    	"github.com/r3labs/diff/v3"
    )
    
    type MyType struct {
    	MyField []byte
    }
    
    func main() {
    
    	left := &MyType{[]byte{208, 72, 51, 52, 175, 134, 76, 84, 143, 38, 99, 184, 128, 24, 107, 163}}
    	right := &MyType{[]byte{91, 102, 170, 173, 254, 105, 66, 81, 177, 175, 32, 173, 173, 165, 129, 192}}
    
    	changelog, err := diff.Diff(left, right)
    	if err != nil {
    		fmt.Println(err)
    	}
    
    	dest := &MyType{}
    	_ = diff.Patch(changelog, dest)
    
    	fmt.Println(left.MyField)  // [208 72 51 52 175 134 76 84 143 38 99 184 128 24 107 163]
    	fmt.Println(right.MyField) // [91 102 170 173 254 105 66 81 177 175 32 173 173 165 129 192]
    	fmt.Println(dest.MyField)  // [32 173 173 165 254 192] ?
    
    	// Patchlog errors:
    
    	// [MyField 0] 208 91  Value index 0 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 1] 72 102  Value index 1 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 2] 51 170  Value index 2 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 3] 52 173  Value index 3 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 5] 134 105  Value index 5 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 6] 76 66  Value index 6 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 7] 84 81  Value index 7 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 8] 143 177  Value index 8 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 9] 38 <nil>  Value index 9 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 10] 99 32  Value index 10 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 11] 184 173  Value index 11 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 12] 128 173  Value index 12 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 13] 24 165  Value index 13 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 14] 107 129  Value index 14 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    	// [MyField 15] 163 192  Value index 15 is invalid (cause count 1)
    	//  scanning for Value index (cause count 0)
    
    }
    
  • option for ignoring unexported fields

    option for ignoring unexported fields

    👋🏻 hey there I'm using this library as it has been the most consistent way to compare structs that I've found so far, thanks!

    one feature that I'm missing is the option to ignore unexported fields from being compared and marked as differences, I know I can strip them by marshal/unmarshal but if that functionality would be provided by the library itself that would be amazing 🙂

    I'm ok with implementing it myself and submitting a pr! I would like to know

    • is this something you're ok with adding?
    • is there some limitation that I could hit while adding this feature?
    • any hints where I should start from?

    I haven't checked the codebase yet, that's why I'm asking this 😅

  • net.IP comparable as String.

    net.IP comparable as String.

    How do I force differ function so it would compare net.IP as a string and not as a slice?

    https://go.dev/play/p/0VMlM5RQ9WO

    package main
    
    import (
    	"log"
    	"net"
    
    	"github.com/davecgh/go-spew/spew"
    	"github.com/r3labs/diff/v2"
    )
    
    type LoadBalancer struct {
    	IP []net.IP
    }
    
    func main() {
    	x := LoadBalancer{
    		IP: []net.IP{
    			net.ParseIP("192.0.2.1"),
    			net.ParseIP("192.0.2.2"),
    		},
    	}
    
    	y := LoadBalancer{
    		IP: []net.IP{
    			net.ParseIP("192.0.2.1"),
    			net.ParseIP("192.0.2.3"),
    		},
    	}
    
    	changelog, err := diff.Diff(x, y)
    	if err != nil {
    		log.Fatal(err)
    	}
    
    	spew.Dump(changelog)
    }
    
    (diff.Changelog) (len=1 cap=1) {
     (diff.Change) {
      Type: (string) (len=6) "update",
      Path: ([]string) (len=3 cap=3) {
       (string) (len=2) "IP",
       (string) (len=1) "1",
       (string) (len=2) "15"
      },
      From: (uint8) 2,
      To: (uint8) 3,
      parent: (interface {}) <nil>
     }
    }
    
    
  • Show entire struct data in From/To even though it's only partially updated

    Show entire struct data in From/To even though it's only partially updated

    package main
    
    import (
    	"fmt"
    
    	"github.com/r3labs/diff/v2"
    )
    
    type Data struct {
    	ID    int32  `json:"id" diff:"ID"`
    	Value string `json:"value" diff:"Value"`
    }
    
    type Order struct {
    	Items []Data `diff:"Items,ID"`
    }
    
    func main() {
    	a := Order{
    		Items: []Data{Data{ID: 1, Value: "foo"}},
    	}
    
    	b := Order{
    		Items: []Data{Data{ID: 1, Value: "bar"}, Data{ID: 2, Value: "paper"}},
    	}
    
    	changelog, err := diff.Diff(a, b, diff.DisableStructValues())
    	if err != nil {
    		fmt.Println("ERROR", err)
    		return
    	}
    	fmt.Printf("%+v\n", changelog)
    
    }
    

    Output

    [{Type:update Path:[Items 0 Value] From:foo To:bar parent:{ID:1 Value:foo}} {Type:create Path:[Items 1] From:<nil> To:{ID:2 Value:paper} parent:<nil>}]
    

    What I would like is a way so that instead of

    {Type:update Path:[Items 0 Value] From:foo To:bar parent:{ID:1 Value:foo}}
    

    I get

    {Type:update Path:[Items 0 Value] From:{ID: 1 Value: foo} To:{ID: 1 Value: bar} parent:{ID:1 Value:foo}}
    
Visualize your Go data structures using graphviz

memviz How would you rather debug a data structure? "Pretty" printed Visual graph (*test.fib)(0xc04204a5a0)({ index: (int) 5, prev: (*test.fib)(0xc0

Dec 22, 2022
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.

Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package. The library allows you to call Go service methods from PHP with a minimal footprint, structures and []byte support.

Dec 28, 2022
Govalid is a data validation library that can validate most data types supported by golang

Govalid is a data validation library that can validate most data types supported by golang. Custom validators can be used where the supplied ones are not enough.

Apr 22, 2022
Golang library to act on structure fields at runtime. Similar to Python getattr(), setattr(), hasattr() APIs.

go-attr Golang library to act on structure fields at runtime. Similar to Python getattr(), setattr(), hasattr() APIs. This package provides user frien

Dec 16, 2022
A Go (golang) library for parsing and verifying versions and version constraints.

go-version is a library for parsing versions and version constraints, and verifying versions against a set of constraints. go-version can sort a collection of versions properly, handles prerelease/beta versions, can increment versions, etc.

Jan 9, 2023
A well tested and comprehensive Golang statistics library package with no dependencies.

Stats - Golang Statistics Package A well tested and comprehensive Golang statistics library / package / module with no dependencies. If you have any s

Dec 30, 2022
go-sundheit:A library built to provide support for defining service health for golang services
go-sundheit:A library built to provide support for defining service health for golang services

A library built to provide support for defining service health for golang services. It allows you to register async health checks for your dependencies and the service itself, and provides a health endpoint that exposes their status.

Dec 27, 2022
Cogger is a standalone binary and a golang library that reads an internally tiled geotiff

Cogger is a standalone binary and a golang library that reads an internally tiled geotiff (optionally with overviews and masks) and rewrites it

Dec 12, 2022
GoLang port of Google's libphonenumber library

phonenumbers golang port of Google's libphonenumber, forked from libphonenumber from ttacon which in turn is a port of the original Java library. You

Jan 4, 2023
A concurrent rate limiter library for Golang based on Sliding-Window rate limiter algorithm.

ratelimiter A generic concurrent rate limiter library for Golang based on Sliding-window rate limitng algorithm. The implementation of rate-limiter al

Jan 6, 2023
MNA - stands for mobile number assignment - a small zero external dependency golang library that is used to identify mobile number assignment in tanzania

MNA - stands for mobile number assignment - a small zero external dependency golang library that is used to identify mobile number assignment in tanzania

Nov 29, 2021
Easy to use, light enough, good performance Golang library
 Easy to use, light enough, good performance Golang library

指令使用 特性 简单易用、足够轻量,避免过多的外部依赖,最低兼容 Window 7 等老系统 快速上手 安装 $ go get github.com/sohaha/zlsgo HTTP 服务 // main.go

Dec 29, 2022
GoDynamic can load and run Golang dynamic library compiled by -buildmode=shared -linkshared

GoDynamic can load and run Golang dynamic library compiled by -buildmode=shared -linkshared How does it work? GoDynamic works like a dynamic

Sep 30, 2022
The main goal of this code is to create a basic dnstap printing tool based on the golang-dnstap library.

dnstap-parse The main goal of this code is to create a basic dnstap printing tool based on the golang-dnstap library. The output is supposed to mimic

Nov 14, 2021
A prototype code-generator library for golang.

A prototype code-generator library for golang.

Jul 28, 2022
Gopher protocol library for Golang

Gopher protocol library for Golang This is a standards compliant Gopher library for the Go programming language implementing the RFC 1436 specificatio

Nov 13, 2021
A golang library to validate and format swiss social security numbers

s3n is a golang library to validate and format swiss social security numbers (aka. AVS in french and AHV in german).

Nov 15, 2021
maskerito is masking library for golang. Especially for indonesia dictionary.

maskerito maskerito is masking library for golang. Especially for indonesia dictionary. Library maskerito provides a library to do masking struct and

Jul 28, 2022
Go-Utils is a library containing a collection of Golang utilities

Go-Utils is a library containing a collection of Golang utilities

Jun 2, 2022