I want to compare all fields in two large structs, except the unexported fields. Is there any way to achieve that without manually specifying each field in a SStruct? I might be missing an operator/helper function, but the best solution that comes to my mind is to compare the JSON representation of the two structs.

Example for clarification:

type X struct {
  Field1      int
  Field2      int
  Field3      int
  Field4      int
  Field5      int
  fieldIgnore int // <--- want to ignore this when doing a t.Cmp(x1, x2)
}

Here to annoy you again :P

I'm aiming to support the encoding of reflect.Type and reflect.Value in my encoding library. go-testdeep has been amazing for tests, but test cases where I compare a reflect.Value values, go-testdeep compares them like a typical struct, and doesn't compare the underlying value inside reflect.Value.

This means a reflect.Value of a value is only equal if the underlying value is at the same pointer address, and the reflect.Value was created in the same way due to how reflect uses the flag field in reflect.Value.

In any case, I don't think the semantics of how reflect.Value works is important. For my use case, I'm looking to compare the value inside reflect.Value, and I don't care so much about the reflect.Value itself, e.g if both are addressable or if only one is, although an argument could be made for this. I believe something like

if got.Type() == reflect.TypeOf(reflect.Value{}) {
    return deepValueEqual(ctx, got.Interface().(reflect.Value), expected.Interface().(reflect.Value))
}

in deepValueEqual could work, but I'm unsure how this might work with the rest of the codebase. I also know that the ptr field of reflect.Value could introduce an infinite recursion, an extreme case being a reflect.Value that is the value of itself, but less contrived examples do exist.

I'd be happy to experiment implementing this and submit a PR. Is this something you'd want to support?

Is there a way to compare submaps/supermaps recursively (make sure that partial compare is propagated to nested maps)? Based on what I see, the default behavior is that only the parent map is compared partially, while all nested maps are expected to have exact match.

The TestingFT interface includes the Run method, which uses the *testing.T parameter directly, which means that go-testdeep cannot be used with *testing.B or alongside other testing helper packages such as quicktest.

If you embed testing.TB inside T instead of defining your own testingFT interface, this problem goes away, and additionally you can take advantage of new features added to testing.TB (Cleanup for example) automatically and without breaking your API by adding methods to an interface type.

Problem: We use different operators in JSONs which are then used for comparison. If we add regular operators to json string, e.g. Len, HasPrefix, etc, the JSON becomes invalid and formatters, validators cannot parse the string.

Proposal: Add support for quotation marks for operators in JSON strings, similarly to currently it is being used for shortcut operators "$^NotEmpty". Ideally it could look similar "$^Len(10)"

julien@julien-VirtualBox:~/git/go-testdeep$ go test ./... -cover
ok      github.com/maxatome/go-testdeep 0.038s  coverage: [no statements]
ok      github.com/maxatome/go-testdeep/helpers/nocolor 0.047s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/helpers/tdhttp  0.075s  coverage: 99.4% of statements
ok      github.com/maxatome/go-testdeep/helpers/tdhttp/internal 0.017s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/helpers/tdsuite 0.019s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/helpers/tdutil  0.004s  coverage: 99.4% of statements
ok      github.com/maxatome/go-testdeep/internal/anchors        0.022s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/color  0.011s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/ctxerr 0.007s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/dark   0.003s  coverage: 95.6% of statements
ok      github.com/maxatome/go-testdeep/internal/flat   0.011s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/hooks  0.009s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/json   0.006s  coverage: 91.6% of statements
?       github.com/maxatome/go-testdeep/internal/location       [no test files]
?       github.com/maxatome/go-testdeep/internal/test   [no test files]
ok      github.com/maxatome/go-testdeep/internal/trace  0.005s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/types  0.008s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/util   0.016s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/internal/visited        0.007s  coverage: 100.0% of statements
ok      github.com/maxatome/go-testdeep/td      0.209s  coverage: 99.8% of statements

testdeep encourages wrapping the *testing.T into a *td.T. This unfortunately makes it difficult to call t.Parallel() on the *testing.T once hidden behind the *td.T.

func TestFoo(tt *testing.T) {
    t := td.NewT(tt)
    
    t.Run("foo", func(t *td.T) {
        // t.Parallel() <<< errors because Parallel() does not exist
        //                  on the testing.TB interface.
        
        t.Cmp(...)
    })
    
    t.Run("bar", func(t *td.T) {
        // t.Parallel()
        
        t.Cmp(...)
    })
}

It's possible to call t.Parallel() on the *testing.T with some upcasting on the wrapped testing.TB.

func Parallel(t *td.T) {
	tp, ok := t.TB.(interface{ Parallel() })
	if ok {
		tp.Parallel()
	}
}

If you're open to addressing this within testdeep, there are two straightforward options:

• Add a top-level func Parallel(*td.T) that does the same as above. Usage would then be,

  t.Run("foo", func(t *td.T) {
      td.Parallel(t)
  
      t.Cmp(...)
  })
  

• Add a Parallel() method on td.T. Usage:

  t.Run("foo", func(t *td.T) {
      t.Parallel()
  
      t.Cmp(...)
  })
  

The latter is closest to what people already do with *testing.T so it might be preferable.

Separately, either choice opens room for a setting in testdeep to always call t.Parallel() for subtests invoked with a specific *td.T.Run. For example,

func TestFoo(tt *testing.T) {
    t := td.NewT(tt).AlwaysParallel(true)
    
    t.Run("foo", func(t *td.T) {
        // already parallel
        t.Cmp(...)
    })
    
    t.Run("bar", func(t *td.T) {
        // already parallel
        t.Cmp(...)
    })
}

But that can be a separate discussion.

I've found an infinite recursion case with go-testdeep. It's a bit contrived I know; I was actually trying to break things. I'm currently developing an encoding library and trying to break it with test cases galore. Interestingly enough encoding/gob, reflect.DeepEqual (fixed in go1.15), and every 3rd party library I've found suffers from this same issue.

If given a map containing a struct that contains the same map, deepValueEqual recuses infinitely.

This test case reproduces the issue

func TestEqualRecursMap(t *testing.T) {
	gen := func() interface{} {
		type S struct {
			Map map[int]S
		}

		m := make(map[int]S)
		m[1] = S{
			Map: m,
		}

		return m
	}

	checkOK(t, gen(), gen())
}

Thanks so much for your work. Let me know if I can help.

Using two new kinds of function:

• func(t *td.T, arg)
• func(assert, require *td.T, arg)

this way the usage of *td.T methods is secure. Note that these functions do not return anything.

At the same time, using a *td.T instance as Cmp* first parameter allows to inherit its configuration.

td.Cmp(td.Require(t), got, 42)

is the same as:

td.Require(t).Cmp(got, 42)

Given the 5 hooks:

• Setup
• PreTest
• PostTest
• BetweenTests
• Destroy raise an error when a method is defined on a tdsuite with one of these names but without matching the proper interface.

I'm building some Stripe tests with large, deeply nested structs, where using anchors is very helpful. They work great when I have a single test, but if I'm calling t.Cmp multiple times or have table driven tests, anchors are being compared against the wrong values. Since the anchors have to be set outside the inner test run loop, I'm not sure how td knows when to iterate between them. Am I doing something incorrectly?

Table driven test

func Test_payInvoiceWithStripe(stdT *testing.T) {
	tdt := td.NewT(stdT)

	tests := []struct {
		name                  string
		stripeCustomerID      string
		stripePaymentMethodID string
		wantPaymentIntent     *stripe.PaymentIntent
		wantErr               error
	}{
		// lots of tdt.A
	}

	stripeClient := client.New(apiKey, nil)
	invoice := &billingpb.Invoice{
		InvoiceId:            111,
		AccountId:            22222,
	}

	for i, tt := range tests {
		tdt.Run(tt.name, func(t *td.T) {
			gotPaymentIntent, err := chargeCustomer(context.Background(), stripeClient, tt.stripeCustomerID, tt.stripePaymentMethodID, invoice, int64(i))
			if tt.wantErr == nil {
				t.CmpNoError(err)
			} else {
				t.Cmp(e.Cause(err).(*stripe.Error), e.Cause(tt.wantErr).(*stripe.Error))
			}
			t.Cmp(gotPaymentIntent, tt.wantPaymentIntent)
		})
	}

Output

        DATA.ChargeID: values differ
        	     got: "ch_3KH0DO4HQ2O7mN7V1o3cxNfi"
        	expected: "<testdeep@anchor#12>"

Thanks for a great library!

It would be helpful to have a comparison table or something in the readme to help people understand the difference between this package and https://github.com/go-test/deep. That has more stars and shows up better in search, while this library seems much more active.

The request in a nutshell: can enough internal details be exposed to allow building custom operators externally? Ideally in a stable way.

I very much realize this is not a simple request, but someone has to make it :) Maybe it can serve as a placeholder for planning / discussion. Some context / broader reasoning is below.

TestDeep looks super interesting! From a quick glance it looks like there are a lot of good, flexible, well-thought-out features, especially around good error messages (that's so rare, thank you!).

While there are quite a few built-in operators, there are always some edge cases that benefit from custom ones. The JSON operators serve as a nice example of this: it's just a string, but there's a lot of structure that can imply much-better error reporting and a simpler API, so it has a custom operator to improve things.
Unfortunately, though td/types.go exposes the TestDeep interface, the base type is not exposed, and location.GetLocationer deals with the (afaict) internal-only Location type, so custom operators cannot be built outside this library. There may be other causes as well, this one was just one I saw while reading.

This is... probably a very reasonable choice for API stability, but somewhat crippling for less-common needs. I assume you don't want to collect every ill-advised and poorly-implemented operator that every user/company/etc dreams up, so being able to build things from the outside would be great. It could also enable "enriching" libraries that enhance TestDeep.

Is there a way this can be done?