The problem that if you assert if an array has items in it, to be able to assert the data in the items, and if the array is empty a panic will occur in Goconvey. This is not the case if running just go test. The default behaviour of other test suites is that if an assertion fails it stop and don't assert any further in that test case.

I noticed that running goconvey also navigates my /examples folder for tests and tries to build go files found inside them. In my project, this causes a build failure as some of these examples require a specific setup (e.g. appengine).

My suggestion is the have a .goconvey file containing folder names to skip (similar to .gitignore).

So... that wasn't too bad either :) However, the test output (both story and json) are basically garbled junk :(.

What's the right thing to do with reporting? I could:

1. buffer everything till the end and emit it in a stable sort order
2. stream the output (but emit stories multiple times)
3. ???

1 is pretty explanatory

2 would be something like:

Convey("A", t, func() {
  Convey("B", func() {
    Convey("C", nil)
    Convey("D", nil)
  })  
  Convey("F", func() {
    Convey("G", nil)
  })
})

Could emit:

A
  B
    C
  F
    G
  B
    D

(assuming that's how they randomized).

Or is there another idea?

[19:59:38.121] Compiling package statistics
[19:59:38.133]     Assertions: 47
[19:59:38.134]         Passed: 47
[19:59:38.134]        Skipped: 0
[19:59:38.135]       Failures: 0
[19:59:38.135]         Panics: 0
[19:59:38.135] Build Failures: 0
[19:59:38.136]       Coverage: 0%

I just tried out the new UI, it is awesome! But I got 0% test coverage :(

We'd like to open the project up to additional maintainers who want to move the project forward in a meaningful way.

We've spent significant time at SmartyStreets building GoConvey and it has perfectly met (and exceeded) all of our initial design specifications. We've used it to great effect. Being so well-matched to our development workflows at SmartyStreets, we haven't had a need to hack on it lately. This had been frustrating to many in the community who have ideas for the project and would like to see new features released (and some old bugs fixed). The release of Go 1.5 and the new vendoring experiment has been a source of confusion and hassle for those who have already upgraded and find that GoConvey needs to be brought up to speed.

Comment below if you're interested. Preference will be given to those that have already contributed to the project. Checkout the issues listing if you need some ideas for contributing.

GoConvey is a popular 2-pronged, open-source github project (1,600+ stargazers, 100+ forks):

• A package you import in your test code that allows you to write BDD-style tests.
• An executable that runs a local web server which displays auto-updating test results in a web browser.

http://goconvey.co/ https://github.com/smartystreets/goconvey https://github.com/smartystreets/goconvey/wiki

I should mention that the assertions package imported by the convey package is used by other projects at SmartyStreets and so we will be continuing to maintain that project internally.

We hope to hear from you soon. Thanks!

--- FAIL: TestGetAccount (0.00s) panic: Top-level calls to Convey(...) need a reference to the testing.T. Hint: Convey("description here", t, func() { / notice that the second argument was the *testing.T (t)! */ }) [recovered] panic: Top-level calls to Convey(...) need a reference to the testing.T. Hint: Convey("description here", t, func() { / notice that the second argument was the *testing.T (t)! */ }) [recovered] panic: Top-level calls to Convey(...) need a reference to the testing.T. Hint: Convey("description here", t, func() { / notice that the second argument was the *testing.T (t)! */ })

` func TestGetAccount(t *testing.T) {

Convey("test1", t, func() {
	Convey("test1-1", func() {
		 //...
	})
})

}`

I'm collecting some ideas for a major upgrade to the graphical interface for GoConvey, so long as my free time permits. (Free time? What's that.)

What would you like to see new or improved? How's the layout? Is everything intuitive? What about it drives you crazy that you'd like changed? (Or what do you really like that you want to make sure never goes away?)

I will consider all suggestions but I will also accept them at my own discretion... and I'm not promising any timeline.

Not guaranteed, but on the list already (depending on feasibility):

• [ ] ~~Preserve package "ignore" settings across server instances~~
• [x] Preserve collapsed packages across test runs
• [x] Full coverage report from the cover tool by clicking on package name (requires server changes)
• [ ] ~~Open browser window to localhost:8080 when GoConvey server starts up~~ (not for now)
• [x] Light and dark theme
• [x] Custom themes
• [x] Test history
• [x] Pause auto-execution of tests
• [x] Keyboard shortcuts (issue #102)
• [x] Better of use of space, depending if tests pass/fail/etc.
• [x] Resources are locally stored (can work offline)
• [ ] ~~Basic delta between tests~~ (not for now)
• [x] Better coverage stats (aggregated results, etc.)
• [x] Toggle debug output that appears alongside test cases
• [x] Tests skipped with t.Skip() should be marked as skip, not failure (issue #115)

Feel free to +1 any already mentioned that you really want.

From the web UI, we should be able to specify fully qualified package names to ignore in case large dependencies (or broken dependencies) start to get in the way or slow things down.

package tutorial

import (
    "fmt"
    . "github.com/smartystreets/goconvey/convey"
    "testing"
)

func TestFizzBuzz(t *testing.T) {
    cases := map[int]string{
        1:  "1",
        2:  "2",
        3:  "fizz",
        4:  "4",
        5:  "buzz",
        6:  "fizz",
        7:  "7",
        8:  "8",
        9:  "fizz",
        10: "buzz",
        15: "fizzbuzz",
    }

    Convey("Given a bunch of test cases", t, func() {
        for input, expected := range cases {
            Convey(fmt.Sprintf("Fizzbuzz for %d should equal '%s'", input, expected), func() {
                So(fizzbuzz(input), ShouldEqual, expected)
            })
        }
    })
}

===================


$ go test -v
=== RUN TestFizzBuzz

  Given a bunch of test cases 
    Fizzbuzz for 1 should equal '1' ✔

1 assertion thus far

--- PASS: TestFizzBuzz (0.00 seconds)
PASS
ok      github.com/smartystreets/tutorial   0.022s

Fix #81 for real this time, Closes #261.

Includes:

• New tree-based state in Convey execution, using jtolds/gls for goroutine-local storage
• New (optional) explicit context passing when you need to bridge a goroutine invocation
• Minimal fixes for the reporting package to ensure they work with the tree-based state

I have test output that looks like this:

--- FAIL: TestParseErr (0.00 seconds)
        parse_test.go:317: Parse error case: conditional arg  1:
                 Got: "/err_test/path:1: expected [\"(\", \"<TYP_SINGLE_QUOTED_STR>\", \"<TYP_DOUBLE_QUOTED_STR>\"]; found: \"a1\""
                Want: "/err_test/path:1: expected [\"<TYP_SINGLE_QUOTED_STR>\", \"<TYP_DOUBLE_QUOTED_STR>\"]; found: \"a1\""
FAIL
exit status 1

Notice the single extra space before the word "Got" that causes it to align with the word "Want". Also, notice the horizontal scrollbar that appears when github displays the message (i.e. no wrapping). Together these two properties make it easy for me to compare the two lines visually.

It would be nice if the "FAILURES" or "STORIES" sections could be configured to display their output in this way. It's not too big a deal if they don't since I can always copy the output into a text editor and format it accordingly. Those are just extra steps that would be nice to be able to skip.

https://nvd.nist.gov/vuln/detail/cve-2022-24785

Severity: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:H/A:N -- 7.5 High

Current version in head branch: https://github.com/smartystreets/goconvey/blob/883c12515e6101b79f430987b4fd0ee50841bfe6/web/client/resources/js/lib/moment.js#L1-L2

I'm personally using Goland, code snippet is like below

SoMsg(testCase.desc, runFunc(flow), ShouldEqual, testCase.expectedRes)

Test report is like following in IDE: Line 70: Expected: 'true' Actual: 'false' (Should be equal)

Didn't see any msg from test case

Version: v1.7.2 ApplyFunc use github.com/agiledragon/gomonkey/v2 v2.1.0

Code:

func Test(t *testing.T) {
	Convey("Test", t, func() {
		defer ApplyFunc(json.Marshal, func(v interface{}) ([]byte, error) {
			return nil, errors.New("test shouldNotEqual")
		}).Reset()

		_, err := json.Marshal("")
		So(err, ShouldNotEqual, nil)
	})
}

Result:

Failures:
  Line 97:
  Expected     'test shouldNotEqual'
  to NOT equal '<nil>'
  (but it did)!


1 total assertion

--- FAIL: Test (0.00s)

FAIL

Using go 1.19 (also happened with go 1.18) and goconvey v1.7.2 both locally and in a GitHub action, my tests pass locally but fail in GitHub:

https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true

Failures:
[144](https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true#step:4:145)

[145](https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true#step:4:146)
  * /home/runner/work/wrstat/wrstat/server/server_test.go 
[146](https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true#step:4:147)
  Line 450:
[147](https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true#step:4:148)
  Expected: '[]*server.DirSummary{(*server.DirSummary){Dir:"/a", Count:15, Size:86, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"adm", "docker", "root"}, FileTypes:[]string{"bam", "cram", "temporary"}}, (*server.DirSummary){Dir:"/a/b", Count:9, Size:80, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"bam", "cram", "temporary"}}, (*server.DirSummary){Dir:"/a/b/d", Count:7, Size:70, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"cram"}}, (*server.DirSummary){Dir:"/a/b/e/h", Count:2, Size:10, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"bam", "temporary"}}, (*server.DirSummary){Dir:"/a/c/d", Count:5, Size:5, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root"}, Groups:[]string{"adm"}, FileTypes:[]string{"cram"}}}'
[148](https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true#step:4:149)
  Actual:   '[]*server.DirSummary{(*server.DirSummary){Dir:"/a", Count:15, Size:86, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"adm", "docker", "root"}, FileTypes:[]string{"bam", "cram", "temporary"}}, (*server.DirSummary){Dir:"/a/b", Count:9, Size:80, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"bam", "cram", "temporary"}}, (*server.DirSummary){Dir:"/a/b/d", Count:7, Size:70, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"cram"}}, (*server.DirSummary){Dir:"/a/b/e/h", Count:2, Size:10, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"bam", "temporary"}}, (*server.DirSummary){Dir:"/a/c/d", Count:5, Size:5, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root"}, Groups:[]string{"adm"}, FileTypes:[]string{"cram"}}}'
[149](https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true#step:4:150)
  (Should resemble)!
[150](https://github.com/wtsi-ssg/wrstat/runs/7804713143?check_suite_focus=true#step:4:151)
  Diff:     '[]*server.DirSummary{(*server.DirSummary){Dir:"/a", Count:15, Size:86, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"adm", "docker", "root"}, FileTypes:[]string{"bam", "cram", "temporary"}}, (*server.DirSummary){Dir:"/a/b", Count:9, Size:80, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"bam", "cram", "temporary"}}, (*server.DirSummary){Dir:"/a/b/d", Count:7, Size:70, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root", "runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"cram"}}, (*server.DirSummary){Dir:"/a/b/e/h", Count:2, Size:10, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"runner"}, Groups:[]string{"docker"}, FileTypes:[]string{"bam", "temporary"}}, (*server.DirSummary){Dir:"/a/c/d", Count:5, Size:5, Atime:time.Time{1970-01-01 00:00:50 +0000 UTC}, Users:[]string{"root"}, Groups:[]string{"adm"}, FileTypes:[]string{"cram"}}}'

You will note that the "Expected" and "Actual" strings that it prints out are actually identical.

Why would this be happening, and what can I do about it?

Comparison of proto Messages performed using ShouldResemble is unreliable.

Consider this code:

package main

import (
	. "github.com/smartystreets/goconvey/convey"
	"google.golang.org/protobuf/types/known/wrapperspb"
	"log"
	"testing"
)

func TestConveyBug(t *testing.T) {
	Convey("Given something", t, func() {
		var x = &wrapperspb.BoolValue{Value: false}
		var y = &wrapperspb.BoolValue{Value: false}
		log.Printf("Receive type %v", x)

		So(x, ShouldResemble, y)
	})
}

The call to Printf causes x.String() to be called. This in turn initializes the state and its atomicMessageInfo field on the proto message. Therefore x has the atomicMessageInfo set, while y doesn't. This causes reflect.DeepEqual to return false.

Another problem stems from the rendering of the diff (#660).

Could a ShouldResembleProto be added for expliciting comparing proto messages? It should call proto.Equal.