gogiven - BDD testing framework for go that generates readable output directly from source code

gogiven

An alternative BDD spec framework for go. Builds on "go test" tool and builds on the go testing package.

Build status Go Report Card GoDoc Coverage Status

Inspired by YATSPEC. Another similar idea is JGiven, although both are Java based.

Check out the HTML output generator used as a default here: htmlspec - using go's own template/html package.

Feel free to contact me and help improve the code base or let me know if you have any issues or questions!

To contribute, please read this first.

Table of Contents

  1. Introduction
  2. Example One - GoGivens in practice
  3. Example Two - Table Tests
  4. Content Generation
  5. List of pre-written output generators

Introduction

Go Givens is a lightweight BDD framework for producing test specifications directly from the code you write.

Go Givens parses your test file and produces a human-readable output in a specified directory, containing all the tests, captured data and other related information regarding your test such as success or failure.

Go Givens was inspired by YATSPEC, a BDD framework employed extensively by Sky Network Services (part of Sky, a UK tv company). As mentioned above, another similar product is JGiven.

Why?

Capturing your test method as test output is the only real way to show it's intention. You can refactor a test, and have the output update accordingly when the test runs. Unlike other go BDD frameworks, you can use function names to declare intent, and refactoring the function will affect the test. E.g.

//Hello, spec world!
func TestMyTest(...){
	Given(testing, someData()). // This is a test
	When(fooBarBaz()).
	Then(baz())
}

.. will be rendered as (but in html, or json, or whatever):

My Test
Hello, spec world!

Specification

Given testing some data noting that this is a test
When foo bar baz
Then baz 

Test data (set by the func someData in the example above) can be captured in a map of interesting givens, and as the test progresses, the actuals can be captured in a map of captured inputs and outputs.

Interesting whats?

Interesting givens are data points that you want to use in your tests that are important for it to function correctly. "Given" data is then used by your application to fulfil some sort of request or process.

Interesting givens are generated as output along side your test output in a table so interested parties can examine them.

Captured Whos??

Captured inputs and outputs are data points that are registered by either your system under test (stubs, mocks etc) or the output from your system its self.

Captured inputs and outputs are logged along side your test, for each test, so that interested parties can view them.

That's all great, but still.. WHY would I want to log this stuff??

In BDD, a system's inputs and outputs are important to the business. Capturing the relationships between your data and the way the system handles can be demonstrated to your client. For example, your system could call a 3rd party, and you might want to model the interaction with stubs.

GoGivens gives you a standardised way of rendering captured data alongside your tests so you don't have to worry about it.

Rendered how?

The test framework parses your test file, and grabs the content. It strips all non-interesting parts out and leaves the Given/When/Then format in plain text ready for a GoGivensOutputGenerator to process the text. Interesting givens and Captured inputs and outputs are maps, which are rendered alongside your test givens as table data -- interesting givens are tablulated, and captured IO is listed.

A complete example of how to write a GoGivensOutputGenerator is given in the sister project html spec - written in Go.

Example One - GoGivens in Practice

import (
	"github.com/corbym/gocrest/has"
	"github.com/corbym/gocrest/then"
	"github.com/corbym/gogiven/base"
	"github.com/corbym/gogiven/testdata"
	"testing"
	"github.com/corbym/gocrest/is"
)

func TestMain(testmain *testing.M) {
	runOutput := testmain.Run()
	GenerateTestOutput() // You only need test main GenerateTestOutput() if you want to produce HTML output.
	os.Exit(runOutput)
}

func TestMyFirst(testing *testing.T) {
	Given(testing, someDataSetup).

		When(somethingHappens).

		Then(func(testing base.TestingT, actual testdata.CapturedIO, givens testdata.InterestingGivens) { // passed in testing should be used for assertions
		//do assertions
		then.AssertThat(testing, actual["actual"], is.EqualTo("some output"))
	})
}

Note you do not have to use "gocrest" assertions, you can still call all of testing.T's functions to fail the test or you can use any go testing assertion package compatible with testing.T.

When run, the above will produce an HTML output:

Example Html

Example Two - Table Tests

Table tests work the same way as normal go table tests. GoGivens will then mark which test failed, if they do, in your test output.

Example:

...
func TestMyFirst(testing *testing.T){
   var someRange = []struct {
		actual   string
		expected int
	}{
		{actual: "", expected: 0},
		{actual: "a", expected: 2},
	}
	for _, test := range someRange {
	   tst.Run(test.actual, func(weAreTesting *testing.T) {
	   	Given(weAreTesting, someDataSetup).
			When(someAction).
			Then(func(t TestingT, actual CapturedIO, givens InterestingGivens) {
			//do assertions
		AssertThat(t, actual.CapturedIO["actual"], is.EqualTo("some output"))
	   	})
	   }	
	}
}
...

The above test will still fail the test function as far as Go is concerned, but the test output will note that the iteration failed like this:

Ranged Example Html

Note that comments are now rendered. Test function comments appear as part of the spec, and inline comments appear as "Noting that ..". In the above, the comment //do assertions would become "Noting that do assertions".

More Examples

Content Generation

Gogivens comes defaultly configured with an html generator (htmlspec.NewTestOutputGenerator) that is consumed by a file generator (generator.FileOutputGenerator) (see the godoc for more information). The content generator implements the following interface:

type GoGivensOutputGenerator interface {
	Generate(data *PageData) (output io.Reader)
	//ContentType is text/html, application/json or other mime type
	ContentType() string
}

The generated content (output io.Reader) is then consumed by an OutputListener:

type OutputListener interface {
	Notify(testFilePath string, contentType string, output io.Reader)
}

If you want your own output listener just create your own and replace and/or append to the default listeners in your TestMain:

GoGiven's html spec now generates an index file, which is always stored along side the test html output.

func TestMain(testmain *testing.M) {
	gogiven.OutputListeners = []generator.OutputListener{new(MyFooListener)}
	// or alternately (or inclusively!)
	gogiven.OutputListeners = append(OutputListeners, new(MyBarListener))
	runOutput := testmain.Run()
	gogiven.GenerateTestOutput() // generates the output after the test is finished.
	os.Exit(runOutput)
}

Setting the test file output (for the generator.FileOutputGenerator)

You can add the environment variable GOGIVENS_OUTPUT_DIR to your env properties that points to a directory you want goGivens to report the test output to.

Default is the os's tmp directory.

List of Pre-written Ouput Generators

GoGiven comes with the following output generators:

Owner
Matt
Java Dev, 20 years industry experience. Software Engineer at eBay
Matt
Comments
  • Render test comments in GivenWhenThen output

    Render test comments in GivenWhenThen output

    Expected Behavior

    ...
    //This comment should appear as notes next to the test output.
    func TestMyTest(t *testing.T){
        Given(someFoo).
         When(somethingHappens).
         Then(...){
         //assertions go here
         }
    }
    

    should display as:

    Specification
    This comment should appear as notes next to the test output
            Given some foo
            When something happens
            Then
            noting that assertions go here
    

    Actual Behavior

    The test comment is not rendered.

    Steps to Reproduce the Problem

    1. This is an enhancement.

    Specifications

    • Go Version:
    • Platform:
  • Create index file of generated output.

    Create index file of generated output.

    As a user, I want to be able to view an index of test data generated by the framework, so that I can find a test easily.

    I especially want to be able to easily navigate to failed tests, and want them indexed in a way that preserves information about the package the test was in.

  • CSS stylesheet update

    CSS stylesheet update

    Expected Behavior

    Output should look and feel like godoc.

    Actual Behavior

    Looks old and in need of updating.

    Steps to Reproduce the Problem

    1. This is an enhancement.

    Specifications

    • Go Version: n/a
    • Platform: n/a
  • Given/When/Then inlined variables

    Given/When/Then inlined variables

    As a developer, I want to be able to inline variables I pass to my functions in a formatted way, so that my tests are more understandable.

    Example:

    func TestMyStuff(testing testing.T){
        Given(iAmMaking_CupsOf_Tea(3, "Earl Grey")).
             When(iPour_MillilitersOfWater("250")).
             Then(func(...){
             // some assertions
             }
    
    }
    

    .. should render as:

    Given I am making 3 cups of Earl Grey Tea
    When I pour 250 milliliters of water
    Then
        // some assertions 
    
  • Better test fluidity

    Better test fluidity

    As a developer, I would prefer not to have to chain the given/when/then functions so that I can space the code out better.

    Example:

    Given(testing, clockParametersUnder(test)).
    			When(func(capturedIO testdata.CapturedIO, givens testdata.InterestingGivens) {
    			clock, err = berlinclock.Clock(test.time)
    		}).
    			Then(func(testingT base.TestingT, actual testdata.CapturedIO, givens testdata.InterestingGivens) {
    			then.AssertThat(testing, err, is.Nil())
    			then.AssertThat(testing, clock, is.
    				EqualTo(test.expected).
    				Reasonf("time incorrect for %s", test.time))
    		})
    

    should be more like this:

    func TestMyTest(testing *testing.T) {
        Given(testing, clockParametersUnder(test))
    
        When(func(capturedIO testdata.CapturedIO, givens testdata.InterestingGivens) {
    			clock, err = berlinclock.Clock(test.time)
    		})
    
        Then(func(testingT base.TestingT, actual testdata.CapturedIO, givens testdata.InterestingGivens) {
    			then.AssertThat(testing, err, is.Nil())
    			then.AssertThat(testing, clock, is.
    				EqualTo(test.expected).
    				Reasonf("time incorrect for %s", test.time))
        })
    }
    
    
  • Listener hook

    Listener hook

    As a developer, I want to be able to get information from my currently running test state, so that I have up to the minute information about the build.

    Implementation detail: the test should be able to report to some channel or feed the current state of the test.

A helper tool for getting OpenShift/Kubernetes data directly from Etcd.

Etcd helper A helper tool for getting OpenShift/Kubernetes data directly from Etcd. How to build $ go build . Basic Usage This requires setting the f

Dec 10, 2021
Terraform-in-Terraform: Execute Modules directly from the Terraform Registry

Terraform-In-Terraform Provider This provider allows running Terraform in Terraform. This might seem insane but there are some edge cases where it com

Dec 25, 2022
Bitrise step to parse a JaCoCo generated report and output the code coverage percentages to be used by other steps.

JaCoCo Report Parser This step parses a JaCoCo generated XML report in the jacoco_report_path and outputs the coverage percentages in a String format

Dec 6, 2021
Stuff to make standing up sigstore (esp. for testing) easier for e2e/integration testing.
Stuff to make standing up sigstore (esp. for testing) easier for e2e/integration testing.

sigstore-scaffolding This repository contains scaffolding to make standing up a full sigstore stack easier and automatable. Our focus is on running on

Dec 27, 2022
Boxygen is a container as code framework that allows you to build container images from code

Boxygen is a container as code framework that allows you to build container images from code, allowing integration of container image builds into other tooling such as servers or CLI tooling.

Dec 13, 2021
KubeOrbit is an open-source abstraction layer library that turns easy apps testing&debuging on Kubernetes in a new way
KubeOrbit is an open-source abstraction layer library that turns easy apps testing&debuging on Kubernetes in a new way

KubeOrbit is an open-source abstraction layer library that turns easy apps testing&debuging on Kubernetes in a new way

Jan 6, 2023
dfg - Generates dockerfiles based on various input channels.

dfg - Dockerfile Generator dfg is both a go library and an executable that produces valid Dockerfiles using various input channels. Table of Contents

Dec 23, 2022
cluster-api-state-metrics (CASM) is a service that listens to the Kubernetes API server and generates metrics about the state of custom resource objects related of Kubernetes Cluster API.

Overview cluster-api-state-metrics (CASM) is a service that listens to the Kubernetes API server and generates metrics about the state of custom resou

Oct 27, 2022
Schematic - Generates model and validators by schema definition

schematic Generates model and validators by schema definition. Install Warning:

Feb 10, 2022
debiman generates a static manpage HTML repository out of a Debian archive

debiman Goals debiman makes (Debian) manpages accessible in a web browser. Its goals are, in order: completeness: all manpages in Debian should be ava

Jan 4, 2023
An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers
An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers

Developer-oriented Continuous Delivery Product ⁣ English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? How to

Oct 19, 2021
the simplest testing framework for Kubernetes controller.

KET(Kind E2e Test framework) KET is the simplest testing framework for Kubernetes controller. KET is available as open source software, and we look fo

Dec 10, 2022
Golang Integration Testing Framework For Kong Kubernetes APIs and Controllers.
Golang Integration Testing Framework For Kong Kubernetes APIs and Controllers.

Kong Kubernetes Testing Framework (KTF) Testing framework used by the Kong Kubernetes Team for the Kong Kubernetes Ingress Controller (KIC). Requireme

Dec 20, 2022
Open Source runtime tool which help to detect malware code execution and run time mis-configuration change on a kubernetes cluster
Open Source runtime tool which help to detect malware code execution and run time mis-configuration change on a kubernetes cluster

Kube-Knark Project Trace your kubernetes runtime !! Kube-Knark is an open source tracer uses pcap & ebpf technology to perform runtime tracing on a de

Sep 19, 2022
Source code and slides for Kubernetes Community Days - Bangalore.
Source code and slides for Kubernetes Community Days - Bangalore.

kcdctl This is the source code for the demo done as part of the talk "Imperative, Declarative and Kubernetes" at the Kubernetes Community Days, Bengal

Sep 19, 2021
io-hat go board source code

io-hat go Kaip veikia Kaip paruošti Raspberry Pi Šaltiniai io-hat go Kaip veikia Įėjimų statusas publikuojamas periodiškai ir kai pasikeičia reikšmė s

Dec 24, 2021
BuildKit - A toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner
BuildKit - A toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner

BuildKit BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner. Key features: Automati

Feb 19, 2022
K8s - A Collection of tools, hands-on walkthroughs with source code
K8s - A Collection of tools, hands-on walkthroughs with source code

The Ultimate Engineer Toolbox ?? ?? A Collection of tools, hands-on walkthroughs

Feb 14, 2022