Elegant generics for Go

genny - Generics for Go

Build Status GoDoc

Install:

go get github.com/cheekybits/genny

=====

(pron. Jenny) by Mat Ryer (@matryer) and Tyler Bunnell (@TylerJBunnell).

Until the Go core team include support for generics in Go, genny is a code-generation generics solution. It allows you write normal buildable and testable Go code which, when processed by the genny gen tool, will replace the generics with specific types.

  • Generic code is valid Go code
  • Generic code compiles and can be tested
  • Use stdin and stdout or specify in and out files
  • Supports Go 1.4's go generate
  • Multiple specific types will generate every permutation
  • Use BUILTINS and NUMBERS wildtype to generate specific code for all built-in (and number) Go types
  • Function names and comments also get updated

Library

We have started building a library of common things, and you can use genny get to generate the specific versions you need.

For example: genny get maps/concurrentmap.go "KeyType=BUILTINS ValueType=BUILTINS" will print out generated code for all types for a concurrent map. Any file in the library may be generated locally in this way using all the same options given to genny gen.

Usage

genny [{flags}] gen "{types}"

gen - generates type specific code from generic code.
get <package/file> - fetch a generic template from the online library and gen it.

{flags}  - (optional) Command line flags (see below)
{types}  - (required) Specific types for each generic type in the source
{types} format:  {generic}={specific}[,another][ {generic2}={specific2}]

Examples:
  Generic=Specific
  Generic1=Specific1 Generic2=Specific2
  Generic1=Specific1,Specific2 Generic2=Specific3,Specific4

Flags:
  -in="": file to parse instead of stdin
  -out="": file to save output to instead of stdout
  -pkg="": package name for generated files
  -tag="": build tag that is stripped from output
  • Comma separated type lists will generate code for each type

Flags

  • -in - specify the input file (rather than using stdin)
  • -out - specify the output file (rather than using stdout)

go generate

To use Go 1.4's go generate capability, insert the following comment in your source code file:

//go:generate genny -in=$GOFILE -out=gen-$GOFILE gen "KeyType=string,int ValueType=string,int"
  • Start the line with //go:generate
  • Use the -in and -out flags to specify the files to work on
  • Use the genny command as usual after the flags

Now, running go generate (in a shell) for the package will cause the generic versions of the files to be generated.

  • The output file will be overwritten, so it's safe to call go generate many times
  • Use $GOFILE to refer to the current file
  • The //go:generate line will be removed from the output

To see a real example of how to use genny with go generate, look in the example/go-generate directory.

How it works

Define your generic types using the special generic.Type placeholder type:

type KeyType generic.Type
type ValueType generic.Type
  • You can use as many as you like
  • Give them meaningful names

Then write the generic code referencing the types as your normally would:

func SetValueTypeForKeyType(key KeyType, value ValueType) { /* ... */ }
  • Generic type names will also be replaced in comments and function names (see Real example below)

Since generic.Type is a real Go type, your code will compile, and you can even write unit tests against your generic code.

Generating specific versions

Pass the file through the genny gen tool with the specific types as the argument:

cat generic.go | genny gen "KeyType=string ValueType=interface{}"

The output will be the complete Go source file with the generic types replaced with the types specified in the arguments.

Real example

Given this generic Go code which compiles and is tested:

package queue

import "github.com/cheekybits/genny/generic"

// NOTE: this is how easy it is to define a generic type
type Something generic.Type

// SomethingQueue is a queue of Somethings.
type SomethingQueue struct {
  items []Something
}

func NewSomethingQueue() *SomethingQueue {
  return &SomethingQueue{items: make([]Something, 0)}
}
func (q *SomethingQueue) Push(item Something) {
  q.items = append(q.items, item)
}
func (q *SomethingQueue) Pop() Something {
  item := q.items[0]
  q.items = q.items[1:]
  return item
}

When genny gen is invoked like this:

cat source.go | genny gen "Something=string"

It outputs:

// This file was automatically generated by genny.
// Any changes will be lost if this file is regenerated.
// see https://github.com/cheekybits/genny

package queue

// StringQueue is a queue of Strings.
type StringQueue struct {
  items []string
}

func NewStringQueue() *StringQueue {
  return &StringQueue{items: make([]string, 0)}
}
func (q *StringQueue) Push(item string) {
  q.items = append(q.items, item)
}
func (q *StringQueue) Pop() string {
  item := q.items[0]
  q.items = q.items[1:]
  return item
}

To get a something for every built-in Go type plus one of your own types, you could run:

cat source.go | genny gen "Something=BUILTINS,*MyType"

More examples

Check out the test code files for more real examples.

Writing test code

Once you have defined a generic type with some code worth testing:

package slice

import (
  "log"
  "reflect"

  "github.com/stretchr/gogen/generic"
)

type MyType generic.Type

func EnsureMyTypeSlice(objectOrSlice interface{}) []MyType {
  log.Printf("%v", reflect.TypeOf(objectOrSlice))
  switch obj := objectOrSlice.(type) {
  case []MyType:
    log.Println("  returning it untouched")
    return obj
  case MyType:
    log.Println("  wrapping in slice")
    return []MyType{obj}
  default:
    panic("ensure slice needs MyType or []MyType")
  }
}

You can treat it like any normal Go type in your test code:

func TestEnsureMyTypeSlice(t *testing.T) {

  myType := new(MyType)
  slice := EnsureMyTypeSlice(myType)
  if assert.NotNil(t, slice) {
    assert.Equal(t, slice[0], myType)
  }

  slice = EnsureMyTypeSlice(slice)
  log.Printf("%#v", slice[0])
  if assert.NotNil(t, slice) {
    assert.Equal(t, slice[0], myType)
  }

}

Understanding what generic.Type is

Because generic.Type is an empty interface type (literally interface{}) every other type will be considered to be a generic.Type if you are switching on the type of an object. Of course, once the specific versions are generated, this issue goes away but it's worth knowing when you are writing your tests against generic code.

Contributions

Comments
  • Remove `// +build ignore` tags from generated output

    Remove `// +build ignore` tags from generated output

    To prevent a template file from being included in the build process a build tag can be added at the top:

    // +build ignore
    
    package foo
    

    Unfortunately, these aren't removed from the generated output. As a result the generated code is also not included in the build.

  • genny leaves around an empty file on failure.

    genny leaves around an empty file on failure.

    if genny fails to generate a code due to not being able to locate the file. it still generates a empty file leading to subsequent calls to fail with the follow error btree.gen.go:1:1: expected 'package', found 'EOF'

    should probably not create the file until we are ready to write its contents.

  • How do I... basetypes?

    How do I... basetypes?

    Hey. I'm having good success with genny but now I have a spazzy-issues.

    Code:

    func (arr *T) pop () BASETYPE { l := len(*arr)-1; val := (*arr)[l]; *arr = (*arr)[:l]; return val }
    

    Question

    One. is there any way to automatically deduce BASETYPE to int from T being []int? I tried:

    func (arr *TSlice) pop () T { l := len(*arr)-1; val := (*arr)[l]; *arr = (*arr)[:l]; return val }
    

    but after running go generate Tslice is named "intSlice" so I cannot export it, which is my next question: Two. I'm exporting my package so custom types have to be Uppercased, so the way I was building slices is no longer work. any suggestions on how to get "intSlice" to "IntSlice" if i generate with "int"? (so I can use it as as the base type)

    edit: NVM I should just use a second var. I'm still kind of curious as to the 2nd question, or if theres a basetype() API for genny though. Cheers and thanks for the software, most simple generator I've seen for Go.

  • genny can not support slice type

    genny can not support slice type

    //go:generate genny -pkg chr -in=../gen/grow.go -out=chargrow.go gen "SliceType=float32,float64,[]int"

    package gen

    import ( "github.com/cheekybits/genny/generic" )

    func GrowSliceType(sli *[]SliceType, n int) { dif := n - cap(*sli) if dif > 0 { *sli = append((*sli)[:cap(*sli)], make([]SliceType, dif)...) } else { *sli = (*sli)[:n] } }

    error Failed to goimports the generated code: ../gen/grow.go:36:10: expected '(', found '[' char.go:1: running "genny": exit status 4

  • Invalid code generated

    Invalid code generated

    If I have something like the following and replace X with int

    type _X_ generic.Type
    
    type Cell_X_ struct {
        Value int
    }
    
    func m(_X_) {}
    
    func example() {
        a := Cell_X_{}
        m(Cell_X_{})
    }
    
    

    The following code which wont compile is generated.

    type CellInt struct {
        Value int
    }
    
    func m(int) {}
    
    func example() {
        a := CellInt{}
        m(Cellint{})
    }
    

    Is this a limitation in how genny works or a bug?

  • Not all instances of the argument type getting rewritten

    Not all instances of the argument type getting rewritten

    So, I decided to take a look at genny, just poking around, implementing a pretty dumb digraph...

    package digraph
    
    import "github.com/cheekybits/genny/generic"
    
    type Node generic.Type
    
    type DigraphNode struct {
        nodes map[Node][]Node
    }
    
    func NewDigraphNode() *DigraphNode {
        return &DigraphNode{
            nodes: make(map[Node][]Node),
        }
    }
    
    func (dig *DigraphNode) Add(n Node) {
        if _, exists := dig.nodes[n]; exists {
            return
        }
    
        dig.nodes[n] = nil
    }
    
    func (dig *DigraphNode) Connect(a, b Node) {
        dig.Add(a)
        dig.Add(b)
    
        dig.nodes[a] = append(dig.nodes[a], b)
    }
    

    But when I run

    $ cat generic.go | genny gen Node=int
    

    I get

    // This file was automatically generated by genny.
    // Any changes will be lost if this file is regenerated.
    // see https://github.com/cheekybits/genny
    
    package digraph
    
    type DigraphInt struct {
        nodes map[int][]Node
    }
    
    func NewDigraphInt() *DigraphInt {
        return &DigraphInt{
            nodes: make(map[int][]Node),
        }
    }
    
    func (dig *DigraphInt) Add(n int) {
        if _, exists := dig.nodes[n]; exists {
            return
        }
    
        dig.nodes[n] = nil
    }
    
    func (dig *DigraphInt) Connect(a, b int) {
        dig.Add(a)
        dig.Add(b)
    
        dig.nodes[a] = append(dig.nodes[a], b)
    }
    

    As you can see, not all instances of the Node type in the code got rewritten.

  • Release 1.0.0

    Release 1.0.0

    The PR is in accordance with (this milestone)[https://github.com/cheekybits/genny/milestone/1] containing changes in pull requests https://github.com/cheekybits/genny/pull/51, https://github.com/cheekybits/genny/pull/50 and some more bug fixes and tests.

  • Fix #49 and #36 - use go scanner for parsing lines and identifiers

    Fix #49 and #36 - use go scanner for parsing lines and identifiers

    I ran into some issues of genny finding identifier boundaries. It seems that others have run into these issues as well:

    • https://github.com/cheekybits/genny/issues/49
    • https://github.com/cheekybits/genny/issues/36

    The parser was updated to use go's golang scanner (go/scanner) to find identifier boundaries instead of splitting on whitespace and checking for characters at the alphanumeric boundaries.

  • Generates specific type with lower case when it should be capitalized

    Generates specific type with lower case when it should be capitalized

    The code generated by genny generates the object name (ObjInt) with the specific type in lower case when it should be capitalized. This occurs when the object name is the parameter of new function. Does not occur if preceded by a space:

    package dsp
    
    import "github.com/cheekybits/genny/generic"
    
    //go:generate genny -in=$GOFILE -out=gen-$GOFILE gen "NumberType=int"
    
    type NumberType generic.Number
    
    type ObjNumberType struct {
    	v NumberType
    }
    
    func NewNumberType() *ObjNumberType {
    
    	// Works
    	//return &{ObjNumberType}
    
    	// Works (there is a space preceding the object name)
    	//return new( ObjNumberType)
    
    	// Does not work -> n := new(Objint) (Lowercase Int)
    	n := new(ObjNumberType)
    	return n
    }
    
  • Some imports being lost on generation

    Some imports being lost on generation

    It appears than non-built-in import statements are being lost on generation. As an example, take a "simpletest.go" file that looks like this:

    package test
    
    import (
            "fmt"
            "github.com/gorilla/mux"
            "github.com/cheekybits/genny/generic"
    )
    
    type MyType generic.Type
    
    func MyTypeFunction() {
            _ := mux.NewRouter()
            fmt.Println("Hello")
    }
    

    if I then run the command cat simpletest.go | genny gen "MyType=ConcreteType" > simpletest_gen.go I get the following output:

    // This file was automatically generated by genny.
    // Any changes will be lost if this file is regenerated.
    // see https://github.com/cheekybits/genny
    
    package test
    
    import "fmt"
    
    func ConcreteTypeFunction() {
        _ := mux.NewRouter()
        fmt.Println("Hello")
    }
    

    You can see the import statement has correctly dropped the genny reference, but also incorrectly removed the mux reference which means the code is now invalid.

    Am I doing something wrong?

  • -pkg doesn't appear to work

    -pkg doesn't appear to work

    The help text and the readme seem to be implying that I can use -pkg to change the package statement at the top of the file, but this doesn't seem to work. I'm using the same template to generate a bunch of API clients for various microservices in different packages, and being able to name the microservice-specific subpackages for the microservices would be nice. At the moment I have to name them all client and alias.

    If -pkg is supposed to do something else, then consider this a potential feature request. It's barely mentioned in the first place, so i'm not discounting me misunderstanding what it's for.

  • Please create a new tag for the latest master branch

    Please create a new tag for the latest master branch

    Using go install to install genny will not include the change in the latest master branch, namely the -tag flag features. Is it possible to create a new tag for the latest commit? I believe the -tag will be helpful in many use cases.

  • import stmt removed in output (sometimes)

    import stmt removed in output (sometimes)

    I have some imports like:

    import (
    	"fmt"
    	"environment/types"
    	"environment/utils"
    )
    

    But those stmt disappeared from output (sometimes) even if the semantics of my code is completely vaild

  • Support for type names with the full package path

    Support for type names with the full package path

    This PR adds support for types that require package imports (i.e. the type will not sit in the same package as the generate code), for example genny gen "ValueType=example.com/a/b.MyType".

    My first PR in this repo, please give feedback as appropriate! In accordance with the readme file, I've read the godoc for the parse package and this PR doesn't break or change any of that, and I've added tests for the new methods.

  • Check if specific type name has a valid syntax

    Check if specific type name has a valid syntax

    Hi!

    Looks like I misunderstood the concepts of genny and passed a specific type with a package name as genny arguments. The problem is that an error message is not pointed to the cause directly it's not trivial to debug. Actually, I resorted to binary search to figure it out.

    Code:

    //go:generate genny -pkg=test -in=../templates/main.go -out=main.go gen "MyType=package.SpecificType"
    

    Error:

    Failed to goimports the generated code: ../templates/main.go:10:12: expected type, found '.' (and 1 more errors)
    main.go:3: running "genny": exit status 4
    
  • running

    running "genny": exec: "genny": executable file not found in $PATH

    //go:generate genny -in=$GOFILE -out=gen-$GOFILE gen "KeyType=string,int ValueType=string,int"

    go generate ./... command with above comment reproduce running "genny": exec: "genny": executable file not found in $PATH messege for me.

    However, if I do that command with //go:generate $GOPATH/bin/genny -in=$GOFILE -out=gen-$GOFILE gen "KeyType=string,int ValueType=string,int" this comment, genny generation works for me well.

    I want to use //go:generate genny -in=$GOFILE -out=gen-$GOFILE gen "KeyType=string,int ValueType=string,int" this comment which is document's recommended comment.

    How can I use this one with the best?

    below is my go env

    GO111MODULE=""
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/Users/jayden/Library/Caches/go-build"
    GOENV="/Users/jayden/Library/Application Support/go/env"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="darwin"
    GOINSECURE=""
    GOMODCACHE="/Users/jayden/go/pkg/mod"
    GONOPROXY=""
    GONOSUMDB=""
    GOOS="darwin"
    GOPATH="/Users/jayden/go"
    GOPRIVATE=""
    GOPROXY="https://proxy.golang.org,direct"
    GOROOT="/usr/local/go"
    GOSUMDB="sum.golang.org"
    GOTMPDIR=""
    GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
    GCCGO="gccgo"
    AR="ar"
    CC="clang"
    CXX="clang++"
    CGO_ENABLED="1"
    GOMOD="/Users/jayden/Desktop/vitalcare-backend/go/go.mod"
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/sn/k8tknb_14qdb9nwxb6qd7cv80000gn/T/go-build483121234=/tmp/go-build -gno-record-gcc-switches -fno-common
    

    Thanks!

Related tags
Aboriginal Generics: the future is here!
Aboriginal Generics: the future is here!

Aboriginal Generics: the future is here! Inspired by this gem of an idea (click the image to go to the original comment): Installation go get github.c

Oct 3, 2022
An implementation of standard generics APIs in Go.

generics This package shows an implementation outlook of proposed generics APIs import "changkun.de/x/generics" Related issues: golang/go#45458 golang

Dec 5, 2022
Go 1.18 Generics based slice package

The missing slice package A Go-generics (Go 1.18) based functional library with no side-effects that adds the following functions to a slice package:

Jan 8, 2023
Advent of Code 2021 solutions using Go 1.18 Generics

advent-of-code-2021 Here are my solutions for Advent of Code 2021. This year, I chose to write my solutions using Go 1.18 with generics (by building t

Dec 18, 2022
Go-generic - A collection of experiments using Go Generics coming out in Go 1.18

Go Generic - experiments with Go 1.18 beta Data structures: iter.Iter[T any] - l

Aug 15, 2022
Generic-list-go - Go container/list but with generics

generic-list-go Go container/list but with generics. The code is based on contai

Dec 7, 2022
Go-generics-simple-doubly-linked-list - A simple doubly linked list implemented using generics (Golang)

Welcome to Go-Generics-Simple-Doubly-Linked-List! Hi, This repository contains a

Jun 30, 2022
Amber is an elegant templating engine for Go Programming Language, inspired from HAML and Jade

amber Notice While Amber is perfectly fine and stable to use, I've been working on a direct Pug.js port for Go. It is somewhat hacky at the moment but

Jan 2, 2023
Elegant Scraper and Crawler Framework for Golang

Colly Lightning Fast and Elegant Scraping Framework for Gophers Colly provides a clean interface to write any kind of crawler/scraper/spider. With Col

Dec 30, 2022
Yaegi is Another Elegant Go Interpreter
Yaegi is Another Elegant Go Interpreter

Yaegi is Another Elegant Go Interpreter. It powers executable Go scripts and plugins, in embedded interpreters or interactive shells, on top of the Go

Dec 30, 2022
🍐 Elegant Golang REST API Framework
🍐 Elegant Golang REST API Framework

An Elegant Golang Web Framework Goyave is a progressive and accessible web application framework focused on REST APIs, aimed at making backend develop

Jan 4, 2023
A modern, fast and scalable websocket framework with elegant API written in Go
A modern, fast and scalable websocket framework with elegant API written in Go

About neffos Neffos is a cross-platform real-time framework with expressive, elegant API written in Go. Neffos takes the pain out of development by ea

Jan 4, 2023
A modern, fast and scalable websocket framework with elegant API written in Go
A modern, fast and scalable websocket framework with elegant API written in Go

About neffos Neffos is a cross-platform real-time framework with expressive, elegant API written in Go. Neffos takes the pain out of development by ea

Dec 29, 2022
Elegant Scraper and Crawler Framework for Golang

Colly Lightning Fast and Elegant Scraping Framework for Gophers Colly provides a clean interface to write any kind of crawler/scraper/spider. With Col

Jan 9, 2023
Yaegi is Another Elegant Go Interpreter
Yaegi is Another Elegant Go Interpreter

Yaegi is Another Elegant Go Interpreter. It powers executable Go scripts and plugins, in embedded interpreters or interactive shells, on top of the Go

Jan 5, 2023
🍐 Elegant Golang REST API Framework
🍐 Elegant Golang REST API Framework

An Elegant Golang Web Framework Goyave is a progressive and accessible web application framework focused on REST APIs, aimed at making backend develop

Jan 9, 2023
🍐 Elegant Golang Web Framework

Goyave Template A template project to get started with the Goyave framework. Getting Started Requirements Go 1.13+ Go modules Running the project Firs

Nov 22, 2022
📖 Elegant changelog generator

English | 中文įŽ€äŊ“ whatchanged An elegant changelog generator. Focus on Elegant/Simple/Efficient/Scalable Feel the magic online Feature: Cross-platform su

Nov 11, 2022
Chore is a elegant and simple tool for executing common tasks on remote servers.
Chore is a elegant and simple tool for executing common tasks on remote servers.

Chore is a tool for executing common tasks you run on your remote servers. You can easily setup tasks for deployment, commands, and more.

May 20, 2022
CasaOS - A simple, easy-to-use, elegant open-source home server system.
CasaOS - A simple, easy-to-use, elegant open-source home server system.

CasaOS - A simple, easy-to-use, elegant open-source home server system. CasaOS is an open-source home server system based on the Docker ecosystem and

Jan 8, 2023