Generate type-safe Go converters by simply defining an interface

goverter

a "type-safe Go converter" generator

Build Status codecov Go Report Card latest release

goverter is a tool for creating type-safe converters. All you have to do is create an interface and execute goverter. The project is meant as alternative to jinzhu/copier that doesn't use reflection.

Features

  • Automatic conversion of builtin types (house example), this includes:
    • slices, maps, named types, primitive types, pointers
    • structs with same fields
  • Extend parts of the conversion with your own implementation: Docs
  • Optional return of an error: Docs
  • Awesome error messages: mismatch type test
  • No reflection in the generated code

Usage

  1. Create a go modules project if you haven't done so already

    $ go mod init module-name
  2. Add goverter as dependency to your project

    $ go get github.com/jmattheis/goverter
  3. Create your converter interface and mark it with a comment containing goverter:converter

    input.go

    package example
    
    // goverter:converter
    type Converter interface {
      Convert(source []Input) []Output
    }
    
    type Input struct {
      Name string
      Age int
    }
    type Output struct {
      Name string
      Age int
    }
  4. Run goverter:

    $ go run github.com/jmattheis/goverter/cmd/goverter module-name-in-full
    # example
    $ go run github.com/jmattheis/goverter/cmd/goverter github.com/jmattheis/goverter/example/simple
    
  5. goverter created a file at ./generated/generated.go, it may look like this:

    package generated
    
    import simple "github.com/jmattheis/goverter/example/simple"
    
    type ConverterImpl struct{}
    
    func (c *ConverterImpl) Convert(source []simple.Input) []simple.Output {
      simpleOutputList := make([]simple.Output, len(source))
      for i := 0; i < len(source); i++ {
        simpleOutputList[i] = c.simpleInputToSimpleOutput(source[i])
      }
      return simpleOutputList
    }
    func (c *ConverterImpl) simpleInputToSimpleOutput(source simple.Input) simple.Output {
      var simpleOutput simple.Output
      simpleOutput.Name = source.Name
      simpleOutput.Age = source.Age
      return simpleOutput
    }

Docs

Rename converter

With goverter:name you can set the name of the generated converter struct.

input.go

// goverter:converter
// goverter:name RenamedConverter
type BadlyNamed interface {
    // .. methods
}

output.go

type RenamedConverter struct {}

func (c *RenamedConverter) ...

Extend with custom implementation

With goverter:extend you can instruct goverter to use an implementation of your own. You can pass multiple function names to goverter:extend, or define the tag multiple times.

See house example sql.NullString

input.go

// goverter:converter
// goverter:extend IntToString
type Converter interface {
    Convert(Input) Output
}
type Input struct {Value int}
type Output struct {Value string}

// You must atleast define a source and target type. Meaning one parameter and one return.
// You can use any type you want, like struct, maps and so on.
func IntToString(i int) string {
    return fmt.Sprint(i)
}

Reuse generated converter

If you need access to the generated converter, you can define it as first parameter.

func IntToString(c Converter, i int) string {
    // c.DoSomething()..
    return fmt.Sprint(i)
}

Errors

Sometimes, custom conversion may fail, in this case goverter allows you to define a second return parameter which must be type error.

// goverter:converter
// goverter:extend IntToString
type Converter interface {
    Convert(Input) (Output, error)
}

type Input struct {Value int}
type Output struct {Value string}

func IntToString(i int) (string, error) {
    if i == 0 {
        return "", errors.New("zero is not allowed")
    }
    return fmt.Sprint(i)
}

Note: If you do this, methods on the interface that'll use this custom implementation, must also return error as second return.

Struct field mapping

With goverter:map you can map fields on a struct that have the same type but different names.

goverter:map takes 2 parameters.

  1. source field name
  2. target field name
// goverter:converter
type Converter interface {
    // goverter:map Name FullName
    Convert(source Input) Output
}

type Input struct {
    Name string
    Age int
}
type Output struct {
    FullName string
    Age int
}

Struct ignore field

With goverter:ignore you can ignore fields on the target struct

goverter:ignore takes multiple field names separated by space .

// goverter:converter
type Converter interface {
    // goverter:ignore Age
    Convert(source Input) Output
}

type Input struct {
    Name string
}
type Output struct {
    Name string
    Age int
}

Versioning

goverter use SemVer for versioning the cli.

License

This project is licensed under the MIT License - see the LICENSE file for details

Logo by MariaLetta

Owner
Jannis Mattheis
Self-taught programmer with great interest in code quality.
Jannis Mattheis
Comments
  • Flatten nested fields

    Flatten nested fields

    Have you read the project readme?

    • [x] Yes, but it does not include related information regarding my question.
    • [ ] Yes, but the steps described do not work.
    • [ ] Yes, but I am having difficulty understanding it and want clarification.

    Describe your question

    If I want to map the following scenario, do I need to go custom or is there a way to do that using goverter:map ? I basically want to flatten the nested Address field at the root of APIPerson.

    // goverter:converter
    type Converter interface {
    	ConvertPerson(source Person) APIPerson
    }
    
    type Person struct {
    	Name    string
    	Address Address
    }
    
    type Address struct {
    	Civic  string
    	Street string
    	City   string
    }
    
    type APIPerson struct {
    	Name   string
    	Civic  string
    	Street string
    	City   string
    }
    
  • feat: goverter:matchIgnoreCase to enable case-insensitive match for field names

    feat: goverter:matchIgnoreCase to enable case-insensitive match for field names

    This PR enables goverter:matchIgnoreCase: case insensitive comparison for field names. Reason: we use goverter to convert database objects from their models to a protobuf (for the database row to be sent out to other systems on write). Both model and protobuf structs are auto-generated (by go-jet and protoc), and there are lots of cases in which the the fields names only differ in casing: for example protoc converts proto name id to golang field Id while go-jet converts equivalent database column "id" as ID. There are many similar cases.

    We want to automate codegen for such conversions with minimal (ideally ZERO) human intervention. To reach that perfection, case-insensitive matching is needed. It is highly unlikely that either of the tools can auto-generate two fields on their own struct that only differ in casing (possible - but it is highly unlikely, hope that devs do not do so because it is simply a bad practice). If such case does happen (e.g. username and user_name => Username and UserName), then devs can solve the ambiguity using goverter:map or goverter:ignore, or avoid adding goverter:matchIgnoreCase.

    I though of limiting this feature to a specific interface only, rather than making it a 'global flag' because of its nature - it should be used "with care". I added code to handle special cases, including testing:

    • exact matches always preferred (even if ambiguity exists)
    • if target has two fields that only differ in casing and goverter:matchIgnoreCase is enabled, and both fields match to a single source - that is OK, source is replicated into both target fields
    • if target field can read from two source fields, and one of them is an exact match: it is OK - exact match is used as a source
    • if target field can read from two source fields, and none of them exact: it is not OK (error reported), devs can fix with explicit map or ignore
    • goverter:map is always case-sensitive (exact match)
    • goverter:ignore excludes field from being selected for case-insensitive match.
  • Converting struct with time.Time fields generate invalid converter

    Converting struct with time.Time fields generate invalid converter

    Describe the bug When generating a converter between two struct with time.Time fields, the generated converter doesn't compile.

    To Reproduce Here is the code to generate the issue

    //go:generate go run github.com/jmattheis/goverter/cmd/goverter github.com/jmattheis/goverter/example/simple
    package simple
    
    import "time"
    
    // goverter:converter
    type Converter interface {
    	Convert(source []Input) []Output
    }
    
    type Input struct {
    	Name     string
    	Birthday time.Time
    	Age      int
    }
    
    type Output struct {
    	Name     string
    	Birthday time.Time
    	Age      int
    }
    
    

    Expected behavior Since the source and destination types are both time.Time, I would expect a simple assignment to be done in the converter, but instead, I have a timeLocationToTimeLocation and timeTimeToTimeTime methods generated and trying to play with private fields of the time type.

  • feat: extend with external packages and name regexp

    feat: extend with external packages and name regexp

    Hey Jannis, thanks for building this wonderful tool!

    We would love to use the goverter in our projects in Outreach, and we already know we need to pass a LOT of extend methods in many places to enable custom conversions. The goal of this PR is to enable consolidation of multiple extend conversion methods in a set of dedicated 'packages', then passing this set as a one-line extend to the goverter by invoking goverter as a lib from inside our own codegen tool (or via goverter's command line - we have not decided yet).

    This PR enables external packages and name regexp usage in the extend statements.

    • extend with method name and external package goverter:extend github.com/google/uuid:FromBytes
    • extend with method name regexp, external package (golang style regexp pattern) goverter:extend strconv:Parse.*
    • extend with method name regexp, local package goverter:extend MyStringTo.*
    • existing extend with name only did not change goverter:extend MyStringToInt
    • update readme with new examples
    • update unit tests to include sunny day and error conditions
    • allow extends to be provided via command line, very useful if goverter is invoked from another codegen, knowing that 99% of covered projects will always need set of conversion methods.
  • feat: enable working directory

    feat: enable working directory

    This PR enables working directory support for all packages.Load call. It is required when running unit tests in isolated directories with their own go.mod files - in this case the packages.Load call must be executed inside that directory.

    This PR preserves the original ParseDocs method signature for back compat, introducing new DocsParser to process the comments.

    This PR also removes the use of go/importer package. It does not support the new go/modules well and both of its Import and ImportFrom methods are marked as deprecated. The golang.org/x/tools/go/packages is supposed to supersede the go/importer package, and it is already heavily used to parse docs and find convert methods, so there is no reason using the go/importer anymore. Moreover, there is no need to re-parse sources ... so I updated code to store the scope on the Converter itself.

  • Add goverter:mapExtend

    Add goverter:mapExtend

    Hi Jannis, here's a start - parsing. Comments?

    I think the name mapExtend is fine, as Extend lines up with the "external" // goverter:extend, and it's a mapping. You could call it mapFunction but then you've got Extend verses Function.

    Also I added in a feature for limiting the tests that are run, I could remove this later if you don't like it. I'd remove the spew module before release.

  • Self import when output is in same package

    Self import when output is in same package

    When overriding the package and output the resulting generated code imports it's own package for custom implementations.

    See example project: https://github.com/hshorter/goverterissue When the generate stage runs, the resulting code contains a self import error:

    // Code generated by github.com/jmattheis/goverter, DO NOT EDIT.
    
    package converter
    
    import converter "github.com/hshorter/goverterissue/pkg/converter"
    
    type DeviceConverterImpl struct{}
    
    func (c *DeviceConverterImpl) ConvertToOutput(source converter.Input) converter.Output {
    	var converterOutput converter.Output
    	converterOutput.Name = source.Name
    	converterOutput.Time = converter.timeToTime(source.Time)
    	return converterOutput
    }
    
  • Goverter:ignore Not Respected When Converter Handles Pointers

    Goverter:ignore Not Respected When Converter Handles Pointers

    Describe the bug The //goverter:ignore comment on the interface method is not respected when the input and output types are pointers. This causes an error to be thrown that the field does not exist on the source when generating a converter for pointer types, even thought the field has been marked as ignored.

    I've been looking at the generated code to try and trace down where the instruction is getting dropped, it looks like the internal converter function that is being generated with the case of a pointer to pointer conversion that is named {package}{T1}To{package}{T2} (e.g. govertertestInPetToGoverterTestOutPet) is what is skipping over the ignore instructions.

    To Reproduce

    package govertertest
    
    type InPet struct {
    	Name        string
    	Description string
    }
    
    type OutPet struct {
    	ID          string
    	Name        string
    	Description string
    }
    
    // This converter generates correctly, with the ID field being ignored
    
    // goverter:converter
    type PetConverter interface {
    	// goverter:ignore ID
    	Convert(in InPet) OutPet
    }
    
    // This converter causes an error to be thrown
    
    // goverter:converter
    type PetPrtConverter interface {
    	// goverter:ignore ID
    	Convert(in *InPet) *OutPet
    }
    

    Expected behavior The ignore field(s) annotation should be respected for pointer conversions the same way that it is handled for regular conversions.

  • slice  goverter:mapIdentity bug

    slice goverter:mapIdentity bug

    Describe the bug A clear and concise description of what the bug is.

    To Reproduce image

    image

    Expected behavior A clear and concise description of what you expected to happen.

  • feat: add PackagePath to prevent loop import of the generated package

    feat: add PackagePath to prevent loop import of the generated package

    This PR prevents loop import of the same generated package inside itself.

    I added a new test case to cover this: 3_struct_extend_in_generated_package.yml. Without the PackagePath param, the test would generate the following import, leading to a loop import of self package and compilation error:

    import (
    	execution "github.com/jmattheis/goverter/execution"
    	// this one is buggy because it imports the generated code's package and golang does not allow this
    	generated "github.com/jmattheis/goverter/execution/generated"
    )
    

    After the fix, the loop import is not generated, see the success in the new test case.

  • missing & in generated code

    missing & in generated code

    Describe the bug Generation doesn't throw any error and successfully generates invalid code.

    To Reproduce

    // goverter:converter
    type Converter interface {
    	// goverter:mapIdentity Address
    	ConvertPerson(source Person) (APIPerson, error)
    
    	// goverter:map Name StreetName
    	ConvertAddress(source Person) (APIAddress, error)
    }
    
    type Person struct {
    	Name   string
    	Street string
    	City   string
    }
    
    type APIPerson struct {
    	Name    string
    	Address *APIAddress
    }
    
    type APIAddress struct {
    	StreetName string
    	City       string
    }
    

    Expected behavior The generated code is missing ampersand sign.

    type ConverterImpl struct{}
    
    func (c *ConverterImpl) ConvertAddress(source proto.Person) (proto.APIAddress, error) {
    	var protoAPIAddress proto.APIAddress
    	protoAPIAddress.StreetName = source.Name
    	protoAPIAddress.City = source.City
    	return protoAPIAddress, nil
    }
    func (c *ConverterImpl) ConvertPerson(source proto.Person) (proto.APIPerson, error) {
    	var protoAPIPerson proto.APIPerson
    	protoAPIPerson.Name = source.Name
    	pProtoAPIAddress, err := c.protoPersonToPProtoAPIAddress(source)
    	if err != nil {
    		return protoAPIPerson, err
    	}
    	protoAPIPerson.Address = pProtoAPIAddress
    	return protoAPIPerson, nil
    }
    func (c *ConverterImpl) protoPersonToPProtoAPIAddress(source proto.Person) (*proto.APIAddress, error) {
    	protoAPIAddress, err := c.ConvertAddress(source)
    	if err != nil {
    		return protoAPIAddress, err
    	}
    	return &protoAPIAddress, nil
    }
    

    protoPersonToPProtoAPIAddress in case of err != nil in returned value & is missing. Btw. why there's doubled P in method's name.

  • Support Nested Goverter Interface Definition

    Support Nested Goverter Interface Definition

    i suggest goverter can support nested interface definition, so that we can assign function into different module. it will be more clear definition.

    type StructA struct {
    	Value  string
    	Target TargetA
    }
    
    type StructB struct {
    	Value  string
    	Target TargetB
    }
    
    type TargetA struct {
    	TA string
    }
    
    type TargetB struct {
    	TB string
    }
    
    // goverter:converter
    type TargetConvertor interface {
    	// goverter:map TA TB
    	TargetAToTargetB(a TargetA) TargetB
    }
    
    // goverter:converter
    type StructConvertor interface {
    	TargetConvertor	
    	StructAToStructB(a StructA) StructB
    }
    
  • interface level option inheritance

    interface level option inheritance

    Let's consider example based on code generated by grpc that adds 3 additional fields state, sizeCache and unknownFields to every generated structure. In case when I've defined interface that will translate multiple structures I have to repeat myself on every structure to ignore those fields and additionally matchIgnoreCase. It would be cool to be able to define ignore and matchIgnoreCase at the level of interface so that every method has them in common defined.

    Instead of:

    type Transformer interface {
     //goverter:matchIgnoreCase
     //goverter:ignore state
     //goverter:ignore sizeCache
     //goverter:ignore unknownFields
     Value(in.Struct1) out.Struct1
     //goverter:matchIgnoreCase
     //goverter:ignore state
     //goverter:ignore sizeCache
     //goverter:ignore unknownFields
     Value(in.Struct2) out.Struct2 
     //goverter:matchIgnoreCase
     //goverter:ignore state
     //goverter:ignore sizeCache
     //goverter:ignore unknownFields
     Value(in.Struct3) out.Struct3
    }
    

    do this:

    //goverter:matchIgnoreCase
    //goverter:ignore state
    //goverter:ignore sizeCache
    //goverter:ignore unknownFields
    type Transformer interface {
     Value(in.Struct1) out.Struct1
     Value(in.Struct2) out.Struct2
     Value(in.Struct3) out.Struct3
    }
    
  • Embedded structs

    Embedded structs

    I have the following scenario:

    package example
    
    // goverter:converter
    type Converter interface {
    	ConvertPerson(source *Person) *APIPerson
    }
    
    type Address struct {
    	Street string
    	City   string
    }
    
    type Person struct {
    	*Address
    
    	Name string
    }
    
    type APIPerson struct {
    	Name   string
    	Street string
    	City   string
    }
    

    How can I use goverter to map from a Person to an APIPerson?

    I am facing two problems:

    1. Mapping pointer values to non-pointer values
    2. Manually listing all embedded fields

    For (1) ideally I would like the target values to either be ignored or set to zero values if the source is nil. For (2) can I somehow map the fields in goverter without having to list them out one by one? I played with mapIdentity but it wasn't clear from the README if this applies to my situation, since it seems like the reverse.

    Thanks in advance for your help.

  • Ignore missing fields by default

    Ignore missing fields by default

    Currently, fields to be ignored must be explicitly specified using a // goverter:ignore FieldName comment.

    I.e.: https://github.com/jmattheis/goverter/blob/edc349e6e6a7f48d692325a952bf4411dcef9470/builder/struct.go#L29

    However, in my case I want to ignore missing fields by default.

    Possible solutions I can think of:

    • A new comment, e.g. // goverter:ignoreAll
    • Allow regexp in the ignore names, e.g. // goverter:ignore To.* (in my case, I could do: // goverter:ignore .*
  • Improve value return on err

    Improve value return on err

    With #18 a compile error was fixed, but not in the most optimal way, because it produces an unnecessary allocation in the error case.

    https://github.com/jmattheis/goverter/blob/5dba9cdc1fa7571c261721ad5a95649657883f27/scenario/3_struct_extend_pattern.yml#L45-L55

    Currently, goverter doesn't know if there is already a variable existing with the right type, maybe there is a smart way to do this?

  • mapping consts

    mapping consts

    Hi Jannis, this is part question, part feature request, part "should I write this?".

    If my target field should be set to a const, how would I do that? For example:

           const ageOfConsent = 18 
       
            type Input struct {
                Name string
            }
    
            type Output struct {
                Name2 string
                Age int
            }
    

    Desired field mappings:

    Input.Name -> OutPut.Name2
    ageOfConsent -> Output.Age
    

    I was wondering if I should attempt to write this (as a feature), but I'd be interested in your ideas. Here's a possible syntax:

            // goverter:converter
            type Converter interface {
                // goverter:const ageOfConsent:Age
                Convert(source Input) Output
            }
    
safe and easy casting from one type to another in Go

cast Easy and safe casting from one type to another in Go Don’t Panic! ... Cast What is Cast? Cast is a library to convert between different go types

Sep 26, 2022
Type-safe atomic values for Go

Type-safe atomic values for Go One issue with Go's sync/atomic package is that there is no guarantee from the type system that operations on an intege

Apr 8, 2022
[TOOL, CLI] - Filter and examine Go type structures, interfaces and their transitive dependencies and relationships. Export structural types as TypeScript value object or bare type representations.

typex Examine Go types and their transitive dependencies. Export results as TypeScript value objects (or types) declaration. Installation go get -u gi

Jul 5, 2022
CapMonsterTool is a set of Go tools designed to simply make requests to the CapMonster Cloud API.

✨ CapMonsterTool ✨ About this module What is ✨ CapMonsterTool ✨ ? CapMonsterTool is a set of Go tools designed to simply make requests to the CapMonst

Jun 29, 2022
A tool to generate Pulumi Package schemas from Go type definitions

MkSchema A tool to generate Pulumi Package schemas from Go type definitions. This tool translates annotated Go files into Pulumi component schema meta

Sep 1, 2022
Toy program for benchmarking safe and unsafe ways of saving a file

save-a-file benchmarks the many strategies an editor could use to save a file. Example output on a SSD: ext4: $ ./save-a-file ~/tmp/foo 29.195µs per s

Sep 13, 2021
Start of a project that would let people stay informed about safe running spaces in their area.

SafeRun Start of a project that would let people stay informed about safe running spaces in their area. Too many people I'm friends with feel unsafe w

Feb 11, 2022
Analyze the binary outputted by `go build` to get type information etc.

Analyze the binary outputted by go build to get type information etc.

May 20, 2022
IBus Engine for GoVarnam. An easy way to type Indian languages on GNU/Linux systems.

IBus Engine For GoVarnam An easy way to type Indian languages on GNU/Linux systems. goibus - golang implementation of libibus Thanks to sarim and haun

Feb 10, 2022
Lithia is an experimental functional programming language with an implicit but strong and dynamic type system.

Lithia is an experimental functional programming language with an implicit but strong and dynamic type system. Lithia is designed around a few core concepts in mind all language features contribute to.

Jun 14, 2022
The gofinder program is an acme user interface to search through Go projects.

The gofinder program is an acme user interface to search through Go projects.

Jun 14, 2021
Simple interface to libmagic for Go Programming Language

File Magic in Go Introduction Provides simple interface to libmagic for Go Programming Language. Table of Contents Contributing Versioning Author Copy

Dec 22, 2021
Go language interface to the PAPI performance API

go-papi Description go-papi provides a Go interface to PAPI, the Performance Application Programming Interface. PAPI provides convenient access to har

Aug 19, 2022
wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion
wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion

wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion. Implements wkhtmltopdf Go bindings. It can be used to convert HTML docume

Sep 15, 2022
A go interface to NotifyMyAndroid

Notify My Android on the Go This is a go client for Notify my Android. With this, you can send simple notifications directly to your phone and other a

Aug 13, 2019
Third party extension interface for sillyGirl.
Third party extension interface for sillyGirl.

Third party extension interface for sillyGirl.

Jan 11, 2022
Simple example program using CRUD operations to interface with azcosmos

Simple example program using CRUD operations to interface with azcosmos

Nov 15, 2021
Data interface for salesforce price bulk get

data-interface-for-salesforce-price-bulk-get 概要 data-interface-for-salesforce-price-bulk-get は、salesforce の価格オブジェクト取得に必要なデータの整形、および作成時に salesforce から返

Nov 27, 2021
Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Search running process for a given dll/function. Exposes a bufio.Scanner-like interface for walking a process' PEB

Apr 21, 2022