structslop is a static analyzer for Go that recommends struct field rearrangements to provide for maximum space/allocation efficiency.

structslop

Build status

Package structslop defines an Analyzer that checks struct can be re-arranged fields to get optimal struct size.

Installation

With Go modules:

go get github.com/orijtech/structslop/cmd/structslop

Without Go modules:

$ cd $GOPATH/src/github.com/orijtech/structslop
$ git checkout v0.0.6
$ go get
$ install ./cmd/structslop

Usage

You can run structslop either on a Go package or Go files, the same way as other Go tools work.

Example:

$ structslop github.com/orijtech/structslop/testdata/src/struct

or:

$ structslop ./testdata/src/struct/p.go

Sample output:

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:30:9: struct has size 24 (size class 32), could be 16 (size class 16), you'll save 50.00% if you rearrange it to:
struct {
	y uint64
	x uint32
	z uint32
}

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:36:9: struct has size 40 (size class 48), could be 24 (size class 32), you'll save 33.33% if you rearrange it to:
struct {
	_  [0]func()
	i1 int
	i2 int
	a3 [3]bool
	b  bool
}

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:59:9: struct has size 40 (size class 48), could be 32 (size class 32), you'll save 33.33% if you rearrange it to:
struct {
	y uint64
	t *httptest.Server
	w uint64
	x uint32
	z uint32
}

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:67:9: struct has size 40 (size class 48), could be 32 (size class 32), you'll save 33.33% if you rearrange it to:
struct {
	y uint64
	t *s
	w uint64
	x uint32
	z uint32
}

Example, for the first report above, the output meaning:

  • The current struct size is 24, the size that the Go runtime will allocate for that struct is 32.
  • The optimal struct size is 16, the size that the Go runtime will allocate for that struct is 16.
  • The layout of optimal struct.
  • The percentage savings with new struct layout.

That said, some structs may have a smaller size, but for efficiency, the Go runtime will allocate them in the same size class, then those structs are not considered sloppy:

type s1 struct {
	x uint32
	y uint64
	z *s
	t uint32
}

and:

type s2 struct {
	y uint64
	z *s
	x uint32
	t uint32
}

have the same size class 32, though s2 layout is only 24 byte in size.

However, you can still get this information when you want, using -verbose flag:

$ structslop -verbose ./testdata/src/verbose/p.go
/go/src/github.com/orijtech/structslop/testdata/src/verbose/p.go:17:8: struct has size 0 (size class 0)
/go/src/github.com/orijtech/structslop/testdata/src/verbose/p.go:19:9: struct has size 1 (size class 8)
/go/src/github.com/orijtech/structslop/testdata/src/verbose/p.go:23:9: struct has size 32 (size class 32), could be 24 (size class 32), optimal fields order:
struct {
	y uint64
	z *s
	x uint32
	t uint32
}

Note

For applying suggested fix, use -apply flag, instead of -fix.

Development

Go 1.15+

Running test

Add test case to testdata/src/struct directory, then run:

go test

Contributing

TODO

Owner
orijtech
Observability, developer tools, cloud, and #golang
orijtech
Comments
  • index out of range crash

    index out of range crash

    When running structslop against pkg/sql of CockroachDB I got the following:

    panic: runtime error: index out of range [11] with length 10
    
    goroutine 10501 [running]:
    github.com/orijtech/structslop.run.func1(0x138cca0, 0xc006541960)
    	/Users/yuzefovich/go/pkg/mod/github.com/orijtech/[email protected]/structslop.go:98 +0xbf2
    golang.org/x/tools/go/ast/inspector.(*Inspector).Preorder(0xc02cc427a0, 0xc004e81b20, 0x1, 0x1, 0xc004f6fb30)
    	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/ast/inspector/inspector.go:77 +0xa2
    github.com/orijtech/structslop.run(0xc02cc400c0, 0xc02cc400c0, 0x0, 0x0, 0x0)
    	/Users/yuzefovich/go/pkg/mod/github.com/orijtech/[email protected]/structslop.go:68 +0x1fa
    golang.org/x/tools/go/analysis/internal/checker.(*action).execOnce(0xc02ec706e0)
    	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:690 +0x87e
    sync.(*Once).doSlow(0xc02ec706e0, 0xc00004f790)
    	/usr/local/opt/go/libexec/src/sync/once.go:66 +0xec
    sync.(*Once).Do(...)
    	/usr/local/opt/go/libexec/src/sync/once.go:57
    golang.org/x/tools/go/analysis/internal/checker.(*action).exec(0xc02ec706e0)
    	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:579 +0x65
    golang.org/x/tools/go/analysis/internal/checker.execAll.func1(0xc02ec706e0)
    	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:567 +0x34
    created by golang.org/x/tools/go/analysis/internal/checker.execAll
    	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:573 +0x125
    
  • Using -fix does not seem to apply fixes

    Using -fix does not seem to apply fixes

    I was testing this on https://github.com/fyne-io/fyne and adding -fix does not seem to do anything. Double checking with git just seems to indicate that no files have been changed.

    image

  • structslop: internal error: package

    structslop: internal error: package "context" without types was imported

    OS: Windows 10 64bit Go version: 1.18.4 structslop v0.0.6 Installed using go get i.e modules My target project mod is using Go version 1.18

    Usage: within target project structslop ./

    Expected outcome: Suggestions or nothing if no suggestions.

    Actual outcome: structslop: internal error: package "context" without types was imported from "docgenapi [docgenapi.test]"

    Maybe i'm reading this wrong perhaps? The message doesn't give any useful information about how to correct my code. Not sure what i'm meant to do from this.

    Any advice appreciated.

  • align values on pointer-size values to avoid false positives that ignore padding

    align values on pointer-size values to avoid false positives that ignore padding

    The current code just performs a packing based off reducing the memories, without accounting for the fact that all struct fields have to be aligned, for example I just added structslop to the Go standard library in a private fork but it is reporting unrealistic sizes that are also odd; there is no way we'll have odd sizes, yet padding exists.

    For an exhibit here is an error message

    ./mbitmap.go:132:15: struct{bytep *uint8; mask uint8; index uintptr}
    has size 24, could be 17, rearrange to struct{bytep *uint8; index uintptr; 
    mask uint8} for optimal size
    

    the error says the value is 24, but could be 17; that's impossible; the size will be 24 on both 32-bit and 64-bit machines given that we pad.

    I believe the remedy for this could be rounding up to the next multiple of the pointer size

  • Output is hard to read

    Output is hard to read

    I ran structslop -fix ./... and got the following output. It does not look too bad when copy-pasted here, but it is impossible to read in the terminal.

    /home/jacob/Git/fyne/internal/driver/glfw/canvas.go:22:15: struct has size 256 (size class 256), could be 240 (size class 240), rearrange to struct{shortcut fyne.ShortcutHandler; size fyne.Size; content fyne.CanvasObject; menu fyne.CanvasObject; context driver.WithContext; painter gl.Painter; onTypedRune func(rune); menuTree *renderCacheTree; overlays *overlayStack; onTypedKey func(*fyne.KeyEvent); onKeyDown func(*fyne.KeyEvent); menuFocusMgr *app.FocusManager; contentFocusMgr *app.FocusManager; contentTree *renderCacheTree; refreshQueue chan fyne.CanvasObject; dirtyMutex *sync.Mutex; onKeyUp func(*fyne.KeyEvent); sync.RWMutex; texScale float32; detectedScale float32; scale float32; dirty bool; padded bool} for optimal size (6.25% savings)
    /home/jacob/Git/fyne/internal/driver/glfw/window.go:47:13: struct has size 400 (size class 416), could be 376 (size class 384), rearrange to struct{pending []func(); clipboard fyne.Clipboard; mousePressed fyne.CanvasObject; mouseLastClick fyne.CanvasObject; mouseOver desktop.Hoverable; mouseDraggedOffset fyne.Position; mouseDragged fyne.Draggable; title string; icon fyne.Resource; mousePos fyne.Position; mouseDragPos fyne.Position; mouseClickCount int; eventQueue chan func(); height int; width int; mainmenu *fyne.MainMenu; canvas *glCanvas; cursor *glfw.Cursor; ypos int; xpos int; mouseButton desktop.MouseButton; onCloseIntercepted func(); onClosed func(); mouseCancelFunc context.CancelFunc; viewport *glfw.Window; viewLock sync.RWMutex; eventLock sync.RWMutex; createLock sync.Once; eventWait sync.WaitGroup; mouseDragStarted bool; visible bool; centered bool; shouldExpand bool; decorate bool; fullScreen bool; fixedSize bool; master bool} for optimal size (7.69% savings)
    /home/jacob/Git/fyne/dialog/fileitem.go:19:21: struct has size 136 (size class 144), could be 128 (size class 128), rearrange to struct{widget.BaseWidget; name string; location fyne.URI; picker *fileDialog; isCurrent bool; dir bool} for optimal size (11.11% savings)
    /home/jacob/Git/fyne/test/testcanvas.go:30:17: struct has size 168 (size class 176), could be 160 (size class 160), rearrange to struct{fyne.ShortcutHandler; size fyne.Size; content fyne.CanvasObject; painter SoftwarePainter; hovered desktop.Hoverable; onTypedRune func(rune); onTypedKey func(*fyne.KeyEvent); overlays *internal.OverlayStack; focusMgr *app.FocusManager; propertyLock sync.RWMutex; scale float32; padded bool} for optimal size (9.09% savings)
    /home/jacob/Git/fyne/internal/driver/gomobile/canvas.go:20:19: struct has size 264 (size class 288), could be 248 (size class 256), rearrange to struct{shortcut fyne.ShortcutHandler; content fyne.CanvasObject; windowHead fyne.CanvasObject; menu fyne.CanvasObject; dragging fyne.Draggable; painter gl.Painter; size fyne.Size; focused fyne.Focusable; touchLastTapped fyne.CanvasObject; onTypedKey func(event *fyne.KeyEvent); onTypedRune func(rune); touched map[int]mobile.Touchable; touchCancelFunc context.CancelFunc; touchTapCount int; lastTapDown map[int]time.Time; lastTapDownPos map[int]fyne.Position; overlays *internal.OverlayStack; refreshQueue chan fyne.CanvasObject; minSizeCache map[fyne.CanvasObject]fyne.Size; scale float32; inited bool; padded bool} for optimal size (11.11% savings)
    

    Impossible to read in terminal: image

    Perhaps a few lines in between would make it more readable? Maybe even better formatting of it?

  • Do not print old struct in report message

    Do not print old struct in report message

    Currently, the report message includes both original struct and optimal struct. This can lead to ugly message, something like:

    struct{x uint32; y uint64; z uint32} has size 24, could be 16, rearrange to struct{y uint64; x uint32; z uint32} for optimal size
    

    It can be better if we can include struct name in report, something like:

    struct X has size 24, could be 16, rearrange to struct{y uint64; x uint32; z uint32} for optimal size
    
  • Fix wrong size/alignment for struct

    Fix wrong size/alignment for struct

    Currently, we use go/types.SizeFor to calculate size/alignment. Unfortunately, go/types chose the most efficient layout model, so it will ignore padding for last field of struct.

    To fix this, we have to implement our own types.Sizes, then plug it in to analysis pass.

    See https://github.com/golang/go/issues/14909#issuecomment-199936232

    Fixes #16

  • Disable -fix for now

    Disable -fix for now

    There're two problem:

    • The comment of struct field is not preserved.
    • The type of struct field is printed in full qualified path, instead of legal Go syntax, e.g "struct{Jump cmd/internal/obj.As; Index int}", should be "struct{Jump obj.As; Index int}"

    Next PR will fix the second problem, while the first is not clear can be fixed at this time or not. To be fair, the suggested fix feature in analysis framework is still experimental, so we should be fine.

  • Struct slop detector rules

    Struct slop detector rules

    There's an existing tool which does report if the struct is not probably aligned https://github.com/mdempsky/maligned

    Things we need to clarify:

    • Is the tool updated?
    • If we can re-use it, is there problem with LICENSE?

    I can discuss with Matthew for those things.

  • add verbose mode option to show struct size info even if it cannot be optimized

    add verbose mode option to show struct size info even if it cannot be optimized

    Thanks for this great tool! I do have an feature request, though.

    To get a feeling of how big a struct is, I'd like to program to be able to print info even if it cannot be optimized. I could not find an option for that.

  • Use runtime actual size class for checking sloppy

    Use runtime actual size class for checking sloppy

    The runtime use diferrent size class to allocate memory. So a struct with size 32 and size 24 are atually consume the same amount of memory, size class 32. So they're not sloppy.

    Instead, we only report if the rearrangement makes actual change in runtime size class.

    Fixes #22

  • why this re-arrange algorithm works?

    why this re-arrange algorithm works?

    https://github.com/orijtech/structslop/blob/13637e228c1a50d8444da1b71a83aff3b6536851/structslop.go#L246-L267 why sort by <align of field, size of field> can minimize the sizeof struct? is there some formal prove?

  • The results for fieldalignment and structslop are not the same. Who is better?

    The results for fieldalignment and structslop are not the same. Who is better?

    package main
    
    type Car struct {
    	flag        bool
    	age         int32
    	F1          int8
    	age2        int32
    	age3        int16
    	F2          int64
    	F3          *int32
    	InnerStruct struct {
    		InnerByte byte
    		//InnerStr string
    	}
    	F4   []byte
    	Name string
    	F5   error
    }
    
    func main() {
    
    }
    

    fieldalignment

    fieldalignment -fix main.go  
    /Users/TiM/Downloads/GolandProjects/helloworld/main.go:3:10: struct of size 104 could be 88
    

    fieldalignment result

    type Car struct {
    	F5          error
    	F3          *int32
    	Name        string
    	F4          []byte
    	F2          int64
    	age         int32
    	age2        int32
    	age3        int16
    	flag        bool
    	InnerStruct struct{ InnerByte byte }
    	F1          int8
    }
    

    structslop

    structslop -apply main.go     
    /Users/TiM/Downloads/GolandProjects/helloworld/main.go:3:10: struct has size 104 (size class 112), could be 88 (size class 96), you'll save 14.29% if you rearrange it to:
    struct {
            F4          []byte
            Name        string
            F5          error
            F3          *int32
            F2          int64
            age         int32
            age2        int32
            age3        int16
            flag        bool
            InnerStruct struct{ InnerByte byte }
            F1          int8
    }
    

    structslop result

    type Car struct {
            F4          []byte
            Name        string
            F5          error
            F3          *int32
            F2          int64
            age         int32
            age2        int32
            age3        int16
            flag        bool
            InnerStruct struct{ InnerByte byte }
            F1          int8
    }
    

    The results for fieldalignment and structslop are not the same

    test the under Car with fieldalignment

    type Car struct {
            F4          []byte
            Name        string
            F5          error
            F3          *int32
            F2          int64
            age         int32
            age2        int32
            age3        int16
            flag        bool
            InnerStruct struct{ InnerByte byte }
            F1          int8
    }
    
    fieldalignment -fix main.go
    /Users/TiM/Downloads/GolandProjects/helloworld/main.go:3:10: struct with 64 pointer bytes could be 48
    
  • Unable to use in windows environment

    Unable to use in windows environment

    When using the proposed installation steps (both cases) the executable produced under the darwin_amd64 folder is (understandably) not valid for windows.

    When I tried building the project myself for windows (GOOS=windows;GOARCH=amd64) and running it I get this error . "The specified executable is not a valid application for this OS platform"

  • README: Install path is deprecated with latest Go

    README: Install path is deprecated with latest Go

    I am getting this (in my home directory, with go version go1.17.2 darwin/amd64):

    % go get github.com/orijtech/structslop/cmd/structslop
    go: downloading github.com/orijtech/structslop v0.0.6
    go: downloading golang.org/x/tools v0.0.0-20200917221617-d56e4e40bc9d
    go: downloading github.com/dave/dst v0.26.2
    go: downloading golang.org/x/mod v0.3.0
    go get: installing executables with 'go get' in module mode is deprecated.
    	Use 'go install pkg@version' instead.
    	For more information, see https://golang.org/doc/go-get-install-deprecation
    	or run 'go help get' or 'go help install'.
    

    With Go 1.17 the new way is:

    % go install github.com/orijtech/structslop/cmd/structslop@latest
    
  • Include bytes saved in the output

    Include bytes saved in the output

    Sometimes, the number of bytes saved is more important to know than a percentage like "15%".

    In this PR, I change the output to include the number of bytes saved so that it looks like this:

    struct has size 200 (size class 208), could be 192 (size class 192), you'll save 7.69% (16 bytes) if you rearrange it to:

Related tags
A static code analyzer for annotated TODO comments
A static code analyzer for annotated TODO comments

todocheck todocheck is a static code analyzer for annotated TODO comments. It let's you create actionable TODOs by annotating them with issues from an

Dec 7, 2022
Analyzer: zapvet is static analysis tool for zap

zapvet zapvet is static analysis tool for zap. fieldtype: fieldtype finds confliction type of field Install You can get zapvet by go install command (

Sep 18, 2022
Go-perfguard - A static analyzer with emphasis on performance

perfguard This tool is a work in progress. It's not production-ready yet. perfgu

Dec 28, 2022
Analyzer: debugcode finds debug codes

debugcode debugcode finds debug codes. builtinprint: finds calling builtin print or println. commentout: finds a commented out debug code without reas

Aug 16, 2021
gostatement is an analyzer checking for occurrence of `go` statements

gostatement gostatement is an analyzer checking for occurrence of go statements. You may want to use a custom func wrapping the statement utilizing re

Nov 4, 2021
This static analysis tool works to ensure your program's data flow does not spill beyond its banks.

Go Flow Levee This static analysis tool works to ensure your program's data flow does not spill beyond its banks. An input program's data flow is expl

Dec 1, 2022
A Golang tool that does static analysis, unit testing, code review and generate code quality report.
A Golang tool that does static analysis, unit testing, code review and generate code quality report.

goreporter A Golang tool that does static analysis, unit testing, code review and generate code quality report. This is a tool that concurrently runs

Jan 8, 2023
GoKart - Go Security Static Analysis
 GoKart - Go Security Static Analysis

GoKart is a static analysis tool for Go that finds vulnerabilities using the SSA (single static assignment) form of Go source code.

Jan 1, 2023
Retnilnil is a static analysis tool to detect `return nil, nil`

retnilnil retnilnil is a static analysis tool for Golang that detects return nil, nil in functions with (*T, error) as the return type. func f() (*T,

Jun 9, 2022
A linter that handles struct tags.

Tagliatelle A linter that handles struct tags. Supported string casing: camel pascal kebab snake goCamel Respects Go's common initialisms (e.g. HttpRe

Dec 15, 2022
Marshal data into commands struct!
Marshal data into commands struct!

Commandarrgh in a nuthsell Commandarrgh is an interface that helps you marshaling data into a command arguments structure. Maybe you have been trying

Dec 18, 2021
:100:Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving

Package validator Package validator implements value validations for structs and individual fields based on tags. It has the following unique features

Jan 1, 2023
πŸ’― Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving

Package validator implements value validations for structs and individual fields based on tags.

Nov 9, 2022
Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer for Nginx.
Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer for Nginx.

Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer, used to analyze Nginx access logs for myself.

Nov 29, 2022
Log-analyzer - Log analyzer with golang

Log Analyzer what do we have here? Objective Installation and Running Applicatio

Jan 27, 2022
β˜„ The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct.
β˜„ The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct.

Gormat - Cross platform gopher tool The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct. δΈ­ζ–‡θ―΄ζ˜Ž Features Data

Dec 20, 2022
Go-archvariant - Go package for determining the maximum compatibility version of the current system

go-archvariant Go package for determining the maximum compatibility version of t

Feb 19, 2022
A Go package providing a generic data type to track maximum and minimum peak values.

go-peak Overview go-peak is a Go package providing a generic data type that tracks the maximum and minimum peak values within a specific period of tim

Mar 26, 2022
An interesting go struct tag expression syntax for field validation, etc.

go-tagexpr An interesting go struct tag expression syntax for field validation, etc. Usage Validator: A powerful validator that supports struct tag ex

Jan 9, 2023