Shotizam analyzes the size of Go binaries

Shotizam

Shotizam analyzes the size of Go binaries and outputs SQL with size info for analysis in SQLite3.

$ shotizam --sqlite /some/go.binary
SQLite version 3.28.0 2019-04-15 14:49:49
Enter ".help" for usage hints.
sqlite> .width 40
sqlite> .mode column

sqlite> select func, sum(size) from bin where func <> '' group by 1 order by 2 desc limit 20;
unicode.init                              21528
debug/dwarf.init                          13764
fmt.(*pp).printValue                      11941
debug/macho.NewFile                       10017
time.Time.AppendFormat                    9949
runtime.gentraceback                      8278
debug/elf.NewFile                         7631
runtime.selectgo                          6722
encoding/binary.Read                      6555
main.main                                 6476
internal/fmtsort.compare                  6373
time.LoadLocationFromTZData               5795
fmt.(*pp).doPrintf                        5520
runtime.findrunnable                      5287
runtime.typesEqual                        4843
time.nextStdChunk                         4736
time.ParseDuration                        4440
time.loadTzinfoFromZip                    4234
runtime.(*pageAlloc).find                 4109
runtime.heapBitsSetType                   3938

sqlite> select what, sum(size) from bin group by 1;
TODO                                      1370985
fixedheader                               101800
funcdata                                  99656
funcname                                  63062
pcdata0-regmap                            54361
pcdata1-stackmap                          47977
pcdata2-inltree                           40024
pcfile                                    38544
pcln                                      141103
pcsp                                      36478
text                                      896722

sqlite> select pkg, sum(size) from bin where pkg <> '' group by 1 order by 2 desc limit 20;
runtime               621236
reflect               187926
fmt                   61727
time                  58786
strconv               55817
syscall               39481
debug/elf             37070
os                    36707
compress/flate        31462
debug/macho           27117
encoding/binary       26637
os/exec               26599
internal/reflectlite  23687
unicode               22316
sync                  21322
flag                  20871
debug/dwarf           17692
internal/poll         17092
strings               15511
sort                  14992

sqlite> select func, length(func) from bin order by 2 desc limit 5;
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89

sqlite> select func, sum(size) from bin where what = 'pcfile' and func <> '' group by 1 order by 2 desc limit 20;
internal/fmtsort.compare                  268
fmt.(*pp).printValue                      227
encoding/binary.Read                      195
os.Getwd                                  174
runtime.sighandler                        172
runtime.(*pageAlloc).scavengeOne          153
runtime.findrunnable                      148
debug/macho.NewFile                       131
runtime.selectgo                          130
syscall.forkAndExecInChild                129
main.main                                 123
runtime.(*mspan).sweep                    123
internal/fmtsort.nilCompare               119
runtime.growslice                         119
debug/elf.NewFile                         112
fmt.intFromArg                            107
runtime.greyobject                        105
runtime.evacuate_fast64                   103
runtime.mapaccess2_faststr                101
runtime.evacuate                          99

sqlite> select func, sum(size) from bin where what = 'pcln' and func <> '' group by 1 order by 2 desc limit 20;
time.Time.AppendFormat                    1347
runtime.gentraceback                      1132
runtime.selectgo                          1102
fmt.(*pp).printValue                      1027
runtime.findrunnable                      830
runtime.heapBitsSetType                   794
internal/fmtsort.compare                  720
runtime.memmove                           720
debug/macho.NewFile                       692
fmt.(*pp).doPrintf                        638
encoding/binary.Read                      616
runtime.runGCProg                         613
syscall.forkAndExecInChild                552
debug/elf.NewFile                         539
runtime.mallocgc                          532
time.LoadLocationFromTZData               530
runtime.duffcopy                          528
aeshashbody                               518
runtime.(*mspan).sweep                    502
runtime.sighandler                        478

sqlite> select * from bin limit 30;
go.buildid                                            fixedheader  40
go.buildid                                            text         112
go.buildid                                            funcname     11
internal/cpu.Initialize                   internal/c  fixedheader  40
internal/cpu.Initialize                   internal/c  pcsp         13
internal/cpu.Initialize                   internal/c  pcfile       5
internal/cpu.Initialize                   internal/c  pcln         21
internal/cpu.Initialize                   internal/c  pcdata0-reg  22
internal/cpu.Initialize                   internal/c  pcdata1-sta  21
internal/cpu.Initialize                   internal/c  text         80
internal/cpu.Initialize                   internal/c  funcname     24
internal/cpu.processOptions               internal/c  fixedheader  40
internal/cpu.processOptions               internal/c  pcsp         24
internal/cpu.processOptions               internal/c  pcfile       6
internal/cpu.processOptions               internal/c  pcln         167
internal/cpu.processOptions               internal/c  pcdata0-reg  30
internal/cpu.processOptions               internal/c  pcdata1-sta  60
internal/cpu.processOptions               internal/c  text         1792
internal/cpu.processOptions               internal/c  funcname     28
internal/cpu.indexByte                    internal/c  fixedheader  40
internal/cpu.indexByte                    internal/c  pcsp         9
internal/cpu.indexByte                    internal/c  pcfile       5
internal/cpu.indexByte                    internal/c  pcln         21
internal/cpu.indexByte                    internal/c  pcdata0-reg  22
internal/cpu.indexByte                    internal/c  text         64
internal/cpu.indexByte                    internal/c  funcname     23
internal/cpu.doinit                       internal/c  fixedheader  40
internal/cpu.doinit                       internal/c  pcsp         22
internal/cpu.doinit                       internal/c  pcfile       5
internal/cpu.doinit                       internal/c  pcln         159

etc

For fun bugs to make Go smaller, see https://github.com/golang/go/labels/binary-size

Owner
Brad Fitzpatrick
Xoogler. Ex @golang team (2010-2020). Currently making WireGuard easier and more magical @Tailscale.
Brad Fitzpatrick
Comments
  • no __gopclntab section found in ELF file when building with PIE enabled

    no __gopclntab section found in ELF file when building with PIE enabled

    I ran (current HEAD, i.e. c9a147c6aa3d4767214712d5886e8a9703f6085b):

    % shotizam --sqlite blaze-bin/…/testsrv
    no __gopclntab section found in ELF file
    

    When searching for __gopclntab, I only find matches in go/src/cmd/internal/objfile/macho.go. Is this a mac-related thing that I’m missing on Linux?

  • Over-Escaping Quotes?

    Over-Escaping Quotes?

    I ran shotizam against Caddy 2 (v2.0.0-rc.3) executable and got about 18 lines of syntax error from SQLite. This is the first offending line and the line that precedes it:

    INSERT INTO Bin VALUES ("golang.org/x/oauth2/jwt.(*jwtSource).Token", "golang.org/x/oauth2/jwt", "funcname", 43);
    INSERT INTO Bin VALUES ("type..eq.struct { AccessToken string \"json:\\\"access_token\\\"\"; TokenType string \"json:\\\"token_type\\\"\"; IDToken string \"json:\\\"id_token\\\"\"; ExpiresIn int64 \"json:\\\"expires_in\\\"\" }", "", "fixedheader", 40);
    

    SQLite's complaint: Error: near line 159205: near "json": syntax error

  • Add support for 1.16

    Add support for 1.16

    This pretty much just consists of integrating the changes to upstream debug/gosym.

    Not very well tested, but results seem OK, and all 0 unit tests pass!

    Fixes #10

  • Add support for linux

    Add support for linux

    The linux objdump (at least on ubuntu) uses <> instead of _. Example output from objdump -D on ubuntu - 0000000000402bf0 <runtime.c64hash>:

    This way it works for compiled binaries on linux (checked on Ubuntu only) with CGO_ENABLE=1 and CGO_ENABLE=0

  • gosym: PCData comes from pctab, not funcdata

    gosym: PCData comes from pctab, not funcdata

    See LineTable.pcvalue and runtime.pcvalue as reference.

    This fixes values from PCData and occassional crashes if the garbage we were reading before happened to be bad varints.

  • Cannot analyze binaries built with Go 1.16

    Cannot analyze binaries built with Go 1.16

    Trying to analyze a binary built with Go 1.16.2 I get:

    $ shotizam --sqlite daemon/cilium-agent
    not a go1.2+ line table
    

    It looks like this is due to the linker changes in Go 1.16. See e.g.https://golang.org/cl/246497 for some of the corresponding changes to the debug/gosym package.

Shotizam analyzes the size of Go binaries

Shotizam Shotizam analyzes the size of Go binaries and outputs SQL with size info for analysis in SQLite3. $ shotizam --sqlite /some/go.binary SQLite

Nov 27, 2022
Shotizam analyzes the size of Go binaries

Shotizam Shotizam analyzes the size of Go binaries and outputs SQL with size info for analysis in SQLite3. $ shotizam --sqlite /some/go.binary SQLite

Nov 27, 2022
go-to64 analyzes Golang main package to convert int/uint to int64/uint64.

go-to64 About go-to64 analyzes Golang main package to convert int/uint to int64/uint64. This is an experiment tool, so be very careful. In a 32-bit en

Oct 31, 2021
Small tool that analyzes the data of my crappy file format for catalogging things

Things Catalog Analyzer I recently started catalogging all the things I own. I simply wanted to have an overview over my things, think about what I ca

Nov 7, 2021
Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data
Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data

Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data throughout the software development life cycle (SDLC) for engineering teams.

Dec 30, 2022
A prediction program which analyzes given numbers and calculates new values

Guess-It-2 About This is a prediction program which analyzes given numbers and calculates new values. Usage To test the program, download this zip fil

Nov 30, 2021
A tool to analyze and troubleshoot a Go binary size.
A tool to analyze and troubleshoot a Go binary size.

goweight A tool to analyze and troubleshoot a Go binary size. For more, see this blog post ✅ Get a breakdown of all modules inside a binary ✅ Supports

Dec 26, 2022
A simple thread-safe, fixed size LRU written in Go. Based on dominictarr's Hashlru Algorithm. 🔃

go-hashlru A simple thread-safe, fixed size LRU written in Go. Based on dominictarr's Hashlru Algorithm. ?? Uses map[interface{}]interface{} to allow

Dec 5, 2022
A tool to filter URLs by parameter count or size

GoFilter A tool to filter URLs by parameter count or size. This tool requires unique sorted URL list. For example: cat hosts.txt | sort -u > sorted &&

Dec 18, 2022
A Connected Graph Generator tool that construct graphs of some given size

graph graph is a Connected Graph Generator tool that construct graphs of some given size. Notice that it generates all possible connected, undirected

Nov 5, 2021
A lightweight replacement for the standard fmt package, reduces binary size by roughly 400kb in a hello world

console This is a lightweight replacement for the fmt package, reduces the binary size by roughly 400kb in a hello world program. Please note: This pa

Nov 7, 2021
Converts a number to its English counterpart. Uses arbitrary precision; so a number of any size can be converted.

Converts a number to its English counterpart. Uses arbitrary precision; so a number of any size can be converted.

Dec 14, 2021
Removes unnecessarily saved git objects to optimize the size of the .git directory.

Git Repo Cleaner Optimizes the size of the .git directory by removing all of the files that are unnecessarily-still-saved as part of the git history.

Mar 24, 2022
This provides the lru package which implements a fixed-size thread safe LRU cache

golang-lru This provides the lru package which implements a fixed-size thread sa

Dec 22, 2021
Image size analyzer for jpg/png/gif/webp

imgsz Image size analyzer for jpg/png/gif/webp Usage // DecodeSize decodes the dimensions of an image that has // been encoded in a registered format.

Jan 8, 2022
Go-binsize-treemap - Go binary size SVG treemap

?? Go binary size SVG treemap Make treemap breakdown of Go executable binary $ g

Dec 21, 2022
YouTube'da altyazısı olan veya otomatik olarak oluşturulmuş altyazılı videolarda istediğiniz kelimenin hangi saat, dakika ve saniye de geçtiğini size gösterip aradığınız şeyi hızlıca bulmanızı sağlar.

YouTube Subtitles YouTube'da altyazısı olan veya otomatik olarak oluşturulmuş altyazılı videolarda istediğiniz kelimenin hangi saat, dakika ve saniye

Mar 4, 2022
A tool for testing, building, signing, and publishing binaries.

gomason Tool for testing, building, signing and publishing binaries. Think of it as an on premesis CI/CD system- that also performs code signing and p

Dec 15, 2022
The simple and easy way to embed static files into Go binaries.

NOTICE: Please consider migrating your projects to github.com/markbates/pkger. It has an idiomatic API, minimal dependencies, a stronger test suite (t

Dec 25, 2022
The forgotten go tool that executes and caches binaries included in go.mod files.
The forgotten go tool that executes and caches binaries included in go.mod files.

The forgotten go tool that executes and caches binaries included in go.mod files. This makes it easy to version cli tools in your projects such as gol

Sep 27, 2022