Embed files into a Go executable

statik

Build Status

statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem.

Is this a crazy idea? No, not necessarily. If you're building a tool that has a Web component, you typically want to serve some images, CSS and JavaScript. You like the comfort of distributing a single binary, so you don't want to mess with deploying them elsewhere. If your static files are not large in size and will be browsed by a few people, statik is a solution you are looking for.

Usage

Install the command line tool first.

go get github.com/rakyll/statik

statik is a tiny program that reads a directory and generates a source file that contains its contents. The generated source file registers the directory contents to be used by statik file system.

The command below will walk on the public path and generate a package called statik under the current working directory.

$ statik -src=/path/to/your/project/public

The command below will filter only files on listed extensions.

$ statik -include=*.jpg,*.txt,*.html,*.css,*.js

In your program, all your need to do is to import the generated package, initialize a new statik file system and serve.

import (
  "github.com/rakyll/statik/fs"

  _ "./statik" // TODO: Replace with the absolute import path
)

  // ...

  statikFS, err := fs.New()
  if err != nil {
    log.Fatal(err)
  }
  
  // Serve the contents over HTTP.
  http.Handle("/public/", http.StripPrefix("/public/", http.FileServer(statikFS)))
  http.ListenAndServe(":8080", nil)

Visit http://localhost:8080/public/path/to/file to see your file.

You can also read the content of a single file:

import (
  "github.com/rakyll/statik/fs"

  _ "./statik" // TODO: Replace with the absolute import path
)

  // ...

  statikFS, err := fs.New()
  if err != nil {
    log.Fatal(err)
  }
  
  // Access individual files by their paths.
  r, err := statikFS.Open("/hello.txt")
  if err != nil {
    log.Fatal(err)
  }    
  defer r.Close()
  contents, err := ioutil.ReadAll(r)
  if err != nil {
    log.Fatal(err)
  }

  fmt.Println(string(contents))

There is also a working example under example directory, follow the instructions to build and run it.

Note: The idea and the implementation are hijacked from camlistore. I decided to decouple it from its codebase due to the fact I'm actively in need of a similar solution for many of my projects.

Deterministic output

By default, statik includes the "last modified" (mtime) time on files that it packs. This allows an HTTP FileServer to present the correct file modification times to clients.

However, if you have a continuous integration task that checks that your checked-in static files in a git repository match the code that is generated on your CI system, you'll run into a problem: The mtime on the git checkout does not match what you have locally, causing tests to fail.

You can fix the test in one of two ways:

  1. In CI, manually set the mtime on the freshly checked out tree: here's a stackoverflow answer that provides a shell command to do that; or,
  2. Instruct statik not to store the "last modified" time.

To ignore the last modified time, use the -m to statik, like so:

$ statik -m -include=*.jpg,*.txt,*.html,*.css,*.js

Note that this will cause http.FileServer to consider the file to always have changed & serve it with a "Last-Modified" of the time of the request.

Comments
  • fixed 'invalid cross-device link' problem

    fixed 'invalid cross-device link' problem

    Renaming the generated source file failed for me with the following message: rename /tmp/statik888425530 statik/statik.go: invalid cross-device link

    Try to rename and if that fails copy byte by byte.

  • Readdir should treat zero for count value as it was negative to be fully compatible with the interface

    Readdir should treat zero for count value as it was negative to be fully compatible with the interface

    Hi @rakyll

    Thanks for the statik, it's a great lib:)

    I found an issue in the Readdir implementation while I was using another library that called statik's fs.Readdir(0) and it resulted in a problem. No file was found in this case.

    It should treat the zero value and the negatives the same as the GoDoc recommends: https://golang.org/pkg/os/#File.Readdir

    The problematic Readdir implementation: https://github.com/rakyll/statik/blob/3bac566d30cdbeddef402a80f3d6305860e59f12/fs/fs.go#L190

    I fould the issue when I was using this library: https://github.com/shurcooL/httpfs/issues/8

    All best, @shakahl

  • support positive count in Readdir()

    support positive count in Readdir()

    follow on #60.

    The Readdir() should support positive count in order to return the specified counts of files.

    For it, I defined statikFS.dirs and httpFile.dirIdx for managing the internal state and use it inside Readdir().

  • Multiple static file packages

    Multiple static file packages

    I'd like to produce two (or more) different statik fs's in different import paths. Due to relying on init() and no public API from the package, it's impossible to use both filesystems at the same time; concurrent access is also impossible as fs.Register is package-scoped.

    I'd propose modifying the API:

    fs.New would take zipDatas ...string and fallback on whatever was put in fs.Register (backwards compatible), The generated filesystem package could add Data() string for this purpose in addition to keeping init() behaviour.

  • Reproductible output

    Reproductible output

    Hello,

    Is there any way to have reproductible output from statik?

    As a context, this is needed for Debian. One of the packages I manage (InfluxDB) uses statik and I would like for it to be reproductible.

  • Offical exmple does not work?

    Offical exmple does not work?

    image

    As you see , when i copy the exmple from offical packages, and run go run main.go, access browser just show 404. What is the mistake I got from , please help! Thanks in advance!

    > go env
    set GOARCH=amd64
    set GOBIN=
    set GOCACHE=C:\Users\Vector\AppData\Local\go-build
    set GOEXE=.exe
    set GOFLAGS=
    set GOHOSTARCH=amd64
    set GOHOSTOS=windows
    set GOOS=windows
    set GOPATH=D:\GoProject
    set GOPROXY=
    set GORACE=
    set GOROOT=D:\Dev\go
    set GOTMPDIR=
    set GOTOOLDIR=D:\Dev\go\pkg\tool\windows_amd64
    set GCCGO=gccgo
    set CC=gcc
    set CXX=g++
    set CGO_ENABLED=1
    set GOMOD=
    set CGO_CFLAGS=-g -O2
    set CGO_CPPFLAGS=
    set CGO_CXXFLAGS=-g -O2
    set CGO_FFLAGS=-g -O2
    set CGO_LDFLAGS=-g -O2
    set PKG_CONFIG=pkg-config
    set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Vector\AppData\Local\Temp\go-build291331690=/tmp/go-build -gno-record-
    gcc-switches
    
  • Document determinism

    Document determinism

    This PR does two documentation-y things:

    • Adds a section in the README on how to achieve deterministic output w.r.t. file contents, with a link to a method that restores file modification times.
    • Adjusts the -help text to point out more clearly -m does and that it helps with deterministic output.

    This addresses #99

  •  undefined: fs.RegisterWithNamespace

    undefined: fs.RegisterWithNamespace

    I am trying to build (GOOS=linux GOARCH=amd64 go build app/main.go) but getting this error: app/statik/statik.go:12:2: undefined: fs.RegisterWithNamespace

    code from auto generated statik: fs.RegisterWithNamespace("default", data)

    how to resolve this?

  • Fix Readdir() and Name()

    Fix Readdir() and Name()

    Statik's Readdir did not behave in the same way as a http.FileSystem's Readdir. Similarly, the Name function in http.FileSystem returned the base name, and not a full file path.

    This commit fixes both issues, making both Name and Readdir behave in the same way as when running the raw filesystem directly using http.Dir()

    The commit is an extension of #55, which was a first step towards replicating the correct behaviour, but failed to account for the Name() issue, and did not include the necessary extra Trim to get rid of leading slash.

    The corresponding tests ensure that the results are identical for http.Dir() and statik.

    Closes #58

  • add custom fs loading

    add custom fs loading

    Closes #56 ; it only takes the first parameter for the FS data, but it would be trivial to support them all, I'm just not sure if it would be expected that they would be loaded one over the other.

  • use static files  in html templates

    use static files in html templates

    Hi, i use Statik for css and js files and wondering if it possible to use compiled html files in templates. Like loginfile, _ := statikFS.Open("/tmpl/login.html") tHeader, err := tHeader.ParseFiles(loginfile.data) Thanks.

  • Doc update, mentioning go's native embedding

    Doc update, mentioning go's native embedding

    This has been a very useful tool. But the introduction of //go:embed in Go 1.16 provides roughly the same functionality, but without requiring a go generate pre-compilation step.

    It would be good to add a comment to the beginning of the readme, saying that it is best to use go's native embedding, and to mention any cases which are not handled by go:embed where statik would still be useful.

  • How I replaced this package with go:embed

    How I replaced this package with go:embed

    Hopefully this issue and PR helps those who want to transition from statik to go:embed.

    My motivation?

    2 fold:

    1. This issue - In short, the Walk function in statik did not include files without extension for some reason.
    2. Remove another external dependency

    It wasn't easy, but it's doable. In the end, I managed to get all features working with go:embed

    https://github.com/countertenor/mozart/issues/53

  • Does not read files without an extension?

    Does not read files without an extension?

    For some reason, the Walk function on a directory is ignoring files without an extension within that directory. I'm not sure if those files are omitted from the static phase itself, because the code inside the Walk function should ideally return all files, including ones without extension.

  • When compling on the CI i get an statik error

    When compling on the CI i get an statik error

    I can compile the app (as usual) locally without issues. After switching to to an azure pipeline, i compile the up under golang:1.15-buster

    I do install statik using

    cd /tmp
    go get github.com/rakyll/statik
    

    Then i call statik as expected

    cd /src
    ${GOPATH}/bin/statik -f -src=./public
    

    And this properly generates the build under ../statik/statik.go in my src folder

    Without any issues. But when i call go build (from the same cwd)

    cd /src
    env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags netgo -ldflags "-X github.com/<redacted>/build.Version=${{ variables.version }}" -o dist/<redacted> <redacted>_static.go
    

    It errors with

    statik/statik.go:6:2: cannot find package "." in:
    	/__w/1/s/vendor/github.com/rakyll/statik/fs
    

    I tried quiet some pieces but yet nothing worked.

    My <redacted>_statik.go looks like

    import (
    	"flag"
    	"github.com/<redacted>/bootstrap"
    	"net/http"
    	// will be generated by statik
    	_ "github.com/<redacted>/statik"
    	"github.com/rakyll/statik/fs"
    	"log"
    )
    
    func main() {...}
    

    Helpful for any clues!

The simple and easy way to embed static files into Go binaries.

NOTICE: Please consider migrating your projects to github.com/markbates/pkger. It has an idiomatic API, minimal dependencies, a stronger test suite (t

Dec 25, 2022
A tool to be used with 'go generate' to embed external template files into Go code.

templify A tool to be used with 'go generate' to embed external template files into Go code. Scenario An often used scenario in developing go applicat

Sep 27, 2022
a better customizable tool to embed files in go; also update embedded files remotely without restarting the server

fileb0x What is fileb0x? A better customizable tool to embed files in go. It is an alternative to go-bindata that have better features and organized c

Dec 27, 2022
Using brotli compression to embed static files in Go.
Using brotli compression to embed static files in Go.

?? Broccoli go get -u aletheia.icu/broccoli Broccoli uses brotli compression to embed a virtual file system of static files inside Go executables. A f

Nov 25, 2022
:file_folder: Embeds static resources into go files for single binary compilation + works with http.FileSystem + symlinks

Package statics Package statics embeds static files into your go applications. It provides helper methods and objects to retrieve embeded files and se

Sep 27, 2022
Generates go code to embed resource files into your library or executable

Deprecating Notice go is now going to officially support embedding files. The go command will support //go:embed tags. Go Embed Generates go code to e

Jun 2, 2021
Embed files into a Go executable

statik statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem. Is this a crazy idea? No

Dec 29, 2022
Embed files into a Go executable

statik statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem. Is this a crazy idea? No

Jan 6, 2023
Get an embed.FS from inside an embed.FS
Get an embed.FS from inside an embed.FS

embed.FS wrapper providing additional functionality Features Get an embed.FS from an embedded subdirectory Handy Copy(sourcePath, targetPath) method t

Sep 27, 2022
Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Dec 29, 2022
Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

kubectl-slice: split Kubernetes YAMLs into files kubectl-slice is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into mu

Jan 3, 2023
The simple and easy way to embed static files into Go binaries.

NOTICE: Please consider migrating your projects to github.com/markbates/pkger. It has an idiomatic API, minimal dependencies, a stronger test suite (t

Dec 25, 2022
A tool to be used with 'go generate' to embed external template files into Go code.

templify A tool to be used with 'go generate' to embed external template files into Go code. Scenario An often used scenario in developing go applicat

Sep 27, 2022
a better customizable tool to embed files in go; also update embedded files remotely without restarting the server

fileb0x What is fileb0x? A better customizable tool to embed files in go. It is an alternative to go-bindata that have better features and organized c

Dec 27, 2022
📦 Package Node.js applications into executable binaries 📦

caxa ?? Package Node.js applications into executable binaries ?? Support Recurring support on Patreon: https://patreon.com/leafac One-time support on

Jan 6, 2023
donLoader is a shellcode loader creation tool that uses donut to convert executable payloads into shellcode to evade detection on disk.

donLoader WARNING: This is WIP, barely anything was tested properly. Use at your own risk. Description donLoader is a shellcode loader creation tool t

Sep 20, 2022
Ghostinthepdf - This is a small tool that helps to embed a PostScript file into a PDF

This is a small tool that helps to embed a PostScript file into a PDF in a way that GhostScript will run the PostScript code during the

Dec 20, 2022
This command line converts .html file into .html with images embed.

embed-html This command line converts .html file into .html with images embed. Install > go get github.com/gonejack/embed-html Usage > embed-html *.ht

Oct 6, 2022
Command line tool for adding Windows resources to executable files

go-winres A simple command line tool for embedding usual resources in Windows executables built with Go: A manifest An application icon Version inform

Dec 27, 2022
Using brotli compression to embed static files in Go.
Using brotli compression to embed static files in Go.

?? Broccoli go get -u aletheia.icu/broccoli Broccoli uses brotli compression to embed a virtual file system of static files inside Go executables. A f

Nov 25, 2022