Read a tar file contents using go1.16 io/fs abstraction

go-tarfs

Go Reference GitHub release (latest SemVer) GitHub Workflow Status License Unlicense

Read a tar file contents using go1.16 io/fs abstraction

Usage

⚠️ go-tarfs needs go>=1.16

Install:

go get github.com/nlepage/go-tarfs

Use:

package main

import (
    "os"

    tarfs "github.com/nlepage/go-tarfs"
)

func main() {
    tf, err := os.Open("path/to/archive.tar")
	if err != nil {
		panic(err)
	}
	defer tf.Close()

	tfs, err := tarfs.New(tf)
	if err != nil {
		panic(err)
	}

	f, err := tfs.Open("path/to/some/file")
	if err != nil {
		panic(err)
	}
	// defer f.Close() isn't necessary, it is a noop
    
	// use f...
}

More information at pkg.go.dev/github.com/nlepage/go-tarfs

Caveats

For now, no effort is done to support symbolic links.

Show your support

Give a ⭐️ if this project helped you!

Author

👤 Nicolas Lepage

Contributors

All Contributors

Thanks goes to these wonderful people (emoji key):


Jonas Plum

⚠️ 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

📝 License

This project is unlicensed, it is free and unencumbered software released into the public domain.

Similar Resources

a tool for handling file uploads simple

baraka a tool for handling file uploads for http servers makes it easier to make operations with files from the http request. Contents Install Simple

Nov 30, 2022

Bigfile -- a file transfer system that supports http, rpc and ftp protocol https://bigfile.site

Bigfile -- a file transfer system that supports http, rpc and ftp protocol   https://bigfile.site

Bigfile ———— a file transfer system that supports http, rpc and ftp protocol 简体中文 ∙ English Bigfile is a file transfer system, supports http, ftp and

Dec 31, 2022

Go file operations library chasing GNU APIs.

Go file operations library chasing GNU APIs.

flop flop aims to make copying files easier in Go, and is modeled after GNU cp. Most administrators and engineers interact with GNU utilities every da

Nov 10, 2022

File system event notification library on steroids.

notify Filesystem event notification library on steroids. (under active development) Documentation godoc.org/github.com/rjeczalik/notify Installation

Dec 31, 2022

Pluggable, extensible virtual file system for Go

vfs Package vfs provides a pluggable, extensible, and opinionated set of file system functionality for Go across a number of file system types such as

Jan 3, 2023

An epoll(7)-based file-descriptor multiplexer.

poller Package poller is a file-descriptor multiplexer. Download: go get github.com/npat-efault/poller Package poller is a file-descriptor multiplexer

Sep 25, 2022

Goful is a CUI file manager written in Go.

Goful is a CUI file manager written in Go.

Goful Goful is a CUI file manager written in Go. Works on cross-platform such as gnome-terminal and cmd.exe. Displays multiple windows and workspaces.

Dec 28, 2022

Open Source Continuous File Synchronization

Open Source Continuous File Synchronization

Goals Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals belo

Jan 9, 2023

Cross-platform file system notifications for Go.

File system notifications for Go fsnotify utilizes golang.org/x/sys rather than syscall from the standard library. Ensure you have the latest version

Jan 1, 2023
Comments
  • Add fstest.TestFS

    Add fstest.TestFS

    Nice package, thanks!

    Go offers a function to test fs.FS implementations: https://tip.golang.org/pkg/testing/fstest/#TestFS. I added a test for this and fixed the revealed issues.

    Sorry for not adding an emoji to the commit message 🙈

  • Infinite loop when walking through archive containing . (dot) entry

    Infinite loop when walking through archive containing . (dot) entry

    STEPS

    1. Create an archive:

      mkdir empty_dir
      tar -cf fs_test_infinite_loop.tar -C empty_dir .
      
    2. Walk the archive using fs.WalkDir function.

    EXPECTED: The walk function is called once. OBSERVED: The walk function is in an infinite loop.

  • Feature request: Support tar files missing directory entries

    Feature request: Support tar files missing directory entries

    Some tar files may have file entries with a path inside of a directory (eg dir1/file11), but miss corresponding directory entries (eg have no entry for dir1).

    ping @mihaibuzgau @viovanov I saw both of you had forks to support this, I just released a v1.1.0 which supports this.

  • WIP: hack archive/tar to prevent having to read entries in memory

    WIP: hack archive/tar to prevent having to read entries in memory

    Following the conversation with @mholt at https://github.com/caddyserver/caddy/issues/4945 and https://github.com/mholt/archiver/issues/323.

    To expose a tar as a http.FileSystem, it needs to support Seek. This repo currently reads all the entries in memory (#4): https://github.com/nlepage/go-tarfs/blob/bec76bb13b0a3be9e1bf8bb03141a6d26ce91902/fs.go#L39-L44

    I wanted to try another way and here is what I managed to achieve:

    1. copy the current stdlib archive/tar as a tar package https://github.com/nlepage/go-tarfs/commit/0f59c8705ff4a868f2265cf6f18586535ad37780
    2. add a SectionReader interface and method to expose the underlying file with a Seek method (hack.go file inside the tar package: https://github.com/nlepage/go-tarfs/commit/e77dab3bb990cad5e1dc7d1eb3f6c56fc08a6e9c#diff-57148db3d25f4f7ecf8bf6a8c57b98f408490d0ccf8e625b832db5afd66d165e)
    3. use this method to make a fs.FS ( hack_fs.go)
    4. profit!

    My goal was to make as few modifications to the stdlib, to maybe have it integrated upstream (currently adding 1 interface and 1 method).

    The work is still incomplete:

    • if a Read happens on the archived file before SectionReader is called, the already read bytes will be missing from the SectionReader
    • the *sparseFileReader is not supported

    Let me know what you think!

Vaala archive is a tar archive tool & library optimized for lots of small files.

?? Vaar ?? Vaala archive is a tar archive tool & library optimized for lots of small files. Written in Golang, vaar performs operations in parallel &

Sep 12, 2022
Tarserv serves streaming tar files from filesystem snapshots.

tarserv A collection of tools that allow serving large datasets from local filesystem snapshots. It is meant for serving big amounts of data to shell

Jan 11, 2022
Read csv file from go using tags

go-csv-tag Read csv file from Go using tags The project is in maintenance mode. It is kept compatible with changes in the Go ecosystem but no new feat

Nov 16, 2022
s3fs provides a S3 implementation for Go1.16 filesystem interface.

S3 FileSystem (fs.FS) implementation.Since S3 is a flat structure, s3fs simulates directories by using prefixes and "/" delim. ModTime on directories is always zero value.

Nov 9, 2022
A FileSystem Abstraction System for Go
A FileSystem Abstraction System for Go

A FileSystem Abstraction System for Go Overview Afero is a filesystem framework providing a simple, uniform and universal API interacting with any fil

Jan 9, 2023
Read data from rss, convert in pdf and send to kindle. Amazon automatically convert them in azw3.

Kindle-RSS-PDF-AZW3 The Kindle RSS PDF AZW3 is a personal project. The Kindle RSS PDF AZW3 is a personal project. I received a Kindle for Christmas, a

Jan 10, 2022
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).

中文 English 愿景:为用户提供最简单、可靠、高效的分布式文件系统。 go-fastdfs是一个基于http协议的分布式文件系统,它基于大道至简的设计理念,一切从简设计,使得它的运维及扩展变得更加简单,它具有高性能、高可靠、无中心、免维护等优点。 大家担心的是这么简单的文件系统,靠不靠谱,可不

Jan 8, 2023
QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file
QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file

QueryCSV enable you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to CSV file

Dec 22, 2021
File Processor in Concurrency Pattern using Golang goroutine.
 File Processor in Concurrency Pattern using Golang goroutine.

File Processor in Concurrency Pattern Implement a file processor solution in concurrency pattern using Golang goroutine. Get Started Run docker-compos

Sep 16, 2022
Abstract File Storage

afs - abstract file storage Please refer to CHANGELOG.md if you encounter breaking changes. Motivation Introduction Usage Matchers Content modifiers S

Dec 30, 2022