Go bindings for unarr (decompression library for RAR, TAR, ZIP and 7z archives)

go-unarr

TravisCI Build Status AppVeyor Build Status GoDoc Go Report Card

Golang bindings for the unarr library from sumatrapdf.

unarr is a decompression library and CLI for RAR, TAR, ZIP and 7z archives.

GoDoc

See https://pkg.go.dev/github.com/gen2brain/go-unarr

Install CLI

go get github.com/gen2brain/go-unarr/cmd/unarr

Example

unarr ./example.7z ./example/

Build

For one-off builds:

go build -o ./unarr ./cmd/unarr/*.go

For multi-platform cross-compile builds:

goreleaser --snapshot --skip-publish --rm-dist

See build.sh for gcc toolchain information and for manual cross-platform build instructions.

Library Examples

Install Library

go get -v github.com/gen2brain/go-unarr

Open archive

a, err := unarr.NewArchive("test.7z")
if err != nil {
    panic(err)
}
defer a.Close()

Read first entry from archive

err := a.Entry()
if err != nil {
    panic(err)
}

data, err := a.ReadAll()
if err != nil {
    panic(err)
}

List contents of archive

list, err := a.List()
if err != nil {
    panic(err)
}

Read known filename from archive

err := a.EntryFor("filename.txt")
if err != nil {
    panic(err)
}

data, err := a.ReadAll()
if err != nil {
    panic(err)
}

Read first 8 bytes of the entry

err := a.Entry()
if err != nil {
    panic(err)
}

data := make([]byte, 8)

n, err := a.Read(data)
if err != nil {
    panic(err)
}

Read all entries from archive

for {
    err := a.Entry()
    if err != nil {
        if err == io.EOF {
            break
        } else {
            panic(err)
        }
    }

    data, err := a.ReadAll()
    if err != nil {
        panic(err)
    }
}

Extract contents of archive to destination path

err := a.Extract("/tmp/path")
if err != nil {
    panic(err)
}
Owner
Milan Nikolic
SysAdmin / Developer
Milan Nikolic
Comments
  • Bump version and add releases to github releases

    Bump version and add releases to github releases

    Two things:

    1. Cross-Platform Releases Now Possible

    You can now cross-build Github releases from MacOS using goreleaser (as described in the latest PR).

    If you get a Github Access Token you can publish versioned builds directly to github:

    git tag -a v1.0.0 -m "First CLI release"
    git push origin v1.0.0
    
    export GITHUB_TOKEN=xxxxxx
    goreleaser
    

    2. Binary name change

    I'd like to add this to https://webinstall.dev as a cross-platform method of un7z.

    There are already many, many, many easily available cross-platform tools, such as tar, unzip, and arc, for the "normal" package and compression formats.

    There are ZERO easy-to-install cross-platform CLI tools for un7z.

    1. Would it be alright if I created a duplicate cmd/un7z/un7z.go (or replaced the existing cmd/un7z) for the purposes of making it easier to identify as the "correct" solution that people are looking for?
    2. Would you be willing (and able) to publish cross-platform binaries for un7z to github releases with goreleaser?
  • macOS:#include

    macOS:#include "external/unarr/_7z/_7z.c"

    GOROOT=/usr/local/Cellar/[email protected]/1.18.6/libexec #gosetup
    GOPATH=/Users/51pwn/go #gosetup
    /usr/local/Cellar/[email protected]/1.18.6/libexec/bin/go build -o /private/var/folders/_l/pnb2t_9s0f192bqlz1348vpr0000gn/T/GoLand/___go_build_github_com_hktalent_51pwnPlatform_pkg_blevExp_sky -gcflags all=-N -l github.com/hktalent/51pwnPlatform/pkg/blevExp/sky #gosetup
    # github.com/gen2brain/go-unarr/unarrc
    ../../../vendor/github.com/gen2brain/go-unarr/unarrc/7z_7z.go:4:10: fatal error: 'external/unarr/_7z/_7z.c' file not found
    #include "external/unarr/_7z/_7z.c"
             ^~~~~~~~~~~~~~~~~~~~~~~~~~
    1 error generated.
    
  • no entries in 7z archive

    no entries in 7z archive

    Go version: 1.11 Platform: Windows 10

    go-unarr installed with the following command:

    go get github.com/gen2brain/go-unarr
    

    Trying to extract data from a .7z archive. The archive contains exactly one file. However, go-unarr can't find any entries, e.g. a.List() returns an empty slice and a.Entry() returns io.EOF

    My code:

    package main
    
    import (
        "github.com/gen2brain/go-unarr"
        "fmt"
        "io"
    )
    
    func main() {
    	a, err := unarr.NewArchive("something.7z")
    	if err != nil {
    		panic(err)
    	}
    
    	defer a.Close()
    
    	err = a.Entry()
    	if err != nil {
    		panic(err)
    	}
    
    	list, err := a.List()
    	if err != nil {
    		panic(err)
    	}
    	fmt.Println(list)
    
    	err = a.Entry()
    	if err != nil {
    		if err == io.EOF {
    			fmt.Println("No files found")
    			return
    		}
    		panic(err)
    	}
    
    	data, err := a.ReadAll()
    	fmt.Println(len(data))
    }
    

    This prints

    []
    No files found
    

    But should print

    [something.txt]
    16
    

    Base64 encoded contents of something.7z:

    N3q8ryccAASTCs7jFAAAAAAAAABiAAAAAAAAAF5649QBAA/vu79oZWxsbywgd29ybGQhAAEEBgABCRQABwsBAAEhIQEADBAACAoBuW/kcgAABQEZDAAAAAAAAAAAAAAAABEdAHMAbwBtAGUAdABoAGkAbgBnAC4AdAB4AHQAAAAUCgEAL22u3BFM1AEVBgEAIAAAAAAA
    
  • go-unarr produces corrupt files

    go-unarr produces corrupt files

    wonko@deepthought:~/unarr$ ls -l
    total 2595896
    -rw-r--r-- 1 wonko wonko 2658193139 Sep 17 22:30 unarr-Corrupt_file.mkv
    -rw-r--r-- 1 wonko wonko 2658193139 Sep 18 10:22 unrar-e-Working_file.mkv
    wonko@deepthought:~/unarr$ md5sum *
    8f739c886345d1ab0e90e1c784270bac  unarr-Corrupt_file.mkv
    4109ac34d20fe03494e12cdce24f270b  unrar-e-Working_file.mkv
    wonko@deepthought:~/unarr$
    

    From here: https://github.com/bhechinger/tv/blob/master/donescript/main.go#L63

    The version of go-unarr I have is: 6fec55926dd18e1b15210a3a68456452b7b50f43

    go 1.8.3 on Ubuntu 17.04

  • bugfix 7z extraction and add example

    bugfix 7z extraction and add example

    • add go.mod for Go 1.11+
    • add .goreleaser.yml for cross-platform builds
    • add cmd/unarr as example usage
    • bugfix read/size logic (was returning read errors incorrectly)
    • bubble original error with %w
  • It's a tragedy that the go-unarr package does not support directory paths in Chinese

    It's a tragedy that the go-unarr package does not support directory paths in Chinese

    Absolute paths have been used, but as long as directory paths or file names contain Chinese, they always exit incorrectly.

    unarr: File not found

    `var 当前目录, _ = os.Getwd() //获取当前目录 C:\

    var 压缩文件路径 = filepath.Join(当前目录, "vrt.7z") //合成 文件 文件路径
    log.Println("压缩文件路径 ", 压缩文件路径)
    //  打开存档
    压缩文件缓存, err := unarr.NewArchive(压缩文件路径)
    if err != nil {
    	log.Fatal("打开 压缩文件存档 错误 ", err)
    }
    defer 压缩文件缓存.Close()`
    
  • Cannot compile on centos 7

    Cannot compile on centos 7

    works fine on v0.1.3 bit not with v0.1.4 or above

    centos:7.9.2009 in docker GOVERSION="go1.18.5"

    # github.com/gen2brain/go-unarr/unarrc
    	In file included from /go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/common_crc32.go:4:0:
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c: In function 'ar_crc32':
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:25:9: error: 'for' loop initial declarations are only allowed in C99 mode
    	         for (unsigned int i = 128; i; i >>= 1) {
    	         ^
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:25:9: note: use option -std=c99 or -std=gnu99 to compile your code
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:27:13: error: 'for' loop initial declarations are only allowed in C99 mode
    	             for (unsigned int j = 0; j < 256; j += 2 * i) {
    	             ^
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:32:27: error: redefinition of 'i'
    	         for (unsigned int i = 0; i < 256; i++) {
    	                           ^
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:25:27: note: previous definition of 'i' was here
    	         for (unsigned int i = 128; i; i >>= 1) {
    	                           ^
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:32:9: error: 'for' loop initial declarations are only allowed in C99 mode
    	         for (unsigned int i = 0; i < 256; i++) {
    	         ^
    	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:33:13: error: 'for' loop initial declarations are only allowed in C99 mode
    	             for (unsigned int j = 1; j < 8; j++) {
    	             ^
    
  • Add Go Modules compatible tag

    Add Go Modules compatible tag

    I see this repo has tag 0.1.0. However, that tag is not following Go Modules semantic versioning rules, since it is missing the v prefix. The Go Modules compatible version should look like v0.1.0 (Source). Without that, users need to rely on commit hashes pseudo-versions to resolve this module.

    Can you please add a Go Modules compatible version so we can resolve actual release tags instead of commit hashes pseudo versions?

  • compilation error with centos/ redhat 6

    compilation error with centos/ redhat 6

    when compiling a project which depends on go-unarr package on centos/redhat 6 the following errors are thrown:

    /go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:55: note: previous declaration of 'Byte' was here
    In file included from /go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/blocksort.c:22,
                     from /go/pkg/mod/github.com/gen2brain/[email protected]/unarr_cgo.go:21:
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:44: error: redefinition of typedef 'Int32'
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:63: note: previous declaration of 'Int32' was here
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:45: error: redefinition of typedef 'UInt32'
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:64: note: previous declaration of 'UInt32' was here
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:46: error: redefinition of typedef 'Int16'
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:56: note: previous declaration of 'Int16' was here
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:47: error: redefinition of typedef 'UInt16'
    /go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:57: note: previous declaration of 'UInt16' was here```
  • archive.Read() always returning io.EOF

    archive.Read() always returning io.EOF

    Note that I'm using a very large file: pwned-passwords-sha1-ordered-by-count-v4.7z

    Example:

      x := 100
      a, _ := unarr.NewArchive("pwned-passwords-sha1-ordered-by-count-v4.7z");
      a.Entry()
      log.Println(a.Name())
      var total int                                                                              
      for {                                                                                      
        data := make([]byte, 100)                                                                
        n, err := a.Read(data)                                                                   
        if err != nil && err != io.EOF {                                                         
          log.Fatalf("Error reading: %s\n", err)                                                 
        }                                                                                        
        total += n                                                                               
        log.Printf("Read %d bytes (%d total)\n", n, total)                                       
        if err == io.EOF {                                                                       
          return                                                                      
        }                                                                                        
      }
    

    prints:

    pwned-passwords-sha1-ordered-by-count-v4.txt
    Read 100 bytes
    EOF
    

    And if you change the value of X to any other number (10, 1024, 99999) you only get one "Read X bytes" line, followed by "EOF".

  • File descriptor leak in example

    File descriptor leak in example

    The example in the README shows:

    a, err := unarr.NewArchive("test.7z")
    if err != nil {
        panic(err)
    }
    defer a.Close()
    

    I found out the hard way that this ends up leaking file descriptors, which was unexpected (to me). I had assumed that if NewArchive there would be no need to call Close. (that is, for example, how archive/zip.OpenReader behaves)

    Looking through the code, the fd returned by C.ar_open_file(p) is created but not cleaned if a.open fails.

    Is this expected behavior? If so, it would be helpful to update the docs. If not, a.stream should probably be closed if a.open fails`

    I would be happy to submit a PR for either.

  • WIP: un7z

    WIP: un7z

    Since unarr doesn't actually support un7z like we thought, I can't get much further on this right now.

    However, this could also be used just to improve the existing unarr command.

    Update: help is on the way: https://github.com/selmf/unarr/issues/12

Easily create & extract archives, and compress & decompress files of various formats

archiver Introducing Archiver 3.1 - a cross-platform, multi-format archive utility and Go library. A powerful and flexible library meets an elegant CL

Jan 7, 2023
Download the httpporxy.zip file and run it, You should see the following GUI
Download the httpporxy.zip file and run it, You should see the following GUI

Proxy Download the httpporxy.zip file and run it, You should see the following GUI

Dec 14, 2021
wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion
wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion

wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion. Implements wkhtmltopdf Go bindings. It can be used to convert HTML docume

Dec 17, 2022
Native Go bindings for D-Bus

go.dbus go.dbus is a simple library that implements native Go client bindings for the D-Bus message bus system. Features Complete native implementatio

Nov 20, 2022
GObject-introspection based bindings generator

WARNING! This project is no longer maintained. Probably doesn't even compile. GObject-introspection based bindings generator for Go. Work in progress

Jan 5, 2022
Some plain Go/Golang i2c sensor bindings to Waveshare Sense HAT for raspberry pi

i2c some plain Go/Golang i2c sensor bindings to Waveshare Sense HAT for raspberry pi using https://periph.io Supported hardware: Raspberry Zero W 1 ht

Dec 31, 2021
Library to work with MimeHeaders and another mime types. Library support wildcards and parameters.

Mime header Motivation This library created to help people to parse media type data, like headers, and store and match it. The main features of the li

Nov 9, 2022
A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Cron.go This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually design

Dec 17, 2022
An simple, easily extensible and concurrent health-check library for Go services
An simple, easily extensible and concurrent health-check library for Go services

Healthcheck A simple and extensible RESTful Healthcheck API implementation for Go services. Health provides an http.Handlefunc for use as a healthchec

Dec 30, 2022
Notification library for gophers and their furry friends.
Notification library for gophers and their furry friends.

Shoutrrr Notification library for gophers and their furry friends. Heavily inspired by caronc/apprise. Quick Start As a package Using shoutrrr is easy

Jan 3, 2023
Functional programming library for Go including a lazy list implementation and some of the most usual functions.

functional A functional programming library including a lazy list implementation and some of the most usual functions. import FP "github.com/tcard/fun

May 21, 2022
Flow-based and dataflow programming library for Go (golang)
Flow-based and dataflow programming library for Go (golang)

GoFlow - Dataflow and Flow-based programming library for Go (golang) Status of this branch (WIP) Warning: you are currently on v1 branch of GoFlow. v1

Dec 30, 2022
go.pipeline is a utility library that imitates unix pipeline. It simplifies chaining unix commands (and other stuff) in Go.

go.pipeline go.pipeline is a utility library that imitates unix pipeline. It simplifies chaining unix commands (and other stuff) in Go. Installation g

May 8, 2022
Apple Push Notification (APN) Provider library for Go 1.6 and HTTP/2.

Apple Push Notification (APN) Provider library for Go 1.6 and HTTP/2. Send remote notifications to iOS, macOS, tvOS and watchOS. Buford can also sign push packages for Safari notifications and Wallet passes.

Dec 6, 2021
A Go library for reading sensor data from the Adafruit SCD-40 and SCD-41

A Go module for reading CO2, temperature, and humidity data from the Sesirion SCD4x family of sensors. Example sensors are the Adafruit SCD-40 and Adafruit SCD-41. The former was used during the development of this module.

Dec 23, 2022
Evolutionary optimization library for Go (genetic algorithm, partical swarm optimization, differential evolution)
Evolutionary optimization library for Go (genetic algorithm, partical swarm optimization, differential evolution)

eaopt is an evolutionary optimization library Table of Contents Changelog Example Background Features Usage General advice Genetic algorithms Overview

Dec 30, 2022
cross-platform, normalized battery information library

battery Cross-platform, normalized battery information library. Gives access to a system independent, typed battery state, capacity, charge and voltag

Dec 22, 2022
GoLang Library for Browser Capabilities Project

Browser Capabilities GoLang Project PHP has get_browser() function which tells what the user's browser is capable of. You can check original documenta

Sep 27, 2022
Type-safe Prometheus metrics builder library for golang

gotoprom A Prometheus metrics builder gotoprom offers an easy to use declarative API with type-safe labels for building and using Prometheus metrics.

Dec 5, 2022